Networking & Content Delivery

Real-time communication at CrazyCall using AWS Global Accelerator

In the telecommunications industry, real-time communication (RTC) refers to live media sessions between two endpoints with minimum latency and jitter. These sessions could be for voice, instant messaging or live video.

Each of these solutions consists of one or more signaling message exchanges that control the call (e.g., authentication, authorization and access control, transcoding, or buffering) and one or more media streams carrying audio, video, or other data. Together, these streams make up a session, which is placed inside internet protocol (IP) packets.

For non-real-time applications like downloading a file or accessing a browser, delays are inconvenient but do not damage the actual outcome. Real-time applications, in contrast, are time sensitive. Delays that might have been acceptable for non-real-time applications can degrade performance and the user experience to an extent where the application is unusable. For example, imagine a telephone conversation where you have to wait a second or two before receiving answer to statements or questions. Clearly that would not be acceptable.

Real-time applications perform best when the underlying internet connection reduces latency and jitter. Reliable and consistent network performance helps ensure real-time data inside IP packets are delivered quickly and in right order. AWS Global Accelerator is a service that helps reduce network latency and jitter.

Global Accelerator improves the availability and performance of applications for local or global users. It provides static IP addresses that act as a fixed entry points for your applications in one or multiple AWS Regions. These fixed entry points include Application Load Balancers, Network Load Balancers, or Amazon Elastic Compute Cloud (Amazon EC2) instances.

Global Accelerator uses the AWS global network to optimize the path from your users to your applications, improving the performance of your TCP and UDP traffic. Global Accelerator also continually monitors the health of your application endpoints, and detects an unhealthy endpoint and redirects traffic to healthy endpoints in less than one minute.

In this blog post I describe how CrazyCall, a cloud-based calling software company, improves performance for their real-time application by using Global Accelerator.

CrazyCall’s cloud-based calling software allows their customers to easily build a reliable helpline, quickly contact customers, and generate leads directly from websites. CrazyCall serves companies of all types, focusing on small-to-medium businesses who want to use the power of phone system and reach their audience worldwide. CrazyCall strives in providing a high-quality experience to all of their customers around the globe.

CrazyCall’s software is based on Kamailio and FreeSWITCH. Kamailio is an open source Session Initiation Protocol (SIP) server released under GPL, able to handle thousands of call setups per second. FreeSWITCH is an open source communication platform that allows to connect to customers and can run on any modern platform.

Solution:

CrazyCall operates a multi-region network. The setup is deployed in two AWS Regions: eu-central-1 and eu-west-1. It is a distributed environment with Kamailio running on Amazon Elastic Container Service (Amazon ECS), and RTPEngine and FreeSWITCH running on Amazon EC2 instances.

  • CrazyCall uses Kamailio as session border controllers (SBC) in front of their FreeSWITCH servers. This allows CrazyCall’s implementation to act as a web proxy plus UDP gateway and also provide all the features of SBC.
  • WebRTC SIP client connects via HTTPS WebSocket to Kamailio.
  • Kamailio does the sanity and security checks of incoming signaling traffic and routes signaling from user’s browser/widget/mobile app directly to their FreeSWITCH server farm.
  • FreeSWITCH act as a media processor and allows CrazyCall to provide various features to their customers like call recording, IVR, and music on hold.

Traffic flow before integrating with Global Accelerator:

Figure 1: CrazyCall without Global Accelerator

Before integrating with Global Accelerator, CrazyCall was using Amazon Route 53’s latency-based routing, Elastic Load Balancing (ELB), and HTTP latency probing to find the least congested link from user to an AWS Region:

  1. User sends a request to sip.example.com.
  2. Amazon Route 53, based on latency-based routing, forwards traffic to ELB in the appropriate AWS Regions.
  3. ELB forwards to Kamailio (SIP Proxy) and RTPEngine.
  4. Kamailio forwards it to FreeSWITCH.

Challenges with the existing solution:

For their real-time application to perform optimally, CrazyCall required a reliable internet connection with consistent network performance. One way to reduce latency and jitter is to reduce the number of hops between the user and the application. It is important to note that points mentioned later in this post are inherent to all DNS-based routing techniques and are not specific to Amazon Route 53.

  • With latency-based routing, DNS systems like Amazon Route 53, based on user’s source IP, can direct users to the lowest-latency CrazyCall endpoints. DNS servers decide, based on recently observed network conditions, which endpoint in which Regions should serve particular users. A user in London will likely be directed to an endpoint in eu-west-1 and a user in Germany will likely be directed to an endpoint in eu-central-1.
  • Latency-based routing is based on the approximate latency between the user and the AWS Region and not the user and the hosted service. Also, a DNS service like Amazon Route 53, receives the user’s internet service provider (ISP) DNS resolver IP address or, if the resolver supports edns-client-subnet extension for EDNS0, receives a truncated version of the user’s IP address. What one really wants is the latency between the user and the endpoint that they are trying to connect.
  • Latency between two hosts on the internet is dependent on the network path and number of hops between hosts. Also, it can change over time as a result of changes in network connectivity and routing.
  • User traffic will travel most of the way to the CrazyCall’s endpoint over the public internet and their internet transit provider (ITP) before entering the AWS global network. This could be halfway across the world. As a result, quality depends on the ISP transit path traversed by the packet as depicted in Figure 2: Network Path

Figure 2: Network Path

  • Domain name is being cached for a predefined time to live (TTL) at each step of the DNS resolution process. As users connect through different networks and DNS servers, finding the right threshold for TTL is difficult. DNS caching can help speed up the delivery, but in an event of network changes, it can also slow down switching to another available endpoint as you have to wait for DNS cache to expire. Hence, CrazyCall didn’t want to rely on TTL of DNS cache in the client device. All of these factors lead to managing route configurations, DNS configurations, network and/or operational complexities.
  • IP addresses for the DNS lookup to CrazyCall domain name can change. Users might not be whitelisting a range of IP addresses. In such situations the change in IP address can make it difficult for users to whitelist external resources based on IP addresses.
  • An internet-facing Application Load Balancer also increases your exposure to attacks from the internet.

To address these challenges, CrazyCall began exploring Anycast routing. Anycast routing is a network addressing and routing rule in which a single destination address has multiple routing paths to two or more endpoint destinations. That’s when they learned Global Accelerator uses Anycast routing.

How Global Accelerator helps CrazyCall:

  • Global Accelerator provides static IP addresses that serves as a fixed entry point to CrayCall’s real-time application hosted in any number of AWS Regions.
  • These static IP addresses are anycast from the AWS edge locations. They distribute incoming application traffic across multiple endpoints in one or more AWS Regions, which increases the availability of CrazyCall’s application.
  • Global Accelerator’s IP addresses are static and serve as the front-end interface to their application. Global Accelerator quickly detects outage of endpoints and directs the IP traffic to an available AWS Region. The traffic is directed over the optimized route to the new location versus continued using of the original route to the original location. It helped CrazyCall to realize the performance gain without dealing with network and/or operational complexities.
  • Global Accelerator allowed CrazyCall to add an internal Application Load Balancer as an endpoint. By using Global Accelerator as the single internet-facing access point, CrazCall is able to protect their application running on AWS from distributed denial of service (DDoS) attacks thanks to AWS Shield.

Traffic flow after integrating with Global Accelerator:

CrazyCall integrated their implementation with Global Accelerator and Amazon Route 53 to achieve improved network performance. Global Accelerator directs user traffic to the nearest application endpoint, thus reducing internet latency and jitter. VoIP traffic from the user is routed to the closest edge location via Anycast. Traffic is then routed to the closest AWS Region hosting CrayzCall’s environment over the AWS global network.

Figure 3: CrazyCall with Global Accelerator

As described in Figure 3, CrazyCall:

  1. Uses the Global Accelerator anycast IP.
  2. Uses Amazon Route 53 to create a CNAME record for sip.example.com referencing the Global Accelerator DNS hostname.
  3. Configures an internal ALB as the Global Accelerator endpoint.
  4. User sends the requests to sip.example.com.
  5. Using the Global Accelerator anycast IP, traffic enters the AWS global network through the edge location closest to the user.
  6. Within the edge location, traffic is forwarded to Global Accelerator.
  7. Global Accelerator forwards VoIP traffic to an ELB in the appropriate Region.
  8. The ALB forwards VoIP traffic to Kamailio (SIP Proxy) and rtpengine.
  9. Kamailio forwards VoIP traffic to FreeSWITCH.

Conclusion:

With Global Accelerator and Amazon Route 53, CrazyCall was able to use static anycast IP addresses within minutes. Global Accelerator allowed user traffic to ingress into the AWS global network closest to the user. Global Accelerator enabled CrazyCall to implement their software with low latency and improved performance within days. Without Global Accelerator, it would have taken months to deploy anycast services. Also, the solution would have been complex and expensive to maintain and deploy for CrazyCall’s scale. Since integrating with Global Accelerator, CrazyCall has noticed a reduction in support calls from customers asking for help with performance and quality.

“We use Global Accelerator’s static anycast IPs and global network to handle Voice over IP (VoIP) traffic, thus ensuring that our customers get the best quality of service.”

—Marcin Kowalczyk, AWS Architect, CrazyCall

 

Blog: Using AWS Client VPN to securely access AWS and on-premises resources
Learn about AWS VPN services
Watch re:Invent 2019: Connectivity to AWS and hybrid AWS network architectures