AWS News Blog

New – Application Load Balancer Support for End-to-End HTTP/2 and gRPC

Thanks to its efficiency and support for numerous programming languages, gRPC is a popular choice for microservice integrations and client-server communications. gRPC is a high performance remote procedure call (RPC) framework using HTTP/2 for transport and Protocol Buffers to describe the interface.

To make it easier to use gRPC with your applications, Application Load Balancer (ALB) now supports HTTP/2 end-to-end, enabling you to publish gRPC services alongside non-gRPC services via a single load balancer. You can use Amazon Elastic Compute Cloud (EC2) instances or IP addresses (for example with AWS Fargate) as gRPC targets, with support for gRPC health checks for the target groups. In this way, you can use ALBs to terminate, route and load balance the gRPC traffic between your microservices or between gRPC-enabled clients and services.

ALB provides rich content based routing features to inspect the gRPC calls and route them to appropriate services. More specifically, the ALB provides health checks that can examine the gRPC status code, metrics for gRPC request count, access logs that differentiate gRPC requests, and gRPC specific response headers. Additionally, you can benefit from native features like stickiness, different load balancing algorithms, and TLS termination.

How to Use gRPC with Application Load Balancer
To test this new feature, I start by preparing the gRPC server application. I am using the Python programming language and the route_guide demo included in the grpc repo. I use this application because it quickly introduces some of the many ways a client and a server can interact via gRPC, such as:

  • A simple unary remote procedure call (RPC) – the client sends a request to the server and waits for a response, similar to how a function call works.
  • Server-side streaming RPC – the client sends a request to the server, gets back a stream of messages, and reads from the stream until there are no more messages.
  • Client-side streaming RPC – the client writes a sequence of messages, sends them to the server, and then waits for the server to read them all and return its response.
  • Bidirectional streaming RPC – client and server both send messages independently, preserving the order.

First, I prepare a Dockerfile to have the route_guide application run in a container. Technically, this isn’t necessary because I could use a plain EC2 instance, but using containers with gRPC is so common than this makes the example more relevant.

FROM python:3.7
RUN pip install protobuf grpcio
COPY ./grpc/examples/python/route_guide .
CMD python route_guide_server.py
EXPOSE 50051

I build the container image and upload it to Amazon Elastic Container Registry.

aws ecr get-login-password --region eu-north-1 | docker login --username AWS --password-stdin 123412341234.dkr.ecr.eu-north-1.amazonaws.com
docker build -t route-guide .
docker tag route-guide:latest 123412341234.dkr.ecr.eu-north-1.amazonaws.com/route-guide:latest
docker push 123412341234.dkr.ecr.eu-north-1.amazonaws.com/route-guide:latest

In the Amazon ECS console, I create a new cluster using the Networking only template. I name the cluster demo and create a new VPC for this cluster, leaving all other values to their default. The ECS console is using AWS CloudFormation to set up the resources for the cluster. After a few minutes, all resources have been successfully created and the cluster is ready.

To give access to the gRPC traffic, I create a security group to allow inbound TCP traffic on port 50051 from my client laptop. I name this security group gRPC.

Now, I create the load balancer. In the EC2 console, I select Load Balancers on the left, and then Create Load Balancer. In the next step, I create an Application Load Balancer and name it route-guide. I leave the default internet-facing Scheme, because I want to connect to the gRPC service over public internet. In the Listeners section, I select HTTPS as protocol and 50051 for the port. Just below, I select the newly created VPC and the two Availability Zones created by the ECS console.

Next, in the security settings, I choose a certificate I created before, managed by AWS Certificate Manager (ACM).

Then, I select the default security group and the gRPC security group I created above. Each VPC automatically comes with a default security group that gives network access to other resources using the same security group.

In the routing section, I create a  new target group named route-guide. Since I am planning to use AWS Fargate to run the gRPC server, I select the IP address target type. The ALB supports both secure and insecure connections for target groups using the gRPC protocol. Here, I use HTTP (because I am using an insecure connection between the load balancer and the containers running the gRPC server) with port 50051. Then, I select the VPC used by the ECS cluster. For Protocol Version, I use gRPC.

In the Advanced health check settings, I can specify which gRPC Success codes to use when checking for a correct response. I leave the default of 12 which means unimplemented.

Code 12 is returned by a gRPC server if a method is not found. It works in this case because I’m using a path that is not implemented by the route_guide application. More generally, checking for code 12 is a quick way to verify that your gRPC server is running correctly. To be more specific in your health checks, you can use a single code, a list, or a range, depending on what you expect from your implementation.

In the next step, I don’t register any target and complete the creation of the target group.

Back in the ECS console, I create a new task definition compatible with the Fargate launch type. I name it route-guideand give it a minimum amount of resources: 0.5 GB of memory and 0.25 CPU units. I add a container definition using the image URI of the container image I uploaded to ECR, and the container port 50051/tcp, the network port exposed in the Dockerfile above.

With the route-guidetask definition selected, I select Create Service in the Actions menu. I use the Fargate launch type, route-guide as service name, and 2 for the number of tasks. In the next step, I add the two subnets in the VPC, and select the default security group to allow the load balancer to reach the tasks. In the Load balancing section, I select Application Load Balancer and the route-guide load balancer.

For the Container to balance, I select route-guide:50051:50051 and Add to load balancer.

Then, I choose the route-guide target group.

At the next step, I select Do not adjust the service’s desired count to not use auto scaling for this demo.

I complete the creation of the ECS service. After a few minutes the two tasks are RUNNING. Looking at the Targets tab of the route-guide target group, I see that the two targets are healthy. The load balancer is now ready to accept traffic.

In the Amazon Route 53 console, I create a DNS A record as an alias for the load balancer. The domain I use is matching the domain name of the certificate I selected when creating the load balancer.

In my client laptop, I edit the route_client.py file in the gRPC repo to use a secure channel when connecting to the load balancer. This is the part of the code that I change:

    # create credentials
    credentials = grpc.ssl_channel_credentials()

    # create secure channel using credentials
    with grpc.secure_channel('<ALB-ALIAS-DOMAIN>:50051', credentials) as channel:
        stub = route_guide_pb2_grpc.RouteGuideStub(channel)
        print("-------------- GetFeature --------------")
        guide_get_feature(stub)
        print("-------------- ListFeatures --------------")
        guide_list_features(stub)
        print("-------------- RecordRoute --------------")
        guide_record_route(stub)
        print("-------------- RouteChat --------------")
        guide_route_chat(stub)

Now, I start the client to test the gRPC channel with some workload. In the output, I see that the route_guide application is using unary, server-side streaming, client-side streaming, and bi-directional streaming in the request and the response.

$ python3 route_guide_client.py                
-------------- GetFeature --------------
Feature called Berkshire Valley Management Area Trail, Jefferson, NJ, USA at latitude: 409146138
longitude: -746188906

Found no feature at 
-------------- ListFeatures --------------
Looking for features between 40, -75 and 42, -73
Feature called Patriots Path, Mendham, NJ 07945, USA at latitude: 407838351
longitude: -746143763

Feature called 101 New Jersey 10, Whippany, NJ 07981, USA at latitude: 408122808
longitude: -743999179

Feature called U.S. 6, Shohola, PA 18458, USA at latitude: 413628156
longitude: -749015468

Feature called 5 Conners Road, Kingston, NY 12401, USA at latitude: 419999544
longitude: -740371136

Feature called Mid Hudson Psychiatric Center, New Hampton, NY 10958, USA at latitude: 414008389
longitude: -743951297

Feature called 287 Flugertown Road, Livingston Manor, NY 12758, USA at latitude: 419611318
longitude: -746524769

Feature called 4001 Tremley Point Road, Linden, NJ 07036, USA at latitude: 406109563
longitude: -742186778

Feature called 352 South Mountain Road, Wallkill, NY 12589, USA at latitude: 416802456
longitude: -742370183

Feature called Bailey Turn Road, Harriman, NY 10926, USA at latitude: 412950425
longitude: -741077389

Feature called 193-199 Wawayanda Road, Hewitt, NJ 07421, USA at latitude: 412144655
longitude: -743949739

Feature called 406-496 Ward Avenue, Pine Bush, NY 12566, USA at latitude: 415736605
longitude: -742847522

Feature called 162 Merrill Road, Highland Mills, NY 10930, USA at latitude: 413843930
longitude: -740501726

Feature called Clinton Road, West Milford, NJ 07480, USA at latitude: 410873075
longitude: -744459023

Feature called 16 Old Brook Lane, Warwick, NY 10990, USA at latitude: 412346009
longitude: -744026814

Feature called 3 Drake Lane, Pennington, NJ 08534, USA at latitude: 402948455
longitude: -747903913

Feature called 6324 8th Avenue, Brooklyn, NY 11220, USA at latitude: 406337092
longitude: -740122226

Feature called 1 Merck Access Road, Whitehouse Station, NJ 08889, USA at latitude: 406421967
longitude: -747727624

Feature called 78-98 Schalck Road, Narrowsburg, NY 12764, USA at latitude: 416318082
longitude: -749677716

Feature called 282 Lakeview Drive Road, Highland Lake, NY 12743, USA at latitude: 415301720
longitude: -748416257

Feature called 330 Evelyn Avenue, Hamilton Township, NJ 08619, USA at latitude: 402647019
longitude: -747071791

Feature called New York State Reference Route 987E, Southfields, NY 10975, USA at latitude: 412567807
longitude: -741058078

Feature called 103-271 Tempaloni Road, Ellenville, NY 12428, USA at latitude: 416855156
longitude: -744420597

Feature called 1300 Airport Road, North Brunswick Township, NJ 08902, USA at latitude: 404663628
longitude: -744820157

Feature called  at latitude: 407113723
longitude: -749746483

Feature called  at latitude: 402133926
longitude: -743613249

Feature called  at latitude: 400273442
longitude: -741220915

Feature called  at latitude: 411236786
longitude: -744070769

Feature called 211-225 Plains Road, Augusta, NJ 07822, USA at latitude: 411633782
longitude: -746784970

Feature called  at latitude: 415830701
longitude: -742952812

Feature called 165 Pedersen Ridge Road, Milford, PA 18337, USA at latitude: 413447164
longitude: -748712898

Feature called 100-122 Locktown Road, Frenchtown, NJ 08825, USA at latitude: 405047245
longitude: -749800722

Feature called  at latitude: 418858923
longitude: -746156790

Feature called 650-652 Willi Hill Road, Swan Lake, NY 12783, USA at latitude: 417951888
longitude: -748484944

Feature called 26 East 3rd Street, New Providence, NJ 07974, USA at latitude: 407033786
longitude: -743977337

Feature called  at latitude: 417548014
longitude: -740075041

Feature called  at latitude: 410395868
longitude: -744972325

Feature called  at latitude: 404615353
longitude: -745129803

Feature called 611 Lawrence Avenue, Westfield, NJ 07090, USA at latitude: 406589790
longitude: -743560121

Feature called 18 Lannis Avenue, New Windsor, NY 12553, USA at latitude: 414653148
longitude: -740477477

Feature called 82-104 Amherst Avenue, Colonia, NJ 07067, USA at latitude: 405957808
longitude: -743255336

Feature called 170 Seven Lakes Drive, Sloatsburg, NY 10974, USA at latitude: 411733589
longitude: -741648093

Feature called 1270 Lakes Road, Monroe, NY 10950, USA at latitude: 412676291
longitude: -742606606

Feature called 509-535 Alphano Road, Great Meadows, NJ 07838, USA at latitude: 409224445
longitude: -748286738

Feature called 652 Garden Street, Elizabeth, NJ 07202, USA at latitude: 406523420
longitude: -742135517

Feature called 349 Sea Spray Court, Neptune City, NJ 07753, USA at latitude: 401827388
longitude: -740294537

Feature called 13-17 Stanley Street, West Milford, NJ 07480, USA at latitude: 410564152
longitude: -743685054

Feature called 47 Industrial Avenue, Teterboro, NJ 07608, USA at latitude: 408472324
longitude: -740726046

Feature called 5 White Oak Lane, Stony Point, NY 10980, USA at latitude: 412452168
longitude: -740214052

Feature called Berkshire Valley Management Area Trail, Jefferson, NJ, USA at latitude: 409146138
longitude: -746188906

Feature called 1007 Jersey Avenue, New Brunswick, NJ 08901, USA at latitude: 404701380
longitude: -744781745

Feature called 6 East Emerald Isle Drive, Lake Hopatcong, NJ 07849, USA at latitude: 409642566
longitude: -746017679

Feature called 1358-1474 New Jersey 57, Port Murray, NJ 07865, USA at latitude: 408031728
longitude: -748645385

Feature called 367 Prospect Road, Chester, NY 10918, USA at latitude: 413700272
longitude: -742135189

Feature called 10 Simon Lake Drive, Atlantic Highlands, NJ 07716, USA at latitude: 404310607
longitude: -740282632

Feature called 11 Ward Street, Mount Arlington, NJ 07856, USA at latitude: 409319800
longitude: -746201391

Feature called 300-398 Jefferson Avenue, Elizabeth, NJ 07201, USA at latitude: 406685311
longitude: -742108603

Feature called 43 Dreher Road, Roscoe, NY 12776, USA at latitude: 419018117
longitude: -749142781

Feature called Swan Street, Pine Island, NY 10969, USA at latitude: 412856162
longitude: -745148837

Feature called 66 Pleasantview Avenue, Monticello, NY 12701, USA at latitude: 416560744
longitude: -746721964

Feature called  at latitude: 405314270
longitude: -749836354

Feature called  at latitude: 414219548
longitude: -743327440

Feature called 565 Winding Hills Road, Montgomery, NY 12549, USA at latitude: 415534177
longitude: -742900616

Feature called 231 Rocky Run Road, Glen Gardner, NJ 08826, USA at latitude: 406898530
longitude: -749127080

Feature called 100 Mount Pleasant Avenue, Newark, NJ 07104, USA at latitude: 407586880
longitude: -741670168

Feature called 517-521 Huntington Drive, Manchester Township, NJ 08759, USA at latitude: 400106455
longitude: -742870190

Feature called  at latitude: 400066188
longitude: -746793294

Feature called 40 Mountain Road, Napanoch, NY 12458, USA at latitude: 418803880
longitude: -744102673

Feature called  at latitude: 414204288
longitude: -747895140

Feature called  at latitude: 414777405
longitude: -740615601

Feature called 48 North Road, Forestburgh, NY 12777, USA at latitude: 415464475
longitude: -747175374

Feature called  at latitude: 404062378
longitude: -746376177

Feature called  at latitude: 405688272
longitude: -749285130

Feature called  at latitude: 400342070
longitude: -748788996

Feature called  at latitude: 401809022
longitude: -744157964

Feature called 9 Thompson Avenue, Leonardo, NJ 07737, USA at latitude: 404226644
longitude: -740517141

Feature called  at latitude: 410322033
longitude: -747871659

Feature called  at latitude: 407100674
longitude: -747742727

Feature called 213 Bush Road, Stone Ridge, NY 12484, USA at latitude: 418811433
longitude: -741718005

Feature called  at latitude: 415034302
longitude: -743850945

Feature called  at latitude: 411349992
longitude: -743694161

Feature called 1-17 Bergen Court, New Brunswick, NJ 08901, USA at latitude: 404839914
longitude: -744759616

Feature called 35 Oakland Valley Road, Cuddebackville, NY 12729, USA at latitude: 414638017
longitude: -745957854

Feature called  at latitude: 412127800
longitude: -740173578

Feature called  at latitude: 401263460
longitude: -747964303

Feature called  at latitude: 412843391
longitude: -749086026

Feature called  at latitude: 418512773
longitude: -743067823

Feature called 42-102 Main Street, Belford, NJ 07718, USA at latitude: 404318328
longitude: -740835638

Feature called  at latitude: 419020746
longitude: -741172328

Feature called  at latitude: 404080723
longitude: -746119569

Feature called  at latitude: 401012643
longitude: -744035134

Feature called  at latitude: 404306372
longitude: -741079661

Feature called  at latitude: 403966326
longitude: -748519297

Feature called  at latitude: 405002031
longitude: -748407866

Feature called  at latitude: 409532885
longitude: -742200683

Feature called  at latitude: 416851321
longitude: -742674555

Feature called 3387 Richmond Terrace, Staten Island, NY 10303, USA at latitude: 406411633
longitude: -741722051

Feature called 261 Van Sickle Road, Goshen, NY 10924, USA at latitude: 413069058
longitude: -744597778

Feature called  at latitude: 418465462
longitude: -746859398

Feature called  at latitude: 411733222
longitude: -744228360

Feature called 3 Hasta Way, Newton, NJ 07860, USA at latitude: 410248224
longitude: -747127767

-------------- RecordRoute --------------
Visiting point latitude: 400066188
longitude: -746793294

Visiting point latitude: 412452168
longitude: -740214052

Visiting point latitude: 401827388
longitude: -740294537

Visiting point latitude: 414777405
longitude: -740615601

Visiting point latitude: 409642566
longitude: -746017679

Visiting point latitude: 406685311
longitude: -742108603

Visiting point latitude: 406523420
longitude: -742135517

Visiting point latitude: 405047245
longitude: -749800722

Visiting point latitude: 418858923
longitude: -746156790

Visiting point latitude: 409532885
longitude: -742200683

Finished trip with 10 points 
Passed 10 features 
Travelled 863981 meters 
It took 0 seconds 
-------------- RouteChat --------------
Sending First message at 
Sending Second message at longitude: 1

Sending Third message at latitude: 1

Sending Fourth message at 
Sending Fifth message at latitude: 1

Received message First message at 
Received message Third message at latitude: 1

Well, the output is pretty long, but as I said at the beginning of this post, the route-guide demo is showing many of the different ways a client and a server can interact using gRPC, beyond basic RPC invocations.

Available Now
End-to-end HTTP/2 and gRPC Support is available today for new and existing Application Load Balancers in all regions. You can use this feature via the console, AWS Command Line Interface (CLI), AWS SDKs. We are working to add AWS CloudFormation support soon.

There is no additional cost for using the gRPC protocol with an ALB. For more information, you can see the Elastic Load Balancing pricing page.

With these new features, it’s much easier to use gRPC to integrate your applications, or to improve client/server communication. To learn more, please see the documentation.

Danilo

Danilo Poccia

Danilo Poccia

Danilo works with startups and companies of any size to support their innovation. In his role as Chief Evangelist (EMEA) at Amazon Web Services, he leverages his experience to help people bring their ideas to life, focusing on serverless architectures and event-driven programming, and on the technical and business impact of machine learning and edge computing. He is the author of AWS Lambda in Action from Manning.