From 6 Weeks to 30 Minutes: AQR Builds Apps Quickly and Securely on Amazon ECS
AQR Capital Management (AQR), a global alternative investment management firm, uses an in-house platform as a service (PaaS) to run microservices that implement its quantitatively driven investing strategies. AQR wanted to scale its workloads more flexibly and free up developers to build investment solutions. However, with its on-premises infrastructure, AQR’s developers had to focus on everything from upgrading compute capacity to adding security patches. AQR also needed a better way to build security and compliance into its microservices—a necessity in the regulated financial services industry, where security is “job zero.”
Turning to Amazon Web Services (AWS), AQR built a PaaS on Amazon Elastic Container Service (Amazon ECS), a fully managed container orchestration service. Amazon ECS runs on top of Amazon Elastic Compute Cloud (Amazon EC2), a web service that provides resizable compute capacity in the cloud and enables AQR to scale as its workloads change. By using Amazon ECS, AQR developers cut the time for building and launching a new app from 6 weeks to 30 minutes, and automatic security and compliance frameworks have simplified development. “Before, the inflexibility of our infrastructure and the time and amount of work it took for a team to create a new microservice affected how developers designed their systems,” says Adam Batkin, head of cloud services and developer tooling at AQR. “Lowering that barrier to entry to almost nothing gives our people a lot more flexibility in how they design and build their services.”
Managing on-premises infrastructure doesn’t add value to our business and was not something we wanted to keep doing. AWS takes care of that for us.”
Head of Infrastructure, AQR Capital Management
Building Finance Microservices at Scale
Founded in 1998, AQR currently has $140 billion in assets under management* and is using a combination of technology, data, and behavioral finance to guide its investing approach. Central to creating its investment models are business applications that AQR builds in house. Before migrating to Amazon ECS, AQR relied on home-grown on-premises infrastructure that hindered its growing workloads. For its on-premises PaaS, it used an Apache Mesos distributed cloud operating system, which required hardware and infrastructure that were difficult to scale. This system lacked the flexibility AQR needed to build new apps, and the company’s infrastructure teams also had to support all hardware, software, and other components.
AQR decided that AWS, as its go-to cloud provider, was the right fit for its PaaS migration and cloud strategy. Amazon ECS offered solutions AQR needed, including compatibility alongside other AWS services and solutions, such as AWS Identity and Access Management (AWS IAM) roles, automatic scaling and load balancing, managing Secure Sockets Layer / Transport Layer Security certificates through AWS Certificate Manager, and logging, metrics, and alerting on Amazon CloudWatch, a monitoring and observability service. Plus, AQR’s development team already had experience using Amazon ECS. “Managing on-premises infrastructure doesn’t add value to our business and was not something we wanted to keep doing,” says Michael Raposa, head of infrastructure at AQR. “AWS takes care of that for us.”
Achieving Fast, Secure Application Development on AWS
AQR commenced its migration to Amazon ECS in 2018. Progressing from the initial concept to the first version of its PaaS required about 3 months, after which developers immediately started iterating new microservices in a fraction of the time previously spent. To achieve this, AQR designed a complete process that starts with requesting a new service. It uses AWS Service Catalog, which enables organizations to create and manage catalogs of information technology services approved for use on AWS. AWS Service Catalog and AWS CloudFormation, a service offering a simple way to model a collection of related AWS and third-party resources, work together to create a solution for modeling and provisioning AWS and third-party resources. A prepopulated template app—based on information from an onboarding form and customized with the selected programming language—builds the deployment plans and provisions service accounts in Active Directory. The entire process, from submitting the form to having an application running in production, takes just 30 minutes.
The system supports hundreds of AQR developers across six business units, with virtually no limit to the number of deployments it can support per day. Development teams can iterate very rapidly, with code changes going from commit to production in under 10 minutes. Already, AQR has deployed nearly 700 microservices across multiple development and test environments, with over 180 microservices in production. Before migrating to AWS, AQR needed to prioritize workloads because of the limitations of its on-premises solution, requiring difficult choices for management. Today, AQR has scaled its application provisioning by more than 20 times, with capacity to go far beyond that. Developers’ productivity has increased now that they no longer have to worry about infrastructure management, and teams across AQR—including research, portfolio implementation, trading, operations, business development, enterprise engineering, and security—have started developing for the first time on AWS. “We have people who had never developed a web application building PaaS and web apps,” says Raposa. “It’s very unusual to see something like that.”
In the past, AQR’s developers devoted significant time and resources to security and compliance. “They’re here to develop research solutions and portfolio optimization systems and to develop our trading models. They’re not here to set up Secure Sockets Layer certificates and Domain Name System entries,” says Raposa. “In our old system, that’s exactly what they were doing.” To bootstrap the security process, the company extended its existing Kerberos and OAuth authentication and authorization infrastructure into Amazon ECS. To accomplish this, the PaaS uses two Docker sidecar containers deployed alongside each application container. One sidecar runs NGINX and operates on the front end of the application, handling inbound authentication, Secure Sockets Layer termination, and certificates. The second sidecar enables applications to use Kerberos to communicate with other AQR systems without storing credentials in code.
Accelerating AQR’s Journey to the Cloud
Migrating to Amazon ECS has enabled AQR to move its business-critical development processes to the cloud and empower teams across the organization to iterate rapidly and securely. The company plans to continue with its lift-and-shift cloud migration, with the goal of reducing its on-premises footprint by 80 percent, along with expanding its portfolio of AWS services.
“On AWS we can deploy apps faster, make updates faster, and do more for our business. Our team members have access to virtually unlimited compute power to test and experiment faster as part of their research and discovery process,” says Raposa. “All those things that we didn’t have on premises are now in play using AWS.”
*Assets under management are approximate as of March 31, 2021, and include assets managed by AQR and its advisory affiliates.
About AQR Capital Management
AQR Capital Management is a global quantitative investment management firm based in Greenwich, Connecticut. Founded in 1998, the company offers clients diversified strategies across equity, fixed income, and alternatives.
Benefits of AWS
- Reduced application provisioning time from 6 weeks to 30 minutes
- Gained the ability to iterate rapidly, making code changes in as few as 10 minutes
- Deployed nearly 700 microservices quickly and securely, with another 180+ in production
- Scaled investment application provisioning capability by 20x
- Saved time by developing automatic security and compliance frameworks
- Empowered teams across the organization to iterate microservices
- Achieved elastic workload scaling
- Reduced the burden of on-premises infrastructure management
AWS Services Used
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers.
Amazon Elastic Container Service (Amazon ECS) is a fully-managed container orchestration service from AWS that is highly secure, reliable, and scalable.
AWS Service Catalog
AWS Service Catalog allows organizations to create and manage catalogs of IT services that are approved for use on AWS.
AWS CloudFormation provides a common language for you to describe and provision all the infrastructure resources in your cloud environment.
Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.