AWS Cloud Enterprise Strategy Blog

Guest Blog: The Consolidated Audit Trail and the Cloud

As an Enterprise Strategist at Amazon Web Services, I have spent the last 3 years of my professional life talking to leadership teams of financial services institutions (FSI) from across the globe. It’s always of great interest to me that while no two customers are alike, the challenges are near-identical the world over.

From my own time working as a divisional CTO in a large FSI for 17 years, it was always fascinating to me that the terrific responsibility I had of looking after other people’s money was, at first glance, a theoretically simple technical problem: largely moving data securely from one place to another and configuring the interfaces to do so.

Where theory and practice meet in the real world is, alas, not so simple. As one financial services customer recently shared with me, “We are a museum of technology.” The customer wanted to move to AWS to end the perpetual hardware–and in many cases, software–upgrade cycle that added no differentiating business value, just achieving the all-important matter of being “in support” with a hardware vendor that had decided the incumbent current generation of technology was now “obsolete.” And therein lies the challenge: as a customer I was always trying to ensure data could flow where it was needed and to maintain secure and reliable integrations, but with other “museums of technology” that were all juggling their own complexities such as upgrades and patching of data transfer mechanisms. Using AWS changed all that, ushering in the cloud era of financial services.

That historical challenge of getting data securely from one point to another has become significantly simpler as the AWS Cloud has simplified the ways FSIs can securely transfer data among themselves. One component of the equation, AWS PrivateLink, removes additional complexity by eliminating the need for data sharing to traverse the internet. In this post, we will explore one particular data transfer challenge that cloud can help address: the Consolidated Audit Trail (CAT).

John Kain, Worldwide Business and Market Development for Banking and Capital Markets at AWS shares his thoughts on what testing for the CAT means for Capital Markets firms and how industry members are using the cloud to comply.

Twitter | LinkedIn | Blogs


Guest post by John Kain, Worldwide Business and Market Development for Banking and Capital Markets, AWS

Financial institutions are already leveraging the scale and agility of the AWS Cloud to build data lakes, run large scale analytics, and accelerate machine learning adoption. More recently, they are also using services like AWS PrivateLink to connect and transfer data both within their own organization and with other organizations with a presence in the AWS Cloud. AWS PrivateLink uses AWS’s internal network connectivity instead of the public internet to ensure secure connectivity (think of it like a virtual cross connect). Not only are financial data providers such as Bloomberg leveraging AWS PrivateLink to distribute real-time market data to its customers, but financial regulators such as FINRA are also leveraging AWS PrivateLink as a connectivity option for broker-dealers to securely send their order and transaction data to comply with the CAT reporting regulation.

The CAT is expected to be the world’s largest repository of securities data and is expected to ingest upwards of 100 billion market events per day. Unlike existing regulations such as Order Audit Trail System (OATS) and Electronic Blue Sheets (EBS), all broker-dealers are required to report to the CAT. The CAT requires that broker-dealers link customer allocation and trade information for both equities and options transactions. This allows FINRA to track market trading activity by following orders throughout their life-cycle. Due to the stress placed on broker-dealers as a result of COVID-19, the CAT is now expected to go-live for all broker-dealers on May 20, 2020.¹

Architecture of FINRA VPCCurrently, many broker-dealers are working with legacy systems that weren’t built to help them simply and securely share the necessary information to comply with the CAT’s data requirements. However, broker-dealers that are AWS customers have the option to use AWS PrivateLink to transmit data from their Amazon Virtual Private Cloud to FINRA CAT’s Virtual Private Cloud without traversing the public internet.

To set up the AWS PrivateLink connection with FINRA CAT, FINRA provides a cloud formation template (CFT) to the broker-dealer that creates an interface VPC endpoint for the FINRA CAT service in the broker-dealer’s VPC. This CFT takes approximately 20 minutes to run and creates an Elastic Network Interface (ENI) in the broker-dealer’s subnet with a private IP address that serves as an entry point for traffic destined to the FINRA CAT service. And that’s it! Once the CFT is executed, the AWS PrivateLink connection is established, and the broker-dealer can send data securely to FINRA CAT, quickly scaling up resources as needed while controlling cost, and eliminating the need for on-premises equipment.



  1. Please note that the deadline of May 20, 2020 could be extended. Visit the CAT NMS Plan website for more information.


John leads Amazon Web Services’ (AWS) worldwide business and market development efforts for the capital markets segment of the Financial Services industry. In this role, he is responsible for the development and execution of AWS’s strategic initiatives across a wide range of capital markets companies, including broker dealers, hedge funds, market data distributors, and exchange operators. He works with these organizations to help them transform their existing businesses and bring new, innovative solutions to market by leveraging AWS services. John has more than 20 years of experience developing solutions for capital markets companies. Before joining AWS, he led the Global Transaction Surveillance Program for J.P. Morgan’s Investment Bank, where he headed the coordination and execution of strategy, technology delivery, and governance to address the organization’s regulatory mandates. Prior to joining J.P. Morgan, John ran NASDAQ’s high-frequency execution, real-time market risk, and reporting services with responsibility for product strategy, development, and operations. He also gained extensive experience in all aspects of product ownership—including market analysis, business development, technology management, and P&L oversight—working at two venture-backed financial technology companies.



Jonathan Allen

Jonathan Allen

Jonathan joined AWS as an Enterprise Strategist & Evangelist in May 2017. In this role, he works with enterprise technology executives around the globe to share experiences and strategies for how the Cloud can help them increase speed and agility while devoting more of their resources to their customers. Prior to joining AWS, Jonathan was Chief Technology Officer and Senior Director in Capital One Banks UK division. Jonathan was part of the banks Global Technology Leadership team that selected AWS as their Predominant Cloud Partner in 2014, and was accountable for architecting, engineering and execution of the technical build out and system migrations of the banks AWS Cloud strategy in partnership with the US divisions until 2017, by which time the all development had moved Cloud First. Jonathan managed a global team and held all budgetary responsibility for the technology operations and strategy execution, adoption of agile only, technical talent transformation and recruitment and creation of the banks Cloud Governance framework. During Jonathan's 17 years at Capital One he also led large scale transformations including the roll out of regulatory compliance, move from outsourcing to out-tasking, engagement with AWS Cloud Partners, adoption of DevOps at scale and the focus of an engineering led culture. In 2012, he was awarded IT Manager of the Year by The Chartered Institute for IT. He holds a Diploma in Computer Studies from Loughborough College and a CIO MBA from Boston University.

John Kain

John Kain

John leads Worldwide Business & Market Development for Banking and Capital Markets at Amazon Web Services (AWS). He works with customers to help them transform their existing businesses and bring new, innovative solutions to market by leveraging AWS services. John has more than 20 years of experience developing solutions for financial institutions. Prior to joining AWS, he led key programs for J.P. Morgan, Nasdaq, and two venture-backed financial technology companies. John holds a Bachelor of Engineering in Electrical Engineering from the Cooper Union for the Advancement of Science and Art.