Gaining Microsecond Advantage: How Amazon EC2’s Precision Timekeeping Empowers Flow Traders

In the High Frequency Trading (HFT) industry, every microsecond counts. Flow Traders, a leading global liquidity provider, has always depended on precise time measurement to maintain an edge over competitors.

Historically, on-premises infrastructure relied on expensive, specialized hardware timing devices to achieve the nanosecond-level of time crucial for determining network, hardware, and software latencies, as well as fulfilling “Time Traceability for Financial Sector” regulatory obligations under MiFID II.

AWS worked with Flow Traders to improve trading systems on the cloud, which included implementing the improved Amazon Time Sync Service. Flow Traders tested both Intel and AMD Amazon Elastic Compute Cloud (Amazon EC2) instance types, recompiling the Amazon EC2 Elastic Network Adapter (ENA) driver and operating system kernel for Flow Traders’ specific Linux distribution.

In this blog, we demonstrate how the enhanced time synchronization capabilities have enabled Flow Traders to synchronize their trading systems with under 100 microseconds precision, which has resulted in improvements in the performance of their trading applications and MiFID II compliance.

Flow Traders and Amazon Time Sync

Flow Traders initial migration of trading infrastructure to the cloud presented challenges, as cloud environments lacked deterministic packet paths and the typical Network Time Protocol (NTP) services offered significantly lower accuracy than on-premises Precision Time Protocol (PTP) solution. Flow Traders decided to test new time synchronization capabilities, driven by commitment to stay ahead of the competition and the impending MiCAR regulations, which are likely to mandate stringent timestamp accuracy for cloud-based exchanges. By using the Amazon Time Sync Service improved service with the new PTP Hardware Clock on Amazon EC2 instances, Flow Traders stayed ahead of the competition and leverage the improved clocks to optimize workloads on AWS.

The following diagram illustrates an example architecture of market maker connected to exchange infrastructure hosted across the AWS cloud and on-premises:

1. Pricing and trading engine management traffic is communicated between EC2 instances. With improved clock precision, it is now easier to determine one-way and round-trip latency with microsecond accuracy

2. Using VPC Peering, Flow Traders’ instances running in multiple regions can communicate with low latency. Time needs to be measured in microsecond accuracy to ensure the correct order of packets across the availability zones and regions.

3. Flow Traders’ instances communicate with the Exchange REST/WebSocket/FIX APIs running in the respective AZs to make or take liquidity by sending order instructions and receiving market data.

4. Crypto Exchanges process and match orders, and provide trade and orderbook updates.

5. Flow Traders using AWS Direct Connect to connect its on-premise infrastructure to AWS

6. Flow Traders’ on-premise infrastructure is interconnected via its Global Low Latency Network, which is used for propagating trading-related information between regions.

The enhanced Amazon Time Sync Service on Amazon EC2 instances, which provides microsecond-accurate clocks, was a welcomed development by Flow Traders. AWS is the only cloud provider offering timing accuracy comparable to on-premises solutions, making it an ideal choice for low-latency trading firms.

The following graph shows latencies of traffic between two instances, measured on both instances with standard NTP as the source clock. The latency as shown below on the graph is calculated by subtracting the sending time from receiving time. Due to the clock drift of typical NTP services, some values might appear as negative:

In comparison, the following graph is the same setup and measurement, but the difference is that each instance uses the PTP Hardware Clock (PHC) provided by the Amazon Time Sync Service as the source clock. This way consistent, deterministic values can be achieved:

When using NTP-based time sources, Flow Traders saw clock drift between instances, greater than the travel time, which results in measurements of negative latencies. It would be a significant scientific breakthrough if they were true. Travel times in these false measurements range from minus 56 microseconds to plus 169 microseconds, while the true travel time was revealed after Flow Traders changed to Amazon Time Sync Service’s PTP Hardware Clocks, which, in this case, was around a constant 25 microseconds for the 50th percentile.

The Amazon Time Sync PHC is part of the AWS Nitro System and is directly accessible in supported Amazon EC2 Linux instances. Access to the PHC is easy and requires minimal configuration.

By leveraging the microsecond-accurate Amazon Time Sync Service, Flow Traders now execute trades with enhanced precision and improved decision-making capabilities. Such technical improvements drive trading profitability through increased trading activity, system efficiency, and an overall stronger competitive advantage. As a pioneer in the cloud low-latency trading space, Flow Traders is excited to collaborate with AWS to highlight the impact of this transformational new service on business operations and the broader trading industry.

About Flow Traders

Flow Traders is a global trading firm, dedicated to driving transparency and efficiency in financial markets. As a leading liquidity provider, the company continuously delivers stable liquidity to market participants, lowering their overall trading costs and improving execution quality. Flow Traders’ strives to drive efficiency, seamless functioning and accessibility of markets, unlocking the potential for investment for all through innovation.

At the core of Flow Traders’ operations is a deep expertise in exchange-traded products (ETPs), relentless focus on cutting-edge technology and risk management. The firm fosters team-oriented culture and operates trading hubs in Europe, the U.S., and Asia-Pacific, providing global liquidity across all major exchanges and trading platforms 24 hours a day.

The company’s commitment to transparency and the efficiency of financial markets is a key differentiator and is a driving force behind its success as a global leader in liquidity provision.