Building a high-performance exchange market data broadcasting platform on AWS

This is a joint post co-authored with Abhishek Chawla, Chief Product and Technology Officer; Kartik Manimuthu, Director of Cloud Engineering; and Digvijay, Director of Application Engineering at SMC Global Securities Ltd.

SMC Global Securities Ltd. (SMC), established in 1990, is a leading Indian financial services company providing trading, wealth advisory, and financial product distribution services to retail investors, corporations, and high-net-worth individuals.

SMC Global Securities (SMC) partnered with Amazon Web Services (AWS) to modernize their retail trading infrastructure. As part of this initiative, SMC built a robust financial data distribution system on AWS that efficiently processes real-time market feeds from major Indian exchanges (NSE, BSE, MCX, and NCDEX). The platform uses multicast networking technology to directly capture and broadcast market price data with high performance and scalability.

Numerous AWS users operate core financial systems on the cloud, typically processing unicast data from their on-premises infrastructure. This often involves converting multicast feeds to unicast before transmitting them to AWS through AWS Direct Connect. However, directly ingesting external multicast data on AWS has posed challenges for many organizations. This post explores SMC’s innovative approach to overcoming these obstacles. We demonstrate how SMC used the AWS Transit Gateway built-in multicast support for communicating within Amazon Virtual Private Cloud (Amazon VPC) environments, combined with Fortinet SD-WAN for overlay solution. This setup enabled SMC to establish Generic Routing Encapsulation (GRE) tunnels over Direct Connect, successfully bringing multicast price feeds from their on-premises systems directly into the AWS Cloud.

AWS Transit Gateway overview

AWS Transit Gateway is a network transit hub that streamlines connectivity by interconnecting multiple VPCs and on-premises networks in a hub and spoke model. It supports various attachment types, such as VPCs, SD-WAN appliances, Direct Connect gateways, VPN connections, and other Transit Gateways. This eliminates the need for complex point-to-point connections. A key capability is its native support for multicast communication, where it automatically handles the complexities of multicast routing for instances sending traffic destined for multiple receiving instances part of same multicast group.

The challenge

SMC began their AWS journey with the core philosophy of providing a “fast and seamless trading experience” for their end users. Having a performant broadcasting platform is key to that experience. This is because the broadcasting platform receives events from stock exchanges and sends them to traders, making it a critical component of modern retail trading applications.

SMC needed a system that could provide near real-time financial market data to traders with zero tolerance for packet loss, because even a single missed trade packet could display incorrect critical market information (such as 52-week highs). This could potentially cause significant financial losses for traders and undermine client trust. SMC’s existing on-premises broadcasting infrastructure faced scalability challenges and could not keep pace with the demanding performance requirements of their modern financial trading applications. This caused frequent downtime and degraded performance of their critical infrastructure.

Solution overview

Organizations need hybrid network connectivity to seamlessly connect their on-premises data centers, remote locations, and cloud infrastructure. AWS provides several connectivity options for hybrid architectures, such as Direct Connect and AWS Site-to-Site VPN, to securely connect on-premises resources with AWS.

SMC is in a highly regulated financial business domain, thus when they started designing their hybrid financial broadcast platform on AWS Cloud, they had four primary objectives:

• Scalability
• Reliability
• Performance
• Security

To meet these objectives at scale, SMC adopted AWS Control Tower to set up their workloads on AWS. AWS Control Tower offers a clear and efficient way to set up and govern a secure, multi-account AWS environment. It establishes a landing zone based on best-practices blueprints and provides governance using controls from a prepackaged list. The landing zone is a well-architected, multi-account baseline that follows AWS best practices.

Design considerations

To meet compliance requirements from traditional exchanges that mandate market data price feed network connection to terminate at on-premises locations, SMC evaluated two architecture options:

1. Terminate the exchange multicast price feeds in the data center, convert them to unicast, and rehost the existing broadcasting solution in AWS. Although this approach would be quicker and necessitate less effort, SMC would need to carry the technical debt burden of the existing solution’s inefficiencies in the cloud.
2. Terminate exchange multicast price feed in the data center and directly send the multicast feed to AWS. This allowed re-architecting of the broadcasting solution using native AWS services.

After careful evaluation, SMC decided to completely re-architect their system on AWS. This strategic decision was driven by two key objectives:

1. Building a future-ready architecture
2. Maximizing the benefits using native AWS services

To achieve these objectives, certain core principles were adopted for the platform design:

Prioritize event freshness over completeness

For performance and real-time relevance, the system should discard stale events. For example, if a security XYZ’s Last Traded Price (LTP) was 100, 101, and 102 at times T1, T2, and T3 respectively, then the system can discard prices 100 and 101 to directly show price 102. Users are primarily interested in current prices rather than historical stale prices when checking the LTP.

Figure 1 – Last traded price

However, historical prices are critical for analyzing trends such as 12-week highs/lows, 52-week highs/lows, or viewing aggregated order lists.

Figure 2 – Price list high-low

Therefore, the solution needed to have a near real-time view of all the latest price data and a desirable mechanism to reconcile the discarded/lost data packets through an alternate path. To address this, a real-time recording and reconciliation system was developed and deployed as independent and scalable microservices on an Amazon Elastic Container Service (Amazon ECS) cluster by using AWS native services for message processing:

• Amazon ElastiCache for caching and reflecting the latest recorded data packets for faster access
• Amazon RDS PostgreSQL for long-term data storage and batch reconciliation

Figure 3 – Price recording and reconciliation

Multi-threaded processing for high throughput

Due to the high volume of market data, multi-threading processing was adopted across all feasible component paths. To further minimize data packet loss over network between nodes due to instance specific network bandwidth limits, multiple TCP connections at different ports on each instance are established as an optimization strategy to send data stream in parallel.

One challenge with this multi-threaded approach was maintaining correct order processing. For example, if security XYZ updated twice in a second—M1 (price changed to 100) and M2 (price changed to 101)—then processing M2 before M1 in a multithreaded environment could be catastrophic. To handle ordering issues, a custom chronological component was developed and placed in the path before data is surfaced in the user view.

Figure 4 – Chronological data ordering

Managing WebSocket connections at scale

WebSocket technology enables real-time, bi-directional communication between clients and servers, making it ideal for broadcasting market data to web and mobile interfaces. Unlike traditional HTTP requests, WebSocket connections:

• Maintain persistent connections, reducing latency
• Minimize network overhead by eliminating repeated handshakes
• Enable server-push updates, critical for real-time market data delivery
• Reduce server load when compared to polling-based approaches

SMC implemented WebSocket connections running on ECS clusters to use these benefits on their web and mobile application interfaces. However, operating WebSocket-based systems at scale presents unique challenges:

1. Connection management challenges

When established, WebSocket clients maintain persistent connections to specific server instances. This creates complexity for traditional load balancing approaches, especially during Amazon ECS scaling events, because existing connections cannot be redistributed easily to new instances due to their stateful nature.

2. WebSocket thundering herd challenges

During market hours, thousands of traders simultaneously attempt to establish WebSocket connections, particularly at market opening or after system maintenance. This “thundering herd” effect can overwhelm servers and lead to connection failures. Furthermore, when containers are replaced during scaling events, multiple clients attempting to reconnect simultaneously can create similar stress on the system.

To address these challenges, a comprehensive connection management strategy was implemented. This included connection-based scaling through custom Amazon CloudWatch alarms and staggered client reconnections using exponential backoff and jitter. For long-term scalability, the team also evaluated alternative approaches such as external state management using ElastiCache to store connection states for seamless client transitions between containers, and progressive scaling with controlled intervals to prevent connection surges.

Production resilience and observability

To make sure of broadcasting system reliability, SMC implemented a custom tooling using an LGTM stack for comprehensive monitoring and observability that combines four key tools: Loki for logs, Grafana for visualization, Tempo for tracing, and Mimir for metrics. This is integrated with a centralized incident notification and management system using AWS Systems Manager Incident Manager to quickly mitigate and recover from incidents affecting applications hosted on AWS.

Figure 5 – Integrated incident manager

Solution architecture

The following high-level reference architecture shows how SMC deployed their broadcasting platform. AWS Transit Gateway plays a crucial role in this architecture by serving as a cloud router that streamlines network topology, while enabling multicast routing across multiple VPCs. It centralizes connectivity between VPCs and on-premises networks, all while supporting high-bandwidth, low-latency connections.

Use of AWS Transit Gateway allowed SMC to scale their network as they add new VPCs or on-premises locations, implement consistent network policies across all attached networks, and reduce the number of connection points needed to connect multiple VPCs and on-premises networks.

SMC used Fortinet Firewall to ingest multicast data over GRE tunnels within an Amazon VPC. They integrated their FortiGate SD-WAN Hub with AWS Transit Gateway to establish secure, high-performance connectivity between on-premises and AWS environments, enable efficient distribution of multicast market data feeds across multiple VPCs, streamline management of complex network architectures, and reduce operational overhead in managing multi-VPC networking.

How to configure FortiGate SD-WAN is beyond the discussion of this post, but full details can be checked here.

Figure 6 – Broadcasting system architecture

Solution Outcome

SMC’s newly designed system in AWS means that their broadcasting system can now handle tens of thousands of concurrent users. It can process 100K+ messages per second from multiple stock exchanges with zero downtime. AWS allows SMC to scale up their platform to meet increased business demand during peak business hours, resulting in better performance and lower latency (approximately 60% improvement) and seamlessly scale down after hours to save cost.

Figure 7 – System latency metrics

Conclusion

SMC’s journey from on-premises to a cloud-native broadcasting platform exemplifies how financial institutions can overcome traditional infrastructure constraints through strategic AWS adoption. The transformation needed bold architectural decisions—choosing to re-architect rather than migrate—and proves that with careful planning, appropriate AWS services, and a commitment to modernization, financial services companies can build infrastructure that meets today’s demanding requirements while scaling for tomorrow’s opportunities.

At SMC Global Securities (and Stoxkart), our digital transformation journey with AWS has significantly enhanced platform stability, performance, and operational agility. Migrating from on-premises to cloud has eliminated the need for manual scaling, patching, and infrastructure upgrades—freeing our teams to focus on innovation. With improved control, visibility, and the ability to auto-scale during peak trading hours, we’ve built a resilient, future-ready broking ecosystem that delivers a seamless experience to our customers.

– Abhishek Chawla (SMC , CTO)

About the authors