How TCS unlocked Open Banking with Amazon Kinesis Data Streams

By Sumitha M, Lead Streaming Solution – Data and Analytics Practice – AI.Cloud
By Ramesh Srinivasan, Head – Data and Analytics Practice – AI.Cloud
By Hari Ramesh, Sr Partner Solutions Architect – AWS
By Tony Vargas, Partner Solutions Architect – AWS

TCS

Australia’s Consumer Data Right (CDR) requires banks to give customers more control over their financial data through open banking. For financial institutions, this means securely sharing customer information with authorized third parties while maintaining reliable banking operations.

A leading Australian bank partnered with Tata Consultancy Services (TCS) to solve this challenge. With millions of daily transactions flowing through their nationwide network of branches, call centers, and subsidiaries, they needed a solution that wouldn’t disrupt their critical banking services, including home loans, savings accounts, and credit cards.

In this post, you’ll learn how TCS designed a practical approach using Amazon Kinesis Data Streams and other AWS services. The solution replicates data from multiple systems in near real-time, enabling open banking for over 400,000 customers without modifying the bank’s core infrastructure. You’ll discover how this architecture processes over 6 million records daily while positioning the bank for future growth in the open banking ecosystem.

The Challenge

The bank’s customers depend on continuous access to services where disruptions would affect their experience and erode trust. Beyond customer impact, mishandling of the data could compromise CDR requirements and regulatory reporting. The migration of such core banking systems traditionally involves significant risks: service disruptions, complex data migrations, and potential impact to operations. These challenges are amplified by the need to maintain transaction processing across multiple banking channels, particularly ensuring end-to-end security while streaming and processing sensitive financial data across hybrid environments.

Solution Overview

TCS worked closely with the customer to understand their business and technical requirements. TCS proposed a real-time replication approach using AWS services instead of a full migration. This strategy eliminated the risks of modifying core banking systems while enabling modern cloud capabilities. The replication system would capture changes from existing systems and stream them to AWS, maintaining business continuity while building new capabilities. Security was paramount, requiring secure storage, data encryption, and comprehensive audit trails for all financial data. By focusing on data replication rather than system migration, the bank could enhance their capabilities while protecting core business operations. This approach positioned the bank to meet immediate regulatory requirements while creating a foundation for future innovation in the open banking ecosystem.

Through a strategic collaboration, TCS successfully implemented a comprehensive banking integration solution that addressed multiple critical objectives. The initiative focused on establishing a robust real-time data replication system, enabling seamless transaction synchronization from various systems to AWS with near-instantaneous processing.

A cornerstone of the implementation was achieving exceptional reliability, with the system maintaining an impressive 99.99% uptime for all banking integrations. This was complemented by advanced scalability features that dynamically adjusted resources to accommodate fluctuating transaction volumes during peak periods. Security remained paramount throughout the implementation, with stringent data protection measures and compliance protocols safeguarding all banking operations.

The solution also incorporated sophisticated monitoring capabilities, ensuring rapid detection of any data pipeline failures and enabling stakeholder notifications within a 5-minute window. Perhaps most significantly, the system successfully served over 400,000 customers, providing them with real-time access to their transaction data. This comprehensive approach demonstrated TCS’s ability to deliver a solution that balanced technical excellence with practical business requirements.

Solution Architecture

The architecture of this banking integration solution embodies a sophisticated and efficient approach to handling financial transactions at scale. At its core, the design is modular, allowing for seamless integration with various banking systems. This flexibility ensures that the solution can adapt to different financial institutions’ unique requirements and existing infrastructure.

To accommodate the ever-increasing volume of transactions in the banking sector, the architecture incorporates an auto-scaling streaming platform. This dynamic component adjusts its capacity in real-time, ensuring optimal performance during peak periods and cost-effectiveness during quieter times.

Security and compliance are paramount in financial systems, and this architecture reflects that priority. Every layer of the system is fortified with robust security and compliance controls, safeguarding sensitive financial data and ensuring adherence to regulatory requirements.

Operational efficiency is achieved through the extensive use of managed services. This approach minimizes the need for manual intervention, reducing the potential for human error and allowing the financial institution’s IT team to focus on strategic initiatives rather than routine maintenance.

The architecture also prioritizes cost efficiency. By leveraging managed services and implementing usage-based pricing models, the system optimizes resource utilization and aligns costs directly with actual usage, providing a more predictable and manageable cost structure for the financial institution.

Figure 1 below illustrates this architecture, showcasing the interconnected components and their roles in creating a robust, secure, and efficient banking integration solution.

Figure 1: Solution Architecture for enterprise data platform on AWS

Solution Walk Through:

1. Data Source: The core banking system managing savings accounts, home loans, and credit card transactions resides on-premises using IBM DB2 with Change Data Capture (CDC) enabled. The channel database runs on Amazon RDS for SQL Server, also configured with CDC to track database changes.
2. Data Ingestion: Oracle GoldenGate and AWS Database Migration Service (DMS) leverage the CDC streams to replicate data from IBM DB2 and Amazon RDS for SQL Server respectively. This enables reliable delivery of critical financial data by preserving transaction integrity.
3. Real-time Streaming: Amazon Kinesis Data Streams processes financial transactions in real time using on-demand capacity mode. This eliminates manual provisioning while automatically scaling to match transaction volumes. The service efficiently processes 200 transactions per second at peak, with the flexibility to scale well beyond this to support business growth.
4. Data Persistence: Financial transactions flow from Amazon Kinesis Data Streams through Amazon Data Firehose for reliable storage. Data Firehose automatically writes data to Amazon S3, ensuring 99.999999999% data durability for all banking transactions. The architecture ensures complete transaction audit trails for compliance requirements while maintaining real-time processing performance.
5. Data Transformation: AWS Glue processes the streaming financial data in real time. This fully managed service prepares transaction data for authorized open banking consumers before loading it into Amazon Aurora MySQL.
6. Open Banking Database: Amazon Aurora powers the bank’s open banking platform, providing reliable data access for authorized consumers. This highly available service delivers 99.99% uptime and automatically scales to handle peak workloads across 15 read replicas. Considering the transaction patterns, a provisioned cluster balanced performance requirements with cost efficiency for sustained data volumes.
7. Security and Monitoring: Security is implemented through defense-in-depth controls including network isolation and security groups. Data is encrypted at rest using AWS Key Management Service (KMS) and in transit using Transport Layer Security (TLS). AWS Identity and Access Management (IAM) service roles enforce least-privilege policies, and AWS Secrets Manager securely manages database credentials. Amazon CloudWatch provides real-time visibility into data pipelines, while Amazon Simple Notification Service (SNS) enables rapid response to operational events. Together, these controls ensure secure, role-based access control and operational reliability across the open banking platform.

Solution Outcome

Working backwards from customer needs, TCS helped the bank successfully launch open banking services for 400,000+ customers while maintaining existing operations and meeting CDR requirements. The solution delivers large-scale open banking performance, processing over 6 million records daily. Automated monitoring and managed services reduce operational overhead while enabling comprehensive compliance controls and audit capabilities. The bank can now quickly onboard new open banking partners, seamlessly scale transaction processing, and rapidly innovate on new digital services.

Conclusion

The Enterprise Data Platform implementation demonstrates how TCS used Amazon Kinesis Data Streams to help an Australian bank meet their open banking objectives while protecting core operations. The pattern established here shows how financial institutions can modernize their data streaming capabilities for open banking and future digital initiatives. AWS recently announced that Amazon Kinesis Data Streams On-Demand mode now supports writing up to 10 GB/s, enabling similar implementations to scale further for growing open banking demands.

For more information on Amazon Kinesis Data Streams services offered by TCS, please refer to the following link how TCS use Amazon Kinesis Data Streams for efficient data streaming

.

.

TCS – AWS Partner Spotlight

TCS is an AWS Advanced Technology Partner and AWS Competency Partner that provides IT services, consulting, and business solutions, headquartered in India with operations in 55 countries, known for providing digital transformation and technology services to enterprises across various industries.

Contact TCS | Partner Overview | AWS Marketplace