AWS Business Intelligence Blog

How PayNet enhanced payment analytics with Amazon QuickSight

This post was written with Jin Quan Goh, Nur Sabrina Binti Md Zaid, and Tzy Wei Ling from Payments Network Malaysia Sdn Bhd (PayNet).

Payments Network Malaysia Sdn Bhd (PayNet), Malaysia’s national payments network and shared central infrastructure provider, plays a vital role in Malaysia’s financial ecosystem, providing critical payment infrastructure that processes transaction values equivalent to 3.4 times Malaysia’s gross domestic product (GDP) as of 2024. Our services include Interbank GIRO (IBG), JomPAY, MyDebit, Financial Process Exchange (FPX), Direct Debit, Shared ATM Network (SAN), eSPICK, and DuitNow, covering domestic clearing and payment services as well as selected cross-border transactions, serving 42 participating banks across Malaysia.

In this post, we explore how PayNet transformed its business intelligence (BI) capabilities using Amazon QuickSight to enable near real-time payment analytics, improve operational visibility, and enhance data quality monitoring across our diverse payment services.

The challenge

Prior to our modernization efforts, our BI infrastructure faced significant operational hurdles. Our legacy BI tool was functional for basic reporting but created substantial challenges in our growing organization. Dashboard sharing was particularly problematic, requiring users to manually download large files that consumed significant storage space and resulted in extended download times. Performance degradation was common when handling large datasets, especially on machines with limited resources. The situation was further complicated by our security requirements, which prevented direct database access for non-technical users. This led to a bottleneck in data refresh processes, because designated personnel had to manually update the data. The lack of robust governance features and row-level security in the desktop version also posed significant risks to our data security and consistency.

Why Amazon QuickSight

After evaluating various options, including self-hosted open source tools and custom development, we chose QuickSight for three primary reasons. First, its seamless integration with our data lake in Amazon Simple Storage Service (Amazon S3) simplified data connectivity and governance. Second, as a web-based platform, it enabled effortless collaboration and maintained data consistency across the organization. Finally, its scalable, managed infrastructure eliminated the need for internal infrastructure management while providing high availability.

Implementation and architecture

Our QuickSight implementation uses Amazon Athena to query our Iceberg tables registered in the AWS Glue Data Catalog, providing a seamless connection to our data lake without requiring data duplication.

Security was paramount in our implementation, shown in the preceding figure. We integrated AWS IAM Identity Center with our existing Okta infrastructure, providing our users with secure single sign-on capabilities. This integration helps ensure that access to sensitive financial data is strictly controlled while maintaining a smooth user experience for our team members. The system currently supports over 120 users across different roles, with more than 100 readers and 20 authors actively creating and consuming analytics content.

Key use cases and dashboards

Our QuickSight implementation supports several sophisticated use cases that directly impact our business operations. The Product Performance dashboard serves as a central hub for monitoring transaction metrics across our payment services. The Summary tab, shown in the following image, provides real-time visibility into daily transaction volumes and amounts, with detailed breakdowns by partners and critical transaction types. This dashboard has become essential for operational monitoring and strategic decision-making.

The Cross-Border tab, shown in the following figure, represents another key implementation, offering comprehensive insights into international payment flows. It visualizes both inbound and outbound transaction volumes and amounts, helping us identify trends and opportunities in our international corridors. This visibility has proven invaluable as we continue to expand our cross-border payment services.

Perhaps our most innovative implementation is the Data Quality Monitoring dashboard, which uses the generative AI capabilities of Amazon Q. This dashboard, shown in the following image, goes beyond traditional visualization by enabling dynamic defect reporting for participating banks. Users can select specific parameters and generate detailed defect records through natural language queries, significantly streamlining the process of identifying and communicating data quality issues to our banking partners.

By going to the Participants tab, users can analyze and compare performance metrics across different payment switch participants and partners. While individual participants usually only see their own transaction data, this feature gives our commercial team valuable market-wide insights. Users have the flexibility to create custom industry benchmarks by selecting specific participants or partners. This allows for targeted comparisons—for example, you can measure how Bank A performs against the market’s top three banks in terms of market share and growth rates.

We can create sample defect records by using filter-based questions. These records can be exported to Excel or CSV formats, making it straightforward for the business team to discuss the findings with the bank. All fields shown in the display are derived from the Named Entity data.

Amazon Q helps non-technical users access information quickly. Instead of navigating through complex dashboards or learning how to apply filters, users can ask questions in natural language and receive immediate responses.

The following are two example use cases:

  • Users can quickly check payment switch performance metrics for any specific quarter by asking Q, as shown in the following image.

  • Users can compare performance metrics between different banks for a target payment switch by asking questions using natural language, as shown in the following image.

Business impact and benefits

The implementation of QuickSight has delivered significant improvements across our operations. We’ve reduced the time required for transaction volume analysis from hours to minutes, enabling more effective infrastructure capacity planning. Tasks that previously required around 2 hours per event can now be completed in approximately 10 minutes, marking a 91.7% efficiency gain. The platform’s SPICE (Super-fast, Parallel, In-memory Calculation Engine) engine delivers highly responsive performance, processing several hundred million rows with 99% of queries completing in under 3 seconds, compared to previous BI’s 30-second average. This has resolved our previous performance limitations, allowing fast processing and visualization of large transaction volumes without needing to worry about infrastructure scalability.

From a cost perspective, transitioning from our previous BI tool has delivered remarkable financial benefits, reducing our BI tool expenditure by approximately 64.8%. This change has eliminated substantial licensing costs amounting to several hundred thousand Malaysian Ringgit annually. Additionally, we’ve optimized operational efficiency by saving approximately 40 person-hours per month previously dedicated to system and database administration tasks. This significant reduction in resources required for infrastructure management and maintenance further contributes to our overall cost savings, allowing our team to focus on more value-adding activities.

The enhanced data governance capabilities through AWS Lake Formation and role-based access controls have strengthened our security posture while maintaining operational efficiency. The platform’s integration with our AWS-based data lake has streamlined our development process, allowing us to combine data from multiple sources in an interactive visual environment.

Future plans

Our vision for QuickSight extends beyond its current implementation. We are actively working to expand the use of generative AI capabilities throughout our organization. By using Amazon Q in QuickSight, we aim to democratize data analysis, so that business users across different departments can independently build and customize dashboards without requiring extensive technical expertise. While Amazon Q in QuickSight is already delivering valuable insights, we are continuously providing our feedback to AWS as part of our collaborative partnership, helping to enhance the service and shape future capabilities that align with our evolving business needs. This ongoing dialogue ensures we maximize the potential of generative AI in our analytics journey while contributing to the advancement of the platform.

We’re also exploring ways to enhance our data quality monitoring system, with plans to implement more sophisticated anomaly detection and automated alerting mechanisms. These improvements will help us maintain the highest standards of data quality across our payment network.

Conclusion

The adoption of Amazon QuickSight has marked a significant milestone in PayNet’s digital transformation journey. By providing a scalable, efficient, and secure BI platform, QuickSight has enabled us to better serve Malaysia’s financial ecosystem. The platform’s integration with AWS services, combined with its advanced features like Amazon Q, positions us well for future growth as we continue to expand our payment services across the region. As we look ahead, QuickSight will remain central to our strategy of fostering a data-driven culture and maintaining our position as Malaysia’s premier payment network provider.

To learn more about implementing Amazon QuickSight for your organization:

About the authors

Jin Quan Goh is a Data Science Engineer with a strong passion for leveraging his expertise in data engineering and data science to solve complex challenges. He currently leads a data engineering team at Payments Network Malaysia (PayNet), where he oversees the development of data pipelines for payment switch systems, focused on reporting, analysis, monitoring, and data exchange. Jin Quan believes that while advanced technology and sophisticated data products are valuable, the true impact lies in delivering simple yet effective solutions that bring tangible benefits to others.

Nur Sabrina Binti Md Zaid is a Data Solutions Engineer at Payments Network Malaysia Sdn. Bhd. who bridges analysis, engineering, and solutioning – with domain expertise in e-payments products. Drawing from experience across the data lifecycle, she focuses on transforming raw information into insights through visualization and thoughtful design. Beyond data pipelines, she aspires to build and implement data-driven solutions that address socio-economic challenges and create positive social impact.

Tzy Wei Ling is a Data Engineer at Payments Network Malaysia Sdn. Bhd. who builds and maintains ETL pipelines and develops dashboards using AWS QuickSight. She helps make data more accessible across the organization for analysis, reporting, and decision-making. Passionate about data engineering and cloud technologies, she enjoys exploring how they can streamline workflows and improve efficiency.

Vinod Jaganathan is a Specialized Senior Solution Architect for Amazon QuickSight, with over 20 years of experience in the data and analytics domain. He has a proven track record of helping Fortune 500 companies derive business value from their data by driving scalable adoption of Enterprise Data Management, Business Intelligence, Embedded Analytics, and Data Monetization solutions.

Chun Yong (CY) Khoo is a Solution Architect focused on supporting Financial Services Industry (FSI) customers in their cloud adoption journeys. With a strong background in software development, CY began his career as a developer at a leading FinTech company. He leverages his hands-on experience and deep industry knowledge to help organizations modernize their technology stacks, drive innovation, and unlock the full potential of AWS solutions.