Migration & Modernization

From Mainframe to AWS Cloud: A comprehensive mapping guide – Part 2 (Databases)

Mainframe databases have served as the foundational infrastructure of enterprise computing for decades, powering mission-critical applications across various commercial and government sectors with their robust transaction processing and data management capabilities.

In part 1 of this blog series, we explored the fundamental components of mainframe systems, focusing specifically on compute and storage capabilities. In this blog post, we explore potential options to migrate IBM mainframe databases to AWS, focusing on three core topics: Db2 for z/OS, Information Management System Database (IMS DB), and Virtual Storage Access Method (VSAM).

Enterprises are transitioning away from these traditional datastores into various AWS databases due to challenges posed by aging technology, skill shortage, architectural complexity, integration hurdles, maintenance and operational costs. This shift is driven by the desire to leverage innovative technologies, advanced analytics, and AI capabilities, while benefiting from modern architectural designs like event-driven and serverless systems.

IBM Db2 for z/OS

IBM Db2 (Db2) has been a strategic choice for large enterprises across diverse industries. With its robust capabilities, Db2 continues to power mission-critical applications for many organizations. AWS offers Amazon Relational Database Service (Amazon RDS for Db2) for running Db2 in the AWS Cloud, enabling organizations to leverage the benefits of a fully managed database service without sacrificing the familiar Db2 environment they rely on.

Amazon RDS for Db2 provides a managed option to run Db2 on AWS, leveraging highly scalable hardware designed for optimal performance and high availability. As a fully managed database service, Amazon RDS for Db2 simplifies the process of setting up, operating, and scaling Db2 deployments in the AWS Cloud, allowing organizations to focus on their applications rather than managing the underlying infrastructure.

Key advantages of migrating Db2 for z/OS to Amazon RDS for Db2:

  • Datatype compatibility between both the platform facilitates data portability.
  • Syntax and functionality of stored procedures largely compatible between the two platform.
  • Maintaining the existing EBCDIC collation behavior.
  • The SQL syntax and capabilities are largely shared between the two platforms, reducing the need for extensive rewrites.
  • A familiar Db2 database environment helps in planning and prioritizing database migration efforts.
  • Encryption at rest and in transit.
  • On-demand backup capability.
  • Automated daily storage snapshots stored in Amazon Simple Storage Service (Amazon S3).

For the complete list of features, refer to Amazon RDS for Db2 features documentation.

Amazon RDS for Db2 provides built-in monitoring and health checks to operate the service. This includes capabilities for alarming or notifying based on monitoring metrics and log collection with publishing capabilities. RDS for Db2 supports integrations with Amazon CloudWatch, Amazon RDS Enhanced Monitoring and the IBM Data Management Console which administer and monitor several Amazon RDS for Db2 database instances.

For more detailed guidance, refer to Getting started with new Amazon RDS for Db2.

Running Db2 in AWS can be approached in several ways beyond Amazon RDS for Db2. For organizations seeking alternatives to Db2, whether driven by budget constraints, licensing considerations, or modern application requirements, AWS offers various relational database alternatives. Those needing more administrative control can implement Db2 on Amazon Elastic Compute Cloud (Amazon EC2) instances directly. When performance and reliability are paramount, IBM Db2 pureScale on RHEL deployed on Amazon EC2 provides robust high availability features, adaptable workload management, and fault-tolerant operations while ensuring minimal application response times.

For data migration and replication, organizations migrating data from Db2 for z/OS to AWS, the AWS Database Migration Service (AWS DMS) can be utilized. AWS DMS supports Db2 z/OS as a source for the full load operational mode. The AWS Schema Conversion Tool (SCT) can be used to convert schemas and code objects from Db2 z/OS to various AWS database targets, including Amazon Aurora MySQL, Amazon Aurora PostgreSQL, Amazon RDS for MySQL, and Amazon RDS for PostgreSQL. Once the schema and objects are in a format compatible with the target database, AWS DMS can be used to migrate data from Db2 running on the z/OS operating system to any supported AWS DMS target as shown in Figure 1.

Customers have several options for migrating data from mainframe data sources to AWS databases. They can leverage AWS Marketplace partner solutions like Precisely to perform full data loads and enable ongoing replication. Alternatively, they can use IBM’s QREP (Q Replication) or IIDR (InfoSphere Data Replication) tools, which support heterogeneous database replication. If they prefer to manage the process themselves, customers can use DB2 native utilities like DB2UNLOAD, DB2IMPORT, and DB2EXPORT to migrate data directly from mainframe databases. For a fully managed service, customers can leverage the AWS Mainframe Modernization service with Precisely Connect, which enables near real-time data replication from various mainframe sources to a wide range of AWS cloud database destinations.

For detailed information, refer to Migrating tables from IBM Db2 for z/OS to Amazon RDS for Db2.

IBM IMS DB

IBM IMS DB (IMS DB), a hierarchical database management system, is widely adopted database by large enterprises due to its robust transaction processing capabilities and support for handling massive data volumes with high performance and reliability. As per IBM over 95% of Fortune 1000 companies use IMS in some capacity, as do all of the top five US banks.

When migrating from IMS DB, organizations have several options available based on their specific use case and migration approach. A common choice among organizations is to migrate from IMS DB to a relational database solution. Careful planning and analysis are required to restructure the data, handle complex relationships, and ensure data integrity and consistency. Additionally, performance considerations, indexing strategies, and potential data denormalization may be necessary to optimize the relational database design and queries. This migration provides several advantages, such as modernizing the database infrastructure and increasing agility through the use of contemporary relational database features. Additionally, this transition opens up new opportunities for innovation by allowing seamless integration with modern technologies like machine learning and serverless computing.

As shown in Figure 2, AWS offers a wide range of database options to facilitate the migration of legacy IMS DB systems to the cloud. The primary recommendation for transitioning IMS DB to AWS is Amazon RDS. When migrating data from an IMS database to Amazon RDS, each IMS segment is converted into a separate table within Amazon RDS. To maintain the hierarchical relationship present in the original IMS database, parent tables are assigned primary keys while child tables are given corresponding foreign keys, thus preserving data integrity and logical connections.

For read-heavy workloads where sub-millisecond response time is required with increased throughput and low-latency, customers can leverage caching solutions such as Amazon MemoryDB for Redis or Amazon ElastiCache. In certain scenarios where large data volumes that demand consistent low latency, or when rapid access to lower-level child segments within the IMS database’s hierarchical structure is necessary, individual IMS segments can be migrated to Amazon DynamoDB, a NoSQL database service.

Additionally, AWS offers partner solutions such as MarkLogic Multi-Model Database through the AWS Marketplace, where customers can directly provision the required database software and launch it in their AWS account.

To migrate data and perform ongoing replication from IMS DB to any of the previously mentioned AWS database services, customers can use IMS native tools to unload data from the mainframe. They can then leverage the AWS Mainframe Modernization data migration and replication solution provided by Precisely to replicate ongoing changes in the target environment.

The application code must be refactored by replacing IMS DB API calls with the corresponding target relational database API calls. In scenarios where customers prefer to continue using IMS DB, they may consider employing the AWS Mainframe Modernization service with Rocket Software (formerly Micro Focus) and UniKix.

VSAM

VSAM is a file management system for IBM z/OS. It is a high-performance access method and data set organization, which organizes and maintains data in a catalog structure.

The VSAM datasets are divided into three types based on the data storage and access methods: entry-sequenced data set (ESDS), key-sequenced data set (KSDS), relative record data set (RRDS). For batch processes, customers can migrate data from ESDS to Amazon S3, Amazon Elastic File System (Amazon EFS), or Amazon FSx for Lustre , which allow sequential access to records, like the access pattern of ESDS. For other cases, Amazon RDS is a migration option. Figure 3 shows the potential options to migrate VSAM to AWS Cloud.

For KSDS and RRDS, application re-architecture is required. When re-architecting applications, customers can migrate data that requires referential integrity to Amazon RDS. For standalone data that needs low latency access or for variable-length records that require advanced querying capabilities, such as secondary indexes and filtering options, customers can choose Amazon DynamoDB. The data mapping between the legacy VSAM files and the target AWS databases will depend on the use case and how the data is organized and grouped at the source. For more complex use cases, such as multi-record layouts or heterogeneous data migrations, customers can consider using partner solutions or the AWS Mainframe Modernization Data Replication with Precisely solution, which provides initial load and data replication capabilities from source data sources like VSAM to AWS database service offerings, ensuring data consistency, accuracy, freshness, and validity. For scenarios where customers prefer to continue using VSAM on AWS, they can consider using AWS Mainframe Modernization services.

Conclusion

Some enterprises still rely on legacy databases like IDMS, Datacom, and Adabas to support their critical business operations. These legacy databases are reliable but migrating away from them can be complex. Recognizing the need to help these customers modernize, AWS has partnered with specialized solution providers. These AWS partners have expertise in migrating data from legacy systems to modern, cloud-based infrastructures. By leveraging these partner solutions, organizations can modernize their IT infrastructure while preserving the value of their historical business data. This approach allows companies to take advantage of the scalability, flexibility, and cost-effectiveness of cloud computing.

Stay tuned for our next post, where we’ll dive into a comparison of security, monitoring, logging, and scheduling options on z/OS and AWS, exploring their features and functionalities.

Further Reading

https://aws.amazon.com/mainframe-modernization/capabilities/data-replication/

https://aws.amazon.com/blogs/architecture/field-notes-set-up-a-highly-available-database-on-aws-with-ibm-Db2-pacemaker/

https://aws.amazon.com/blogs/database/data-migration-strategies-to-amazon-rds-for-Db2/

https://aws.amazon.com/marketplace/pp/prodview-yk3zbwbgavbcy

https://aws.amazon.com/blogs/database/migrate-an-ibm-db2-for-iseries-database-to-amazon-aurora-postgresql-using-blu-age-and-qlik-replicate/