State of Utah Application Modernization: The Express Lane from Mainframe to Cloud

By Hemant Ahire, WW Migrations/Modernization Partner Solutions Architect – AWS
By Stefan Aulbach, Application Modernization Innovation Lab Lead – Deloitte Consulting LLP
By Omer Enaam, Managing Director, Application Modernization and Migration, Deloitte Consulting LLP

Deloitte

The Office of Recovery Services (ORS), a division of the Utah Department of Human Services (DHS), has successfully moved a 25-years-old mainframe application to AWS GovCloud (US)—in budget and on time—using mainframe modernization capabilities from Deloitte and Amazon Web Services (AWS).

The story behind this effort shows how beneficial such a move can be, from a day-to-day perspective and from a strategic perspective. It also demonstrates that a high degree of automation and standardization makes this a riskless approach for getting rid of legacy technology such as COBOL and VSAM.

Deloitte is an AWS Premier Tier Services Partner and Managed Service Provider (MSP) with the AWS Mainframe Migration Competency. Deloitte’s end-to-end capabilities and understanding of your business and industry help amplify the transformative value of cloud.

Introduction and Project Scope

In 2016, the State of Utah’s Office of Recovery Services (ORS) started to explore options to modernize one of their vital applications—the ORSIS child support system. The Office of Recovery Services Information System (ORSIS) is a specialized case management and accounting system for child support. It is deployed on a mainframe platform using COBOL computer programming language.

The existing technology was limiting ORS’s ability to optimize their business and scalability. The maintenance of this application was expensive and required resources that understood the legacy technology. ORS also wants to serve citizens more efficiently by providing more relevant and up-to-date information.

The ORSIS child support system is a central application with connections to more than 100 surrounding systems, including a public-facing web application. Because modernization is challenging, ORS was stuck at “green screen” for a long time.

They were looking for a strategic transformation approach that had a high success rate, high probability of user adoption with the lowest organizational impact, and provided the ability to incrementally scale. The modernization process must ensure business continuity over the duration of the project timeline, minimal impacts on day-to-day operations, and provide the foundation for new cloud functionality.

ORS considered the following options:

• Option one was to manually rewrite the child support application from scratch to remove it from the mainframe. Upon further analysis, this would incur upwards of $200 million for a complete manual rewrite at a higher risk to the migration program with no clear timeframes.
  .
  This option also requires a certification from federal agencies that could potentially add 5-10 years to the rebuild effort. This would mean the State of Utah would not be able to take advantage of services and applications.
  .
• Option two consisted of an “automated refactoring” of the mainframe application running on premises and migrating to AWS using Deloitte’s innoWake toolset. This migration pattern would serve as an intermediate step to modernization and entail converting the COBOL code to Java, running the mainframe system on an underlying Amazon Elastic Compute Cloud (Amazon EC2) compute and a DB2 database engine on AWS.
  .
  This approach would also allow them to modernize the legacy application incrementally and do it “screen by screen” or “function by function.” This option had the biggest benefit to the State of Utah because recertification is not a mandatory process, therefore causing minimal risk to business operations. This approach would also facilitate future modernization to cloud-native technologies, capitalizing budget due to the knowledge of upfront costs.

Deloitte’s solution to the migration focused on automated code and data conversion. The conversion involved code translation from legacy COBOL to modern Java, and the new system was functionally equivalent with business continuity.

They were able to unlock the potential of next-gen technology to drive modern mobile applications, analytics pertaining to citizen provided services, and process automation with artificial intelligence (AI) tools that capitalized end user adoption.

Project Outline

The joint effort followed a standardized innoWake project outline for mainframe modernization. The typical project duration is 15-18 months and was organized in five partially overlapping phases:

Preparation

The automated migration environments are set up, and the source artifacts are checked for completeness.

Automated Code Refactoring, Data Conversion, and Unit Testing

Source code is automatically transformed and deployed to internal test systems. Data is extracted from the legacy system to serve as test data. Test cases for the acceptance of this phase are negotiated and implemented with the client. These tests serve as the basis for the automated testing of the transformed application.

Deloitte leverages TermX, a proprietary testing tool, during the unit testing phase. Its features include mainframe application connectivity, mainframe screen emulation, and screen recording functionality.

Deloitte captures and records business use cases from the original environment using TermX and replays the test cases against the modernized user interface (UI). This helps eliminate the need to manually reproduce the dialog workflow in the migrated environment, reducing error caused by human intervention or misunderstanding.

System Integration Testing, Performance Testing, and Pilot

After the functional tests are conducted, focus shifts to integrating with surrounding systems, as well as performance testing of critical infrastructure. User acceptance tests (UAT) are conducted, and the pilot application is released. This is considered a major milestone and demonstrates go-live readiness. Based on the results from the acceptance tests, the final production environment is sized and architected.

Pilot and Go-Live Preparation

Before go-live, changes from UAT and the pilot demonstration are integrated into the release candidate. Meanwhile, the production environments are prepared for go-live. Appropriate parties will be informed about the downtime of the legacy system.

System Cutover and Post Go-Live Support

At the specified date, the legacy system will be turned off and the modernized system takes over. After the go-live, Deloitte typically provides on-site support for specified duration of time.

Figure 1 – innoWake project outline for mainframe modernization.

Automated Code and Data Conversion

Deloitte’s transformation tools ensure business continuity of the legacy application, including ongoing development. Changes to the legacy applications can be incrementally applied during the transformation process.

The following table shows available options (not exhaustive) for selected mainframe technologies.

Code Conversion

Using the Application Modernization Transformation solution powered by innoWake, Deloitte migrated 2.5 million lines of COBOL to Java in a fully automated fashion.

Figure 2 – Mainframe modernization solution powered by innoWake.

The ORSIS application is comprised of an online workload using CICS, and 400 JCL-based batch jobs.

For the online workload, the innoWake transformation suite provides CICS transaction semantics, queue implementations, data access interfaces to VSAM and DB2, as well as any low-level APIs for interacting with the CICS system. It replaces 250 CICS screens (3270) with a Vaadin-based web application, to immediately leverage modern UI technologies without manual intervention.

The transformation framework provides flexibility to integrate with existing infrastructure, like authentication, authorization, and audit systems, but also logging and tracing. The only requirement for deployment is a Java Servlet container or JEE application server with a JTA-compliant transaction manager for more complex deployments.

For batch workloads, the innoWake transformation suite provides a replacement for JES2 (Job Entry Subsystem) which can handle the semantics of JCL, including step handling, PROC definitions, file disposition, control flow, input/output handling, and other JCL specifics/utilities.

There are replacements for system commands (such as IEBGENER and IEFBR14) as well as frequently used third-party tools like ICETOOL. Existing JCL scripts remain as-is and are interpreted at runtime. The invocation of the batch jobs can be scheduled using any scheduling solution available on the market.

For file operations, the innoWake transformation enabler batch abstracts the mainframe semantics to mimic the record-structured datasets (like QSAM and GDG) atop of regular files, including transparent conversion to Unicode and/or CSV. The innoWake enabler uses a virtual filesystem and can provide access to web resources like Amazon Simple Storage Service (Amazon S3).

Often, program logic and database-stored procedures are written in COBOL. In this case, these also need migration to a Java-enabled DB2 LUW. Although stored procedures do not pose a risk from a technical perspective, they require some attention during the migration process, as there is a different invocation model between DB2/z-COBOL and DB2/LUW-Java. The innoWake enabler provides the necessary wrapper to call converted COBOL stored procedures as external DB2 routines.

To ensure successful application conversion, the innoWake suite provides a set of test tools. Dialog-oriented workloads are being tested with TermX using a capture-replay approach, which records the interaction with the 3270 screens. The captured interaction is then replayed against the transformed application.

For batch workloads, tests are conducted against isolated snapshots extracted from the legacy system. For each job schedule, the before and after snapshots, each intermediate work files, and each result are compared between the original and transformed application.

Data Conversion

The basic conversion process for relational data migration consists of using bulk load and unload standard processes to transfer the 2TB data from IBM DB2 Z/OS to IBM DB2 LUW.

VSAM data is unloaded and transferred as sequential dataset out of the mainframe. The sequential dataset is then loaded into the VSAM emulation to generate the structures in the DB2 database. This emulation model supports ESDS, KSDS, and RRDS datasets, as well as other features like record-level locking. Record structured datasets can be converted to Unicode files that can be placed on any accessible file system.

The innoWake testing tool can generate data verification scripts to ensure data integrity during the data migration. For any data, innoWake products take care of codepage conversion between the mainframe’s EBCDIC codepages and Unicode codepage of the transformation target—without any change to the existing codebase.

The goal of the defined procedure is to keep the number of data conversions minimal. Project-specific solutions for incremental data transfer using techniques like log shipping, changed data capture (CDC), or replication are being available with this solution.

Data consistency is tested twice—once using the above-mentioned verification scripts, and once using the automated testing facility on application level.

Architecture Blueprint for innoWake Transformation

The architecture diagram below depicts a highly available, scalable, and secure AWS blueprint for running mainframe systems on top of AWS infrastructure. This architecture serves as an initial pattern suited for an automated refactoring migration scenario of the mainframe application running on premises and migrating to AWS.

Figure 3 – AWS blueprint for running mainframe systems.

The architecture diagram above depicts a highly available, scalable, and secure AWS blueprint for running mainframe systems on top of AWS infrastructure. This architecture serves as an initial pattern suited for a migration scenario of “automated refactoring” of the mainframe application running on premises and migrating to AWS.

The three-tier architecture consists of the following:

1. Web tier: The web servers are distributed across two or more AWS Availability Zones (AZs) in a given AWS region to provide high availability and fronted by an Application Load Balancer. Each AZ is a physical data center in a different geographic location, and the web tier is running inside a virtual private cloud (VPC) and has its own unique subnet per AZ.
2. Application tier: The application servers are also distributed across two or more AZs in a given AWS region to provide high availability and fronted by an Application Load Balancer.
3. Database tier: The databases are running on Amazon Relational Database Service (Amazon RDS) which is a performant, secure, and cost-efficient database supporting several database engine types optimized for memory, performance, or I/O. Amazon RDS automatically creates a primary DB instance and synchronously replicates the data to a standby instance in a different AZ. Each Availability Zone runs on its own physically distinct, independent infrastructure and is engineered to be highly reliable.

A first-sizing of the AWS environment is determined up-front using a parameterized model translating mainframe metrics like MIPS or TPS into Amazon EC2 machine classes. This sizing is used for cost calculation and as a starting point for refining the sizing during user acceptance tests or pre-production test runs. Sizing can be adjusted after go-live using the elastic scaling capabilities of EC2 and other AWS services.

Conclusions and Outlook

By taking advantage of the combination of AWS as the target platform and innoWake as the facilitator, the State of Utah’s Office of Recovery Services has been able to achieve:

• Processing of up to 600,000 transactions per day with AWS for the thousands of families who depend on child support in Utah.
• Reduced infrastructure procurement and decreased infrastructure setup time from weeks to hours through the AWS and SaltStack solution.
• Easier augmentation of new developers allowed for by transforming the existing COBOL into Java.
• Improved batch performance enabled by horizontal scaling; Amazon EC2 allowed for rapid adjustments to underlying infrastructure based on load (transaction volume).

According to Liesa Stockdale, Director of the Office of Recovery Services at the Utah Department of Human Services, the transformation project was finished within 15 months on budget, and at significantly lower cost compared to traditional software modernization projects.

In her AWS Public Sector Summit 2021 talk, Liesa emphasized the ORSIS system went live on the original scheduled date that was initially communicated in the project planning phase—with no service downtime.

Gene Riggs, Application Development and Systems Support Manager at the Utah Department of Government Operations, emphasizes the operational cost of the modernized system is significantly lower than what they were on the mainframe. According to Gene, the agency is reinvesting the budget into further modernizing the system and its development.

This success story shows that after transformation and re-platforming onto AWS, clients have endless possibilities for digital modernization. A successful modernization uncovers the treasures from the legacy world like data in VSAM, flat files and IMS, and provides modern and sustainable ways of using them. It culminates in new human-centric user experiences for increased productivity, modern microservices architectures, and leveraging AI/ML and cloud-based analytics for deep insights into the business.

AWS provides a set of managed services for the full data life-cycle: ingestion (Amazon Kinesis); storage (Amazon S3, Amazon DynamoDB); processing and analysis (Amazon EMR); visualization (Amazon QuickSight); and automation (AWS Data Pipeline).

AWS’s big data services such as AWS Lake Formation can give clients the opportunity to empower their business, creating new possibilities for not only serving their own customers but also serving across various functions such as sales, marketing, revenue, and fraud detection.

Mainframes have dominated for a long period of time, but their strict separation between online and batch processing negatively impacts the business. The most accurate data is only present moments after the batch runs—with a delay in the range of hours or days.

With AWS and Deloitte, clients can increase the value of their IT landscape by introducing agile development techniques, lowering complexity, and leveraging modern technologies such as AI/ML. Now it is time to break the barriers of the mainframe—in a safe and riskless fashion, as demonstrated by this AWS and Deloitte success story.

Best Migration Solution: 2021 AWS Public Sector Partner Awards

Omer Enaam, Application Modernization Leader at Deloitte, and Bart Mason, Technology Lead for the Office of Recovery Services, Utah Department of Human Services, talk with SiliconANGLE’s Natalie Erlich for the AWS Global Public Sector Partner Awards 2021.

.

Deloitte – AWS Partner Spotlight

Deloitte is an AWS Premier Tier Services Partner and MSP. Through a network of professionals, industry specialists, and an ecosystem of alliances, they assist clients in turning complex business issues into opportunities for growth, helping organizations transform in the digital era.

Contact Deloitte | Partner Overview

*Already worked with Deloitte? Rate the Partner

*To review an AWS Partner, you must be a customer that has worked with them directly on a project.