Unlocking Mainframe Modernization for Success: Best practices to accelerate the mainframe to cloud journey

First introduced in the mid-20th century, mainframe systems continue to be the backbone of many enterprises relying on these systems for mission-critical applications. In fact, it is estimated that at least 71% of Fortune 500 companies use mainframe systems. Globally, 90% of credit card transactions happen on mainframe systems. According to IBM, USD 7.7 trillion in annual credit card payments, 29 billion annual ATM transactions, and 12.6 billion transactions per day at 92 of the world’s top 100 banks rely on mainframes to host their core systems.

Due to the acceleration of digital transformation and virtually unlimited capacity enabled by cloud computing, the number of companies entering mainframe modernization conversations is increasing. The expansion of many nations’ digital economies has been a major factor in the increasing demand for mainframe modernization, but there are additional factors. Legacy systems pose challenges in terms of agility, scalability, and integration with modern technologies. Organizations are increasingly recognizing the need to migrate or modernize their mainframe systems to enhance efficiency, reduce costs, and stay competitive.

Depending on the business drivers, and current technology constraints, there are different modernization strategies that can be implemented to achieve the expected outcomes. Some of these strategies are: Replatform, Automated Refactoring, and Data Augmentation. In most cases, a combination of these strategies is used to achieve a gradual, cost-effective modernization process. Successful mainframe modernization requires careful planning, clear understanding of business objectives, collaboration between business and IT leadership, strong executive support, and a solid “fail fast, learn fast” culture.

Complementary to “Patterns and Best Practices for Mainframe Modernization with AWS”, and based on our extensive experience, this post will share best practices in the mainframe and midrange modernization journeys. Although some of the recommendations might be specific to a modernization strategy, most of the concepts can be extended to all mainframe modernization patterns. We will also use the words mainframe and midrange interchangeably through the article.

Best practice 1: Customer’s business goals and constraints should drive modernization strategy

A customer’s business goals drive directly into the appropriate patterns selected for mainframe modernization. These goals and constraints guide the technical approach such as refactoring, replatforming, or augmenting. Customers must dive deep into the major motivations (reduce operational overhead, accelerate business processes, spark innovation, and strengthen resilience across IT) and consider their intersection.

Reducing operational overhead is a key motivation for mainframe modernization, as legacy systems can be costly to maintain/operate and often require specialized skills. Second, accelerating business processes is crucial for staying competitive in today’s fast-paced market. Customers should evaluate how modernization can help them become more agile, and responsive, enabling them to quickly adapt to changing market conditions and demands. Third, sparking innovation is also a significant motivation, as mainframe modernization can unlock new capabilities and open the door to emerging technologies. Customers often face the challenge of leveraging data to create news channels, user experiences, and roll out new products/services. Finally, strengthening resilience across the IT landscape is essential, as mainframe systems often serve as the backbone of critical business operations. By carefully considering the intersection of these motivations, customers can develop a holistic mainframe modernization strategy that delivers tangible business outcomes.

The selection of one or several mainframe modernization pattern must be closely aligned with the customer’s specific goals and constraints. When this alignment is achieved, mainframe modernization projects can retain strong executive sponsorship and become true enablers for the customer’s business. Through a meticulous analysis of the convergence between reducing operational overhead, accelerating business processes, sparking innovation, and strengthening resilience, customers can develop a modernization strategy that delivers sustainable business outcomes. As a result, the mainframe modernization efforts become strategic enablers for the customer’s business, rather than being perceived as merely an IT expense objective.

Best practice 2: Applications’ discovery requires close collaboration with business stakeholders

Many companies have large monoliths that consolidate several functionalities/applications in one system. In these cases, decoupling them and planning for production cutover increases complexity. Application’s discovery provides the technical input, but it must be complemented with the business perspective. For example, the decision between a single Go-Live versus a phased approach with multiple Go-Lives is a critical one that must be deeply analyzed in close collaboration with key business stakeholders. This decision impacts project aspects like the timeline, cost, team size and expertise, integration requirements, risk of customers’ impact, and regulatory institutions.

You must understand the application’s dependencies within the system, which is a complex task that requires different points of views and perspectives. Some might expect that this will be done automatically with a tool like AWS Blu Insights. However, the analysis and results provided by just a tool are not enough. Business perspective during the Discovery phase is essential. Some reasons are: alignment with business objectives, prioritization with business in mind, understanding of impact on external customers and users, evaluation of regulatory compliance, and alignment with internal initiatives/projects across the company. For example, a recent customer was deciding between a “big-bang” or a phased strategy, and requested an analysis of the dependencies for every application. Nobody held that knowledge of about 20+ years of history. Because of the high level of interdependency between all of the applications, the technical exercise suggested a single go-live. This was reasonable considering the fact that for each go-live you will have additional costs and time because of the development of new interfaces, more testing effort, potential latency issues, and delayed return on investment. Rather than rushing into a single Go-Live, the team recognized the value of investing time to closely collaborate with key business stakeholders. By going into detailed discussions about each individual application, the modernization team was able to gain a deeper, more holistic understanding of the customer’s unique requirements, constraints, and risk factors. This collaborative discovery process enabled the development of a more tailored and phased modernization strategy, ultimately resulting in a plan structured across 9 distinct waves. The decision to pursue a phased approach, rather than a single Go-Live, was a direct outcome of the team’s close collaboration with stakeholders during the discovery process.

Successful mainframe modernization requires a comprehensive understanding of the application landscape and its complex interdependencies. This understanding can only be gained through close collaboration between the technical team and the business stakeholders. While tools can provide valuable insights, the analysis and results they produce are not sufficient on their own. The business perspective, obtained through a thorough discovery phase, is essential for aligning the modernization strategy with the organization’s strategic objectives. This will allow prioritizing the right workloads, assessing the impact on external customers and users, evaluating regulatory compliance, and ensuring cross-functional alignment.

Best practice 3: Implementing a pilot will provide real-world validation

Conducting a pilot serves as a bridge between the theoretical understanding gained from a proof of concept (PoC) and the practical realities of a live production environment. In mainframe modernization, conducting a pilot holds particular significance due to the unique challenges associated with these core systems. A PoC is typically done in a controlled environment with an independent application of less than 100,000 lines-of-code (LOC), which doesn’t necessarily go through every step that will be required to go into production. In most cases you will have successful results in PoCs. However, due to the unique challenges in midrange/mainframe systems, there are key aspects to consider, like integration issues, latency, user acceptance tests, and operational aspects. Most of these (if not all) will only be proven during a pilot. This experience enables you to find unexpected blockers going to production, and gain value from a modernized application, which can help give momentum to the migration at scale. This is why a pilot is strategic in the overall modernization journey, increasing the likelihood of a successful and seamless transition to modernized environment.

As an example, in a midrange modernization project, after a successful PoC, the company decided to move forward with a pilot and realized the strategy defined was not the right one. The technology selected faced challenges when integrating with new technologies and services in the cloud. Because of these events, the organization decided to pause the current approach and evaluate other alternatives that would meet all their expectations. Every migration project will have a first workload migrated to production. This is an important milestone within the migration’s efforts as it will provide you with different learnings directly related to your reality. This experience will allow you to adjust your strategy and methodology as necessary to embrace the future migrations. Addressing technical hurdles with the right level of business support can swiftly resolve these challenges, enabling our customers to realize tangible benefits and propel their modernization initiatives forward.

Best practice 4: Think about the testing strategy early

It is crucial to have dedicated Subject Matter Experts (SMEs) for the applications being modernized. When thinking about the testing strategy, you must define the scope, and for this, the SME’s in-depth understanding of the mainframe application is mandatory. Their work here will be focused on identifying the functionalities and mapping them against test cases with their documentation. For example, when starting the modernization journey with a company, the expectation was that we would need 1/3 of the test cases. After 2 months, during a status meeting, the business leader of the application noticed some of the functionalities identified were no longer in use. At this moment, a dedicated SME was assigned and was able to depurate and map the right functionalities. With the SME’s support, the testing team was able to plan accordingly with the right number of test cases and the necessary documentation.

As part of this definition, it is strongly recommended to implement a single source of truth for the functionalities and test cases that will be part of the scope. It involves centralizing test artifacts, documentation, and test data to avoid discrepancies and ensure consistency across the testing lifecycle. A central repository facilitates efficient communication and collaboration among the teams involved in testing. A lot of time can be saved by managing code version and test cases between business, development, and testing teams.

The implementation of automation in testing strategies is indispensable for the efficiency of the testing process. It not only provides acceleration but also enhances accuracy, particularly in Functional Equivalence Testing (FET), which involves repetitive and high-volume test cases. Testing being a critical success factor for mainframe modernization and accounting for at least 60% of mainframe projects, automation will help setup a factory model and scale to the capacity needed. A manual testing strategy will bring several challenges, and the capacity needed will increase drastically, affecting the timeline and cost of the project. For instance, recently announced in AWS re:Invent 2023, AWS Mainframe Modernization Application Testing, is a feature of the AWS Mainframe Modernization service that automates the execution of application tests at scale. It can be used for mainframe application modernization and migration use cases.

User Acceptance Testing (UAT) requirements are critical for validating end users’ expectations. Clear communication and collaboration between development, testing, and business stakeholders are vital to define and understand UAT requirements accurately. This should also be done at early stages to avoid any road blocks in the future. It is important to share and explain all the different types of testing (Functional Equivalence Testing, Performance, Integration, UAT) that are part of the modernization journey to all parties involved. With this, business users are able to understand the importance of their participation and the objective of each testing type so they actively participate in defining an efficient strategy that avoids retesting efforts.

Best practice 5: Invest time in a comprehensive analysis for data migration

Data migration requires a comprehensive analysis of the existing data to understand its structure, relationships, dependencies, and volume. Identify the core data that must be migrated quickly to support initial testing and ultimately go-live cutover. This data migration can be separated from any which must be migrated for regulatory, historic, and research purposes. This second group is often very large and may benefit from different processes and destinations.

If the character set is changing, establish a clear understanding of the process and effort required. Do this by migrating some of your largest and most complex datasets and validating the results. If the collating sequence is changing, leverage the business stakeholders to identify processes that might be impacted. Be sure they understand what the change will be, for example, sorting numbers before letters. As a case in point, in one migration converting to Unicode it was only discovered in UAT that significant warehouse processes were built around the EBCDIC collating sequence. By investing time on understanding the data ecosystem, you can avoid risks that could delay the timeline and avoid an increase in cost.

One of the key strategies involves grouping workloads that share similar data dependencies and relationships. This approach minimizes the back and forth between the mainframe and the target environment, like AWS, thereby optimizing the migration process. By migrating related workloads as a cohesive unit, an organization can mitigate potential disruptions, facilitate data integrity, and streamline the overall transition. This alignment not only accelerates the migration timeline but also enhances operational efficiency.

Data migration is well-suited to a factory process (described in best practice 7) for the reasons already discussed. Build this factory early and refine it as needed.

Best practice 6: Consistent executive engagement is crucial

Making the project visible to all business units is a cornerstone of effective migration and modernization efforts. Positioning it as a strategic initiative rather than a purely technical one underscores its significance in achieving broader business objectives. Very often, in mainframe/midrange systems you will find applications with years of history that support core business functions of the company. This implies impacting many people from different departments. Support and leadership from stakeholders are mandatory to ensure that the project receives the expected dedication and commitment across all areas. This makes sure the larger organizational business goals are kept in mind during decision making.

Establishing an organizational change management (OCM) strategy is equally crucial. Recognizing the potential impact on processes, roles, technology, customers, and users is indispensable. A well-crafted change management plan helps proactively address resistance, fosters a culture of adaptability, and facilitates a smooth transition. This strategy should involve communication plans, training programs, and mechanisms for feedback and continuous improvement. Changing legacy systems with years of history will generate resistance. Resistance from people supporting the applications and used to the legacy language (Cobol, RPG, CL, etc.). Resistance from the business users accustomed to a specific front-end or functionality that might disappear or change with the modernization. This strategy won’t work by itself, it needs agents of change supported by stakeholders and senior leaders that will be your champions when setbacks arise.

In a recent customer project, the business users raised challenges in the journey because the front-end systems were changing and some features were removed. After different working sessions, the team determined that keeping all the functionalities requested would represent an impact of 5 months to the timeline. Mainly because of testing efforts required for the new development. When presenting the results to the COO, he requested the business area leaders to double check the estimates and prioritize critical features, to avoid stalling or delaying the modernization journey. This resulted in a reduction of 3 months from the original estimation. The migration will encounter roadblocks, and several decisions will have to be made during the project. Having stakeholders and senior leaders committed will be helpful in these situations, ensuring the goals are always considered when setbacks arise and decisions must be made.

Best practice 7: Establish a modernization and migration factory model to scale efficiently

Mainframe modernization projects can easily scale to more than 100 people working together for 3+ years in this journey. A well-defined and standardized process throughout the modernization lifecycle is fundamental to enabling the teams along this journey. Embracing a factory model provides clear steps from discovery to go-live while minimizing errors, and streamlining the overall project execution.

Employ modularization by breaking down the modernization process into manageable components that can be addressed individually. This enables parallel development efforts and accelerates the overall timeline. Well defined processes and templates provide guidance to the team and helps ensure consistency in execution. There might be many mechanisms specific to your company that can help in establishing this factory model. It is important to customize them, train the team, and make them available widely to ensure they are being used. Based on this, you should create your own framework, tailored to your organization’s characteristics and capabilities. This will be the backbone of your factory.

In a modernization engagement, the absence of a modernization and migration factory model became apparent as the organization started this complex journey. After discussing the importance of this model and the benefits seen from different experience, they recognized the need for a more systematic approach, and a factory model was designed and implemented. The team implemented a modernization pipeline where legacy artifacts, such as code and data, were fed into the pipeline, which then output a modernized version of the code and data. After that, a testing step was included to ensure functional equivalence between legacy and modernized versions of the application. This pipeline helped to scale the migration process, by allowing parallelism, standardization, and industrialization of the modernization.

All decisions made during the migration experience should focus on the organization goals and scalability. Given that there will be other workloads to migrate they can leverage the same mechanisms.

Conclusion

Mainframe modernization is a challenging journey due to the intricate nature of these legacy systems, the complexity of the code and technologies involved, data migration risks, and a shortage of skilled professionals. Budget constraints and organization resistance to change further magnify the challenges of modernizing mainframe systems. The success of a mainframe modernization project hinges on applying best practices, ensuring a strong sponsorship, immersing yourself in the day-to-day challenges of a production deployment, to scaling a factory model to ensure standardization and consistency. By intertwining these best practices, organizations can embark on a mainframe modernization journey that is not only efficient and error-resistant, but also strategically aligned with the overarching goals of the business, ensuring a seamless transition to a modernized environment. Do not expect a perfect planning with zero disruptions. It is a path filled with risks where the key is identifying issues early, and fixing them so they stay fixed.

About the authors: