How ClearScale helped SF Match modernize its Microsoft application to optimize performance

By Pavel Vasilyev, Director of Solutions Architecture, ClearScale

While the speed of cloud computing is accelerating, the progress of companies using these technologies is being slowed by their own legacy applications.

That was the case for SF Match, whose legacy Microsoft application was delaying their performance. For more than 40 years, SF Match has used an algorithm-driven platform to provide residency and fellowship matching services to specialties in the healthcare industry. SF Match’s system was plagued by inefficiencies and prone to downtime. Both of these problems made system management and maintenance labor-intensive, costly, and detrimental to the user experience.

To improve its application, SF Match engaged ClearScale. ClearScale is an AWS Premier Consulting Partner that offers cloud consulting, cloud migration, application development, modernization, and data engineering services.

In this post, we explain how ClearScale was able to modernize SF Match’s legacy Microsoft application in the AWS Cloud. They moved onto open-source software with AWS by running .NET Core, migrated their SQL Server workloads onto Amazon Aurora Serverless, and automated their document approval process. After modernizing, SF Match was able to streamline its application development process to build a more reliable, efficient, and cost-effective architecture.

Challenges with modernizing SF Match’s application

SF Match’s application stack was over a decade old. Older legacy applications are common among organizations with businesses that are growing faster than they can stay on top of technology changes. SF Match developed their application by using .NET Web Forms technology. This framework is supported by ASP.NET, which was never designed to support today’s increasingly flexible, loosely coupled approach to web development.

Much of the code was unstable, and a proprietary object-relational mapping (ORM) system hadn’t been updated since 2008. To further complicate things, several different teams worked on the system. Each team had its own code styles and favorite tools and approaches. Unsurprisingly, system performance was low.

Due to the nature of SF Match’s application, there are different user types with widely varying roles. These user types include administrators, applicants, and medical schools.  The application’s .NET Web Forms architecture led the original development team to build different portals for each user type. Over time, as the website evolved, different user interfaces (UI), validations, and forms for different users were created.

It should have been a simple process. During the matching process, SF Match needs to create a phone number validator, implement that validator across the entire website, and ensure that all data could be properly used across the various aspects of the website. Instead, this process became cumbersome. Each time there was a change, SF Match updated each user type’s interface, midtier logic, and database controls—all while ensuring consistency across records. If at any point the data became inconsistent across these user types, interactions between these user types led to bugs and UI quirks. This made general reporting on the data more difficult.

An overview of ClearScale’s modernization: application and legacy code refactoring

In a previous project, we helped SF Match migrate from on-premises hosting to the AWS Cloud. To benefit from cloud-based environments, however, the company needed to change its approach to application development. Something also had to be done to ensure that its legacy applications could benefit from the new environment.

The first step following migration was app refactoring — restructuring the code in the legacy apps to optimize their performance in the AWS Cloud. We worked with SF Match to introduce modern libraries for development support, such as Ninject, Json.NET, Liquibase, and FluentValidator. We created a business logic code layer to encapsulate reusable code in separate services. This helped us introduce a new data access layer with Entity Framework, an ORM that eliminates the need for developers to write data-access code . We were also able to implement system load testing and fix the most critical paths, increasing requests per second (RPS) up to 125/second from 25/second. Finally, the refactoring provided horizontal scaling for web-server instances.

The most recent project consisted of a full rewrite of the backend application using ASP.NET Core, a redesign of the user interface, and a full rewrite of the frontend application using React JS. During the rebuild, ClearScale was able to migrate Microsoft SQL Server engine to the latest MySQL version on AWS Aurora Serverless. We reimagined the application and built using the latest .NET Framework. This helped us implement AWS artificial intelligence services to automate tedious tasks. Additionally, we created a flexible user-friendly user interface with React JS.

The goals of the rebuild were to:

• Enhance the users experience with new features
• Ensure that the system is secure and automated
• Eliminate reliance on the knowledge of any single individual
• Increase the systems performance and allow for more scalable operations
• Reduce the risk of development errors
• Speed up development with use of modern libraries

Rebuilding the Architecture

The rebuild process included gathering all the application’s business logic and rearchitecting the software with an open source platform.

Architecture design overview

To orchestrate the Linux application containers which would be deployed in AWS Fargate, we drew on AWS best practices and used Amazon Elastic Container Service (ECS). Fargate offers seamless scalability and allows for running containers without having to manage servers or clusters.

Then, to automatically start up the database, shut it down, and scale capacity based on application needs, we used Amazon Aurora Serverless. Amazon Aurora Serverless is an on demand, automatic scaling configuration that doesn’t require database instance management. This helps SF Match focus on innovating its technologies, rather than managing its databases.

We used -effective yet fast, highly available, and secure hosting for the front-end static web resources. It works with Amazon CloudFront, a service for creating and provisioning a collection of related AWS resources. CloudFront serves as a content delivery network (CDN) and as a reverse proxy for Kestrel, a cross-platform web server for ASP.NET Core.

For handling small code routines that serve various technical purposes (such as initiating SES notifications), we implemented AWS Lambda. Lambda, another serverless solution, was valuable to SF Match because it requires no provisioning or management of a runtime environment for the code. It’s also cost effective due to its precise metering of the code execution time.

Finally, we also used Amazon Textract, a service that automatically extracts text and data from scanned documents. This serverless solution allowed us to validate the documents to see if they are properly assigned or of a specific document quality.

This rebuild has led to clear results for SF Match. By moving to MySQL and Linux, we eliminated the costs for licensing and enabled the applications to scale infinitely. Using AWS best practices, we also automated full application deployment, making it easier for SF Match to process requests. In addition, we improved the overall user experience through workflows that minimize the number of steps to complete a task. We also optimized the site user interface for mobile, tablet, desktop and popular browsers. We achieved consistency as well: reusable frontend components for UI were also built to ensure consistency across the experience.

Automating SF Match’s document approval process

Previously, SF Match manually approved uploaded documents before they were distributed. This meant that SF Match had to approve thousands of documents per year. The nature of this workflow forced SF Match to review these documents within a short period of only a few weeks per year. ClearScale helped SF Match eliminate this tedious task. This not only freed up SF Match resources for more beneficial tasks, but also allowed their operation to scale by removing a large bottleneck.

SF Match’s CI/CD Pipeline for approved documents

After SF Match manually approves documents, the documents enter SF Match’s CI/CD workflow, shown in the previous diagram. The CI/CD workflow uses AWS Managed Services and Jenkins, an open source solution configured for continuous integration. Jenkins builds artifacts and pushes them to the corresponding S3 object storage bucket. Then,

Manually approving thousands of documents even before they entered the CI/CD pipeline was labor and time-intensive. However, using Amazon Textract allowed SF Match to free up resources by automating this process. The following document illustrates how documents were passed through the AWS Cloud and passed to a physical user for approval.

Document Approval Process before ClearScale Rebuild

We implemented Amazon Textract to build logic into the system. This helped us see if the uploaded document was in the correct context. Then, we used Amazon Textract to gain a level of confidence about the text on the document. This provided the ability to simulate a document that would not be readable by humans.

Document approval process after ClearScale rebuild

Now, SF Match only reviews the documents that have failed the initial Amazon Textract review process. After the rebuild, Amazon Textract allowed SF Match to review a fraction of the total documents, while still providing the same level of quality for the distributed documents.

Summary

With a modernized infrastructure and application approach in place, SF Match now has a faster, easier, and transparent process for application building and deployment. The infrastructure is more efficient, reliable, and cost effective. Much of the previous maintenance burden has been eliminated, freeing up staff for other projects.

Not all companies with legacy applications face a total application rebuild like SF Match did. For some, refactoring or other options suffice. The key is for companies to understand their business needs and goals — both for today and tomorrow.

Contact ClearScale, an AWS Premier Consulting Partner, to learn how they can help your organization transition to modern application development and deployment to better meet your company’s needs.

_________________________________________________________________________

More to come

To see how your organization can, like SF Match, save money while driving innovation and improving performance, please continue to join me as I highlight companies on their modernization journeys onto AWS. As you do, I hope you’ll also ask the question, what’s your plan for moving off Microsoft Azure? Or off Oracle? Or whatever old-guard technology keeps you from modernizing to better care for your customers?

AWS can help you assess how your company can get the most out of cloud. Join the millions of AWS customers that trust us to migrate and modernize their most important applications in the cloud. To learn more on modernizing Windows Server or SQL Server, visit Windows on AWS. Contact us to start your migration journey today.