How Tyler Technologies improved access to justice during the COVID-19 pandemic using .NET on AWS

By Zovin Khanmohammed – Software Architect at Tyler Technologies
    Arun Krishnaswamy – Sr. Solutions Architect at AWS

(Tyler Technologies, Inc. (NYSE: TYL) is the largest provider of software to the United States public sector. The end-to-end solutions that Tyler provides empower the public sector to create smarter, safer, and stronger communities.)

This blog post tells the story of how Tyler Technologies built the Virtual Court solution using the .NET development framework on Amazon Web Services (AWS), and provided much-improved remote access to municipal court infrastructure in the midst of the COVID-19 pandemic.

Physical appearance in court is inconvenient at best and occasionally impossible. Even prior to the onset of the COVID-19 pandemic, defendants were routinely unable to resolve court business due to either lack of childcare, or being unable to miss work. Elderly and sick individuals likewise are often unable to make it to the court’s location in person. Moreover, with small traffic, property, or ordinance-related cases, cities have been overburdened with processes requiring the co-ordination of multiple parties at the same location. Judges are also required to work efficiently through dockets containing hundreds of cases in a single sitting. The pandemic exacerbated what was already a fraught situation into something requiring an urgent solution.

Virtual Court leverages real-time collaboration and communication technologies such as WebRTC, and the scalability and resilience of AWS to provide cities with a reliable turnkey platform for conducting online face-to-face court sessions. Today, Virtual Court serves 40+ municipal courts representing close to 2 million citizens.

Research and Proof of Concept

Tyler has always strived to support and strengthen court communities. Even prior to the pandemic, Tyler was investigating ways to leverage the significant advances in browser and real-time communication technologies to empower courts to work more efficiently for the citizens they serve.

During introductory discovery sessions, Tyler identified a suitable milestone and, over the course of two months, built our first prototype using the .NET Core 2.2 framework hosted on Microsoft IIS server. This prototype used Twilio for video handling, and an Angular 7 frontend built atop the .NET Angular Single Page Application (SPA) template. The prototype was exposed to users over the internet, and the SignalR ASP.NET library was used for two-way communication between the frontend and backend.

Figure 1: Virtual Court – proof-of-concept architecture

This prototype was initially tested at the 2019 Tyler Connect conference, running on a tablet over the conference wireless network. This was a deliberate choice to assess the effectiveness of the video service over a limited bandwidth connection. While there were some issues, the overall user experience and qualitative performance of the technology stack exceeded expectations and suggested a plausible foundation on which to build a production-quality offering.

Evaluating platforms and infrastructure

Virtual Court was a greenfield project within Tyler Technologies, and therefore an excellent opportunity to build an application using state-of-the-art technology and infrastructure. Tyler Technologies had just committed to AWS as the primary cloud provider, offering two major choices for an infrastructure platform – an end-to-end serverless application stack, or the managed Tyler Cloud Platform (TCP), an internal platform effort based on the Amazon Elastic Kubernetes Service (EKS).

Over the course of a week, in collaboration with AWS Solutions Architects, Tyler rebuilt a portion of the Virtual Court prototype using serverless architecture (AWS Lambda, Amazon API Gateway and Amazon DynamoDB). In doing so, Tyler focused on a core document upload feature set in the application. The initial prototype stored documents directly on the IIS server filesystem and streamed them to the end user via the .NET service. Tyler switched to using the Amazon Simple Storage Service (S3) to temporarily store documents in a bucket, and the S3 pre-signed URL feature to retrieve and stream these documents to the user as necessary. This removed the need for persistent storage on the IIS host and offloaded the memory-intensive document streaming functionality to Amazon S3. The durability and scalability of the Amazon S3 service would also ensure that our application could easily handle growing requirements around document storage and streaming.

Figure 2: Using S3 pre-signed URLs for document retrieval and streaming in the Virtual Court POC

The other memory-intensive portion of this prototype involved maintaining WebSocket connections to end user clients. These are persistent TCP connections over TLS held in memory on the IIS server, and the RAM requirement grew linearly with user count. At the time, Amazon API Gateway had just announced support for the WebSocket protocol, and Tyler tested it with a JavaScript-based integration employing AWS Lambda and Amazon DynamoDB to store the state of events. Tyler used the ‘wscat‘ utility (a convenient tool for testing WebSocket APIs) to test this pipeline and found that this combination of API Gateway, Lambda and DynamoDB worked exceedingly well to handle real-time data operations.

The Tyler Cloud Platform is built around Amazon EKS as well as other open-source technologies such as HashiCorp Consul and the NGINX Ingress controller, and is managed by a dedicated team. The Tyler Cloud Platform team also manages all of the underlying shared infrastructure including the virtual machines, load balancers and VPC networking needed to operate the EKS clusters. In addition, the team provides ready-to-go templates and services to manage multi-tenancy, single sign-on authentication (via Okta) and dynamic configuration. Since the application ran within persistent containers on the Cloud Platform, Tyler did not have the scale-to-zero capability offered by a serverless architecture. However, the shared infrastructure and managed nature of the platform allowed  Tyler to offset this cost and take advantage of the utility services that are built and maintained by the TCP team.

Figure 3: Amazon EKS-based Tyler Cloud Platform architecture

Ultimately, the Tyler team wanted to re-use proven and well-understood architectural patterns (within the company) and avoid incidental complexity to the extent possible. Therefore, Tyler opted to build the Virtual Court product with an ASP.NET Core Backend-For-Frontend (BFF) architectural pattern, serving an Angular frontend from containers running on the Tyler Cloud Platform’s EKS offering. This made the best use of the extensive C# and .NET ability within the team and allowed the responsibility for maintaining the product to be spread across both the development and Tyler Cloud Platform teams.

The time spent in evaluating the serverless architecture approach was well worth it since Tyler opted to continue using DynamoDB for persisting operational data, and S3 pre-signed URLs for document storage and streaming. Virtual Court’s data model is relatively simple and has no relational query patterns, making the DynamoDB key-value store implementation an excellent choice for our real time use case. While there were some initial concerns around DynamoDB’s eventually consistent read model, load testing indicated that the propagation of data was fast enough for the use cases in question. S3’s ease of use, scalability, and cost-effectiveness made it an easy choice to persist with.

Building Virtual Court

Tyler designed the application to have three BFF instances, two primary instances for serving different sets of users and a third BFF for configuration. The two main BFF instances were built for government employees and officials who would control the workflow of the virtual courtroom, and the citizens and defendants who would participate in the court processes respectively. These BFF instances support separate SPAs and have different feature sets, authentication modes and levels of device support – for instance the citizen-facing BFF needs to support both mobile and desktop clients, while the employee-facing BFF supports only desktop clients. However, there is enough shared functionality across these two instances and SPAs that it made sense to build a shared Angular library encapsulating the common components.

Figure 4: Virtual Court production architecture and components

Some of the technologies used to build this application were on the cutting edge of development and availability. The Twilio Video APIs are an example – at the time of building the application, the Twilio Video JavaScript SDK was still in beta, but became generally available by the time Virtual Court went into production. In testing the Twilio Video API, the Tyler team had better results with the WebM video codec for streaming, since H.264 support was less than uniform across the targeted client devices. Other examples included WebRTC support in the iOS Safari browser and browser-based screen capture APIs. Tyler went down the path of building an own extension for screen sharing when these APIs became generally available.

The choice of a persistent container architecture allowed the Tyler team to utilize a background queue service pattern with the IHostedService interface in our .NET code. This allows for non-durable long running tasks to be queued up on a background worker thread, thereby making use of any idle CPU time in the container. These tasks can be restarted and resumed since they’re designed to be stateless. They run within the same container that receives the primary request, and are therefore distributed across all available containers.

Figure 5: Background task queueing in the BFF container

Based on work done on the proof-of-concept, Tyler decided to use SignalR as a core component for providing real-time communication capability in the application. The team used dependency injection with the application services in conjunction with the SignalR Hub functionality. These hubs allow for type safety in the PubSub implementation which made building the client-side code easier. SignalR also provides hooks for maintaining a user’s connection status in real time in the database. While SignalR has functionality to manage custom message protocols, Tyler settled on using the default JSON message format as the payloads are quite small.

An interesting feature of SignalR is its ability to scale out using the Redis in-memory data store and message broker as a backplane. Tyler made use of this functionality (via the Amazon ElastiCache for Redis service) not only for scaling out across a fleet of containers, but also for communication between the two primary BFF instances. This enables interactivity between different users across the entire system.

Figure 6: Scaling SignalR across BFF instances using an ElastiCache Redis backplane

Load Testing and Observability

The Tyler team utilized the AWS Distributed Load Testing solution to drive traffic to the Virtual Court application, verifying container stability, deriving baseline metrics, and preparing for the production release and general availability of the application. Part of the load testing goal was to establish and refine Key Performance Indices (KPIs) that could help quantify the end-user experience. Tyler uses Datadog pervasively for Application Performance Monitoring and Real-time User Monitoring during development and testing, and monitors these KPIs in the production environment to ensure user experience is up-to-par.

Impact and Conclusion

Tyler launched Virtual Court into production in the heat of the COVID-19 pandemic at a time that many courts were looking for viable solutions. By switching to the Virtual Court product for court proceedings Tyler’s clients were able to be more efficient when compared to other similar products on the market. In some cases, Virtual Court clients saw a greater than 60% reduction in failure to appear rates. Even today, they have replaced many of their in-person processes with more frequent remote hearings after seeing these benefits.

Tyler also benefitted from building and launching Virtual Court on top of the .NET framework and the AWS cloud by gaining a deeper understanding of the process. The freedom the Tyler team had in terms of picking AWS services and third-party components when building this application allowed them to arrive at a very flexible and scalable architecture. These learnings have been applied to Tyler’s ongoing efforts at modernizing the primary municipal court product, as the architecture translates quite well across multiple domain boundaries in the larger product. Having operated this architecture in production, Tyler better understands the risks and trade-offs involved – something that’ll serve the company well as it modernizes applications across its portfolio.

The choice of Tyler Cloud Platform/Amazon EKS as the basis for building Virtual Courts has paid off as it allows Tyler to immediately take advantage of improvements and contributions from across the company. The undifferentiated heavy lifting in terms of user management, authentication and other shared application functionality is correctly implemented once by the platform team and maintained by domain experts rather than application developers. This allows Tyler to evolve the Virtual Court application to take advantage of new features and functionality rapidly as they become available on the platform.

The Virtual Court application has now been in production use for the better part of two years. Due to the flexibility of the architecture, Virtual Court has been able to integrate with two other Tyler products in that timeframe. These integrations expanded the reach of the application past municipal courts into county-level and civil courts. In hindsight, the Tyler team believes the right choices were made in terms of development and infrastructure platforms (.NET and AWS) to provide an urgently needed solution to a pressing problem.

Author:

Zovin Khanmohammed is a Software Architect at Tyler Technologies where he spent the last 5 years working on Tyler’s Municipal Justice product out of Lubbock, TX. Zovin advocates for stream lining development and domain processes. Outside of work, Zovin spends his time cleaning his car and sitting out on the patio with a good cup of coffee.