Using AI to rethink document automation and extract insights

Documents have come a long way from being inscribed with ink on papyrus or scratched in runes by ancient civilizations. They are now a fundamental tool of modern life, with documents used to capture and record essential information in many ways including application forms, certificates and licenses, purchase orders and invoices, and legal contracts.

While digital transformation made strides in automating many processes, automating document management and entry has not been done until recently. The maturing of artificial intelligence (AI) has brought ready-made services that organizations can use, not only to automate data entry work but also to apply intelligence into the business process. Using modern AI capabilities on Amazon Web Services (AWS), organizations can transform approaches to document management. This allows public sector organizations to save time (enabling faster throughput especially during higher volume paperwork times), so they can help get constituents their services faster, and focus on the most valuable work of the high-touch or high-need cases. Document automation helps reduce human entry error and provide backup services in case of natural disaster.

Extracting insights from huge volumes of historic documents: An Australian federal agency and Cloudten

Cloudten, an AWS Advanced Consulting Partner, worked with an Australian federal agency to automate the ingestion and analysis of scanned documents, and to gain deeper business insights by helping extract the intent and sentiment of processed data. The agency held nearly a million pages of historic scanned PDFs and JPGs of archived data that needed to be ingested and processed. CloudTen delivered a solution designed around Amazon Textract, as well as a range of other cloud-native serverless services such as AWS Lambda to extract, process, and present analyzed data so that it delivered insights with the benefits of scaling in line with the workload.

In addition to using Amazon Textract to convert scanned images into digitized text, the solution also expanded the data ingestion pipeline to incorporate machine learning (ML) capabilities that facilitated natural language processing (NLP). This includes using Amazon Comprehend and Amazon SageMaker to perform advanced pattern analysis and interpretation on the extracted dataset to deliver actionable insights. The agency’s officers were quickly able to extract detailed information and intent from archived business documents that previously required lengthy manual analysis.

Improving citizen services with AI driven form processing: Firemind

Firemind, an AWS Partner based in the United Kingdom, helped local government agencies, such as Maidstone Borough Council and Tunbridge Wells, improve staff efficiency using AI to automate mundane aspects of their workflow, like having to rotate images that arrive in the wrong orientation and then copy key information from scanned forms. Everyday, their citizens upload documents, such as photographs of completed applications related to local parking and housing. In turn, the council’s staff review the uploaded forms and extract key information to drive the relevant business processes to resolution and deliver the citizen services.

Firemind automated this workflow by using Amazon Textract to analyze the uploaded documents and give immediate feedback to the citizen if they’ve mistakenly uploaded the wrong photograph, so they don’t need to wait days until a human worker reaches the item in the queue and flags their error. Once the correct image is processed, Amazon Textract pulls the key information from the form, including the name, address, and vehicle details automatically, and identifies the rotation of the document based on the text orientation. If its the incorrect orientation, it flips the document a number of ways until the service determines that the document is the correct orientation.

The Firemind solution reduced the number of manual tasks that council staff needed to undertake by one-third, enabling them to become more time efficient and focus on more value added work as well as providing the key services to the citizens more quickly.

Supporting citizens through COVID-19: Arizona State University Cloud Innovation Center (CIC)

The Arizona State University Cloud Innovation Center (CIC) built an open source asset to refine the document processing technology of Amazon Textract for utility bill and drivers license data extraction. This solution was recently used by Wildfire, a state association for Community Action Agencies, and Prefix Health Technologies (Prefix), an AWS Partner Network (APN) Technology Partner, to help provide relief to citizens during the COVID-19 pandemic.

The Arizona benefits portal allows COVID-19 impacted households to pre-screen and apply for assistance with rent, mortgage, gas, electric, and water. Applicants can attach document images to the benefit applications using their mobile phone camera. Amazon Textract captures the data from the images and populates or verifies the data entered, which eliminates the need for manual verification and speeds up the processing time. In many cases, eligibility is determined at the point of entry and funds are credited to the customer’s account with little or no delay. For additional details on the solution developed read, “A streamlined, mobile-first approach to service delivery for counties and states” where the solution they developed for Arizona resulted in 49% of the applications to be automatically approved, therefore reducing the time required to verify and distribute funds.

Why Amazon AI and ML services for document automation

Using AWS AI and ML services to automate document processing helps organizations:

• Reduce effort tailoring to each form type: Amazon Textract ingests and reads documents and forms without requiring any extensive pre-work to understand the form’s layout. Instead, the AI-based approach understands the content based on the physical layout, even extracting the data held in tables or forms and mapping that into machine readable structures to indicate what has been written in each part of a form by mapping those values to their respective data fields.
• Scale up and down as needed: Business operations are often challenged by managing peaks in demand, for example, during application deadlines or during events like the COVID-19 pandemic. Scalability and modern serverless cloud architectures are key, which help quickly ramp up to process large volume of documents and then to immediately scale down, minimizing the on-going costs.
• Combine human and AI expertise to confirm or correct data entry more easily and quickly: Tightly integrated augmented AI flags to a human reviewer the aspects of forms which the AI couldn’t read confidently. The combination of AI and a human working together delivers a highly robust approach to efficiently automating a document workflow.
• Recognize sustainability benefits: Organizations can reduce the carbon and energy expended in physically moving tons of physical paper documents between sites, and then storing the same in physical archives. Shifting to electronic document processing, with digital mailrooms ingesting and scanning the media, liberates the workforce away from being physically co-located with the documents. A wider shift for the workforce that AI brings, is the ability to rely on the AI for the mundane tasks and allow the human workforce to focus on more value adding tasks that require uniquely human skills.
• Extract more value from data to improve processes: Using ML techniques also raises the bar on how much value can be extracted from documents. Amazon Rekognition is used to identify and extract images or diagrams embedded within documents, saving time and manual effort by identifying and cropping out images. The text within documents is processed through services Amazon Translate, making it possible to support 55 languages and variants from Afrikaans to Vietnamese, without requiring in-house translators. Amazon Comprehend uses natural language processing techniques to help understand a document. This is often used to triage inbound correspondence by understanding the nature of the request, and directing the task to the best work queue. These can be fed directly into robotic process automation driven workflows to partially or fully undertake work that would require human teams.
• Gather data insights to improve services: Extracted data can be pushed into a graph database, such as Amazon Neptune, for subsequent network analysis. This approach helps detect application fraud where networks of associates, addresses, and businesses are identified from the graph that might be otherwise very hard to recognize.

Getting started

To learn more about how Amazon Textract can be used to help with document automation, check out our webinar, “Automating document management and improving search and discovery,” and review the Getting Started Guide, the technical how-to blog post, “Building an end-to-end intelligent document processing solution using AWS,” and the Service Overview.