Zocdoc builds patient confidence using TensorFlow on AWS

The healthcare industry is complicated. A recent survey showed that more than half of Americans have difficulty understanding their insurance coverage, and three-quarters want an easier way to check if doctors are in-network.

Zocdoc helps patients navigate this maze, allowing individuals with health care needs to make more informed choices and find care that matches their needs. Deep learning on AWS is at the core of Zocdoc’s mission to optimize health care data to help patients. With algorithms built using the TensorFlow deep learning framework, Zocdoc matches patients and doctors more efficiently. Patients can get appointments within 24 hours, compared to a national average wait time of 24 days for new patients.

“As a consumer-facing tech company operating in healthcare, we are eager to bring data driven innovations to improve the patient experience. Our search process uses multiple algorithms to parse a patient’s intent and match their needs to the right specialist,” says Serkan Kutan, CTO of Zocdoc.

Deep learning-powered search experience

With Zocdoc’s Insurance Checker, a patient just has to take a photo of their health insurance card. The system uses deep-learning-based computer vision to scan the ID card and extract the correct policy ID information. Faced with the difficulty of deciphering a wide variety of ID cards, Zocdoc’s engineering and data science teams were able to create a neural network proof-of-concept in just one day by using cloud-based GPU servers from AWS.

After extracting the member ID information, Insurance Checker then verifies the patient’s health coverage in real time, checks in-network benefits, and the estimated co-pay.

Even when patients understand their health plan coverage, there is often a mismatch between patients who wait weeks for appointments versus doctors who have more immediate openings. Zocdoc offers a machine-learning-based digital health marketplace that matches patients with doctors who are appropriate, in-network, and available.

“We use deep learning to classify insurance card images into carriers and plans and to also extract and read key text fields to help patients figure out their coverage and to find the best matching doctor,” explains Brian D’Alessandro, Zocdoc’s director of Data Science.

Under the hood

Zocdoc uses TensorFlow for its identification and matching system. The system uses computer vision and deep neural networks for simultaneous image classification, image cropping, and optical character recognition (OCR). Since OCR is a well-established area, the Zocdoc team sought to use existing services. However, they found that none were flexible enough to deal with variances in image quality in photos submitted by their customers—factors such as blurriness, size, background, and orientation.

The Zocdoc system uses three deep learning models, including a base convolutional neural network (CNN) classification model, an alignment model, and an OCR model. Each model trains algorithms with specific parameters that help the Insurance Checker product. For example, the base model for the neural network takes user images as inputs, and creates outputs consisting of carrier and plan IDs, location of the coordinates of the bounding box for the member ID, and the text orientation.

Using a combination of these models trained on millions of labeled patient insurance cards, Zocdoc was able to achieve accuracy of almost 90%. This was more accurate than the patients’ own input. At this point Zocdoc rolled the Insurance Checker solution into production. Through continuous training and availability of more customer data, the level of accuracy continues to improve.

Conclusion

Zocdoc now runs 100% on AWS. The company uses the AWS Deep Learning AMI to run their deep learning frameworks. The training was run on 8 GPU servers (Amazon EC2 p2.8xlarge instances) using the Keras library in Python with the TensorFlow backend for multiple days for each of the models. The team has found that prototyping capabilities for their customers using deep learning can be a fast and iterative process that helps them get to production-ready solutions.

About the Author

Cynthya Peranandam is a Principal Marketing Manager for AWS artificial intelligence solutions, helping customers use deep learning to provide business value. In her spare time she likes to run and listen to music.