Electronic Caregiver Builds Augmented-Reality Health Monitoring Using Amazon Sumerian

Electronic Caregiver, which has been all in on AWS since its founding in 2009, decided the time was right for home health monitoring to evolve. "We saw an opportunity to use the latest sensing, artificial intelligence, and other cloud-based technologies to address unmet customer needs with a fuller-featured solution than traditional alert devices," says Bryan Chasko, chief technology officer at Electronic Caregiver. "Our goal was a product that would be a better fit for today’s seniors and expand the walls of the medical practice by providing a more complete picture of senior patients' health."

The concept: An interactive voice platform, accessed via connected devices installed throughout the home, that would not just summon help if needed but also perform the ongoing health monitoring and treatment support that has traditionally required onsite human caregivers. To make this product as approachable and easy to use as possible, Electronic Caregiver decided the voice-controlled user interface would feature an animated augmented-reality (AR) character who would guide, prompt, and converse with users.

 

Creating an AR Character on Amazon Sumerian in Three Months

Initially, creating such a character was easier said than done. "For three years, we tried building an AR character from scratch," says Chasko. "We hired game developers and artists and worked in a AAA game engine. It was taking eight months to get through each build."

Then Electronic Caregiver learned about a new AWS service called Amazon Sumerian, which enables developers to create and run device-agnostic virtual reality, augmented reality, and 3D applications quickly and easily, without specialized programming or 3D graphics expertise. "We immediately saw that—right out of the box—Amazon Sumerian solved all the problems we’d struggled with for three years," says Chasko. "Within two hours of first getting access to Amazon Sumerian, our developers were building scenes."

Initially, creating such a character was easier said than done. "For three years, we tried building an AR character from scratch," says Chasko. "We hired game developers and artists and worked in a AAA game engine. It was taking eight months to get through each build."

Then Electronic Caregiver learned about a new AWS service called Amazon Sumerian, which enables developers to create and run device-agnostic virtual reality, augmented reality, and 3D applications quickly and easily, without specialized programming or 3D graphics expertise. "We immediately saw that—right out of the box—Amazon Sumerian solved all the problems we’d struggled with for three years," says Chasko. "Within two hours of first getting access to Amazon Sumerian, our developers were building scenes."

Just three months later, the company had completed an AR character named Addison, the centerpiece of Addison Care.

Customers use voice controls to interact with Addison via tablets installed throughout their homes that are tied to a network of wireless visual sensors, local AI-based processors,

Bluetooth biometric wearables, and emergency monitoring devices. Addison learns about the patient through constant observation, detects gait and other anomalies, and provides treatment and medication reminders through specialized modules for diabetes, heart disease, COPD, and other widespread health conditions.

Addison Care uses Amazon Lex for language understanding and Amazon Polly for lifelike text-to-speech. AWS Lambda, a serverless tool that runs code in response to events, performs pre-processing. The company takes advantage of Amazon DynamoDB to store health markers in addition to business-logic processes that are executed by AWS Lambda. The solution also uses AWS Amplify, a continuous delivery and hosting service for mobile web applications, to securely host and enable seamless navigation between its Amazon Sumerian scenes. Facial-recognition tasks are handled by machine-learning models that can run locally on the tablets in AWS IoT Greengrass, even when not connected to the internet.

 

For Electronic Caregiver, the benefit of Amazon Sumerian is how much easier it is now for small teams to make valuable improvements. "Using Amazon Sumerian, we have moved from a very segmented workforce—teams of 10 to 12 developers needed to execute on a build, and no real interdisciplinary support and collaboration—to now having two-person teams that can iterate very fast," says Chasko. "Because Amazon Sumerian enables us to move so quickly and integrate with other AWS services so easily, we're now averaging about 70 iterations per developer per day and extending front-end features thousands of times faster than before."

Amazon Sumerian requires no specialized knowledge or training, so Electronic Caregiver is also able to make better use of cross-disciplinary teams. "Amazon Sumerian is so simple to work with that even people with zero experience in 3D environments are making valuable contributions every day," says Taylor Bunker, a texture artist and scene builder for Electronic Caregiver. "We have artists working directly in Amazon Sumerian, programmers tackling art tasks, and biomedical researchers diving into the data. By enabling a truly multidisciplinary approach and freeing us from thinking about the nuts and bolts of AR and 3D design, Amazon Sumerian is helping us unlock much more value in our team."

Another reason Electronic Caregiver is so enthusiastic about Amazon Sumerian is the possibilities it opens for the future. "The simplicity of Amazon Sumerian—and its ability to integrate so easily with other AWS services—has unleashed our ability to innovate," says Chasko. "We can send prototypes across the world in a second and have someone testing right in their browser, without any dependencies or setup headaches."

Although Addison Care is still in a pilot phase, patients are already responding well—to the solution as a whole, and to Addison in particular. "In our trials, it's been really interesting to see how much more honest people are with Addison, compared to what they might be willing to share with a person," says Chasko. "The character we built using Amazon Sumerian seems to be striking just the right balance of professionalism and approachability, and that bodes really well for the success of this product."