AWS Public Sector Blog

Harnessing the power of the cloud to design personalized healthcare solutions

man on healthcare app on iphoneGuest post by Andreas Jakl

Data is at the heart of healthcare, but our wellbeing is complex. It is a challenge to find the right approach when analyzing or visualizing data. Technology like text-to-speech, augmented reality (AR), and deep learning can help us better understand and explain health data. By designing solutions in the cloud, we can leverage Amazon Web Services (AWS) to launch and scale helpful tools as needed, paying for only what we use while keeping data secure.

AR and neural speech for personalized health education

Imagine you are waiting for an upcoming surgery. Traditional patient education often includes printed, non-personalized sheets of paper that explain what is about to happen. By developing solutions in the cloud, we can use technology to make complicated concepts easier to understand during consultations.

We developed an app to serve as a complementary tool during patient education for an eye surgery for strabismus. Virtual three-dimensional holograms showing models of your head and eyes project onto the table in front of you. Then, animations walk you through the individual steps of the surgery.

We used cloud-based neural text-to-speech services including Amazon Polly, which reads personalized and translated information in a natural-sounding way. Hearing the explanation in your native language removes another barrier of comprehension. Results of the study indicated that this app helps patients understand the process, which is the basis for making a well-informed decision.

Experiment: Amazon Sumerian for 3D visualization

However, three-dimension (3D) apps usually require advanced development skills plus the corresponding medical background. At St. Pölten University of Applied Sciences, we try to bridge these worlds in the information visualization class. Health students come with only a brief introduction to software development and get the task of visualizing their medical expertise in 3D.

With Amazon Sumerian as a browser-based visual interface, health experts quickly create 3D scenes and focus on explaining their topic through an animated avatar called a host. This virtual person speaks to users with the latest neural voice technology. The Digital Healthcare Explained AR app for Android combines some of the students’ efforts and won the Best AR App award at the DevPost Amazon Sumerian Hackathon.

Learning through voice

To reach an even broader audience, smart speakers are a great channel because users simply interact with their voice. In the voice learning skill, students can easily repeat the most important concepts of classes. Answering questions can improve retention for long-term memory compared to simply reading course materials repeatedly.

Especially in today’s online teaching landscape, where students spend most of their time in front of computer screens, using a different channel for learning helps a lot. During a pilot study of the beta version with students at our university, we found that more than 87 percent agreed that the skill helped them focus on relevant questions and topics. The Alexa skill is available as open source code on GitHub, and you can adapt it to your own education needs.

Understanding health data with deep learning

Universities are integrating deep learning into their curriculums to prepare the next generation of health experts and data scientists. One of their obstacles is the immensely high requirements on computers for learning from data. Usually, student’s laptops are not up to the task, for example of detecting and classifying tumors from MRI images.

However, real-life examples are important so that students can fully understand the individual steps to bringing data to actual insights. With cloud-based machine learning environments like Amazon SageMaker, all students can succeed at the tasks, no matter how powerful their personal computers are.

Apply it to your courses

Health and wellbeing apply to everyone, which motivates students to apply the latest technologies to improve our lives. For some ideas, you don’t need prior development skills to realize your first prototypes. Can you think of a way to explain health in 3D through Amazon Sumerian? How can Amazon Alexa help offer advice for wellbeing? Use the powerful infrastructure of Amazon SageMaker to give your students a taste of what’s possible at no cost through AWS Educate.

Andreas Jakl

Andreas Jakl

Andreas Jakl is lecturer for digital healthcare at the St. Pölten University of Applied Sciences, focusing on mobile technologies, augmented and virtual reality, wearables, and machine learning (ML). The St. Pölten University of Applied Sciences (UAS) in Austria was founded in 1993, offering 26-degree programs. Though the university is relatively small in size, it is a leader in the region, known for its focus on innovation. The St. Pölten UAS is globally connected and has a mission to pursue future-oriented academic education in diverse subject areas, while functioning as an important center of research.