DeepLens Family Assistant
Most of the challenges were DeepLens device related. We appreciate the help of AWS team patiently answering queries on forums and slack channel. The office hours really helped for almost a 1:1 format help by experts to solve specific issues. Here were some of the device related challenges:
- Registering Device - Setting up device took multiple tries with issues like default URL clashing with wifi subnet.
- Playing Audio - It took lot of help from AWS team and fellow participants to get end-to-end flow with audio played out from the device. Manual change of group.json file with each project deployment and setting up the python boto modules accessible to ggc_user were the major steps to get it working.
Accomplishments that we're proud of
We are really proud of creating something that will actually help people in need. Our application and solution will help those suffering from memory loss to be more social and be part of their family/friend circle. The application provides a tool in their hands which can be used by them independently.
What we learned
- Working successfully with several AWS services including Rekognition, Polly.
- Configuring the DeepLens device, setting it up to run Machine Learning models and extending to take action on inferences.
What's next for DeepLens Family Assistant
- Running the Assistant application without an internet connection or connecting to cloud. This will require creating custom face detection Machine Learning models to recognize faces using SageMaker and MXNet. Uploading it to device so that the face recognition inference happens all on the device itself. This will be a major upgrade that would make the device portable. The basic approach and lambda code could be reused with current DeepLens device or any similar device that can act as a true companion.
- Improving end-to-end performance of recognizing faces and playing the matching audio bio. #1 should help significantly with this.
- Securing to be able to associate a device to a particular patient and family.