Jiann
Inspiration
The world is a dangerous place, and isolating ourselves behind walls and cages in the name of safety obstructs crucial moments of cultural exchange. Workplace violence traditionally follows this destructive pattern in statistically high-crime locations. Gas stations, outlet malls, bodegas, and express stores install obstructive cages between customers and their workers to enhance safety. The move improves workplace safety, but at a significant cost to the community. Jiaan is an opportunity to change traditional reactions, workplace culture, and community for the better. The intelligent security system identifies threat in real-time and allows for action to be made on that threat. My goal is to reduce and ultimately eliminate prolonged violent encounters that would otherwise require the employee to notify authorities after the fact.
What it does
Jiaan detects and classifies objects in a live feed and measures a cumulative threat level that will trigger a configured response. Should the system detect an elevated threat level as a result of the presence of a handgun, a notification may be immediately delivered to authorities, security agencies, management, and even to on-site employees. The system significantly reduces the time it takes for personnel and / or authorities to be notified of a potentially violent encounter while also disseminating threat notifications to increase on-site awareness.
Created By: Matthew Pettit
How I built it
A fine-tuned model was trained off the VGG16 Reduced model using Single Shot MultiBox Object Detection on 2.6GB of annotated images that depicted both violent and non-violent interactions. Once the model was producing satisfactory results for this project, the custom model was optimized with Intel's DeepLearning Toolkit and ported to the DeepLens device.
The micro services on Amazon's Lambda Functions were developed in Python to pre-process, interpret, and act on the live feed from the DeepLens device to measure a threat level driven by a calculation of the accuracy of the object detected and the classification of the object.
Challenges
The second challenge was planning training cycles with unique datasets for the project. Training required acquiring a significant collection of new assets, annotating those assets, and training the model on those assets. Collecting and annotating a measurable data set required 100+ hours of manual work. Training the model on the assets took 500+ hours, which had to be mitigated with cost.
Navigating around some of the issues with Intel's model optimizer and inference engine consumed a lot of time that would otherwise have been spent on enhancements. Much of the final week was spent reverse engineering Intel's framework, fixing bugs in the optimizer, and flashing the DeepLens device. The experience, however, proved educational, and I now have a detailed understanding of Intel's deep learning toolset.
Accomplishments that I'm proud of
- Building a functional SSD r-CNN-driven model\
- Broadening the scope of my knowledge to include mxnet and SageMakers' APIs with only prior experience with OpenCV and Numpy
- Acquiring a greater understanding of the technologies underlying Greengrass, DeepLens, and Sagemaker
- Uncanny patience while annotating 1000s of images
What I learned
Building an intelligent application driven by DeepLens, or any ML project, is as much of a project management task as it is a development task. There are a number of variables that affect resource capacity and may result in unexpected downtime that must be addressed as part of planning.
What's next
Jiaan will be expanded to include a broader scope of threat classifications suitable for a live environment where it can be implemented and donated to the community.
Built with
deeplens
python
greengrass
lambda
sagemaker
iot
ssd