Amazon Rekognition announces updates to its face detection, analysis, and recognition capabilities

Today we are announcing updates to our face detection, analysis, and recognition features. These updates provide customers with improvements in the ability to detect more faces from images, perform higher accuracy face matches, and obtain improved age, gender, and emotion attributes for faces in images. Amazon Rekognition customers can use each of these enhancements starting today, at no additional cost. No machine learning experience is required.

“Face detection” tries to answer the question: Is there a face in this picture? In real-world images, various aspects can have an impact on a system’s ability to detect faces with high accuracy. These aspects might include pose variations caused by head movement and/or camera movements, occlusion due to foreground or background objects (such as faces covered by hats, hair, or hands of another person in the foreground), illumination variations (such as low contrast and shadows), bright lighting that leads to washed out faces, low quality and resolution that leads to noisy and blurry faces, and distortion from cameras and lenses themselves. These issues manifest as missed detections (a face not detected) or false detections (an image region detected as a face even when there is no face). For example, on social media different poses, camera filters, lighting, and occlusions (such as a photo bomb) are common. For financial services customers, verification of customer identity as a part of multi-factor authentication and fraud prevention workflows involves matching a high resolution selfie (a face image) with a lower resolution, small, and often blurry image of face on a photo identity document (such as a passport or driving license). Also, many customers have to detect and recognize faces of low contrast from images where the camera is pointing at a bright light.

With the latest updates, Amazon Rekognition can now detect 40 percent more faces – that would have been previously missed – in images that have some of the most challenging conditions described earlier. At the same time, the rate of false detections is reduced by 50 percent. This means that customers such as social media apps can get consistent and reliable detections (fewer misses, fewer false detections) with higher confidence, allowing them to deliver better customer experiences in use cases like automated profile photo review. In addition, face recognition now returns 30 percent more correct ‘best’ matches (the most similar face) compared to our previous model when searching against a large collection of faces. This enables customers to obtain better search results in applications like fraud prevention. Face matches now also have more consistent similarity scores across varying lighting, pose, and appearance, allowing customers to use higher confidence thresholds, avoid false matches, and reduce human review in applications such as identity verification. As always, for use cases involving civil liberties or customer sentiments, where the veracity of the match is critical, we recommend that customers use best practices, higher confidence level (at least 99%), and always include human review.

Now let’s look at some images to see how Amazon Rekognition handles the various aspects of challenging images captured in unconstrained environments.

Pose variations

This issue is encountered in faces captured from acute camera angles (like shots taken from above or below a face), shots with side-on view of a face, or when the subject is looking away. This issue is typically seen in social media photos (for example, when a subject is looking into the distance), selfies, or fashion photoshoots. Face detection algorithms have difficulty in detecting such faces because less than half the face might be visible in many cases, or the faces might be tilted at uncommon angles (like being upside down).

Image 1: Side-on view of faces

Image 2: Faces looking down at the camera at various angles

Image 3: Person looking into the sky and away from the camera

Difficult lighting

Lighting might be challenging due to low contrast, low light setups, or extreme contrast. This pattern is common in stock photography and at event venues. Face detection algorithms can struggle with such examples because there is either not enough contrast between face features and the background in low lighting, or, alternatively, face features can be washed out due to bright lighting, again making them difficult to discern.

Image 4: Bright lighting on face

Image 5: Low contrast and shadows on a face

Image 6: Extreme contrast

Blur or occlusion

This challenge is seen in photos that have artistic effects (selfies or fashion photos, video motion blur), partial occlusion by objects, paint or hair (fashion photography), or less-than ideal sharpness (photos taken from identity documents). All of the features of the face are not clearly visible clearly in such cases, so face detection is challenging.

Image 7: Face obstructed by hair

Image 8: Face obstructed by hands and other objects

Face detection and recognition updates are now available in all AWS Regions supported by Amazon Rekognition except AWS GovCloud  – US East (N. Virginia), US East (Ohio), US West (Oregon), EU West (Ireland), Asia Pacific (Tokyo), Asia Pacific (Mumbai), Asia Pacific (Seoul), and Asia Pacific (Sydney). To get started, you can try the latest version in the Amazon Rekognition console and refer to the documentation.

About the Authors

Ranju Das has been with Amazon for almost five years and leads Amazon Rekognition, a deep learning-based image recognition service which allows you to search, verify and organize millions of images. Before joining Amazon, Ranju worked at Barnes and Noble leading Nook Cloud engineering. His team was responsible for strategy, design, development and SaaS operation of Nook mobile services and Digital Asset Management Services.

Venkatesh Bagaria is a Senior Product Manager for Amazon Rekognition. He focuses on building powerful but easy-to-use deep learning-based image and video analysis services for AWS customers. In his spare time, you’ll find him watching way too many stand-up comedy specials and movies, cooking spicy Indian food, and trying to pretend that he can play the guitar.