Q: What is Amazon Rekognition?
Amazon Rekognition is a fully managed image recognition service powered by deep learning. Amazon Rekognition has been built by Amazon’s Computer Vision teams over many years, and already analyzes billions of images every day. Amazon Rekognition’s easy-to-use API detects thousands of objects and scenes, analyzes faces, compares two faces to measure similarity, and verifies faces in a collection of faces. With Amazon Rekognition, you can easily build applications that search based on visual content in images, analyze face attributes to identify demographics, implement secure face-based verification, and more. Amazon Rekognition is built to analyze images at scale and integrates seamlessly with Amazon S3, AWS Lambda, and other AWS services.
Q: What is deep learning?
Deep learning is a sub-field of Machine Learning and a significant branch of Artificial Intelligence. It aims to infer high-level abstractions from raw data by using a deep graph with multiple processing layers composed of multiple linear and non-linear transformations. Deep learning is loosely based on models of information processing and communication in the brain. Deep learning replaces handcrafted features with ones learned from very large amounts of annotated data. Learning occurs by iteratively estimating hundreds of thousands of parameters in the deep graph with efficient algorithms.
Several deep learning architectures such as convolutional deep neural networks (CNNs), and recurrent neural networks have been applied to computer vision, speech recognition, natural language processing, and audio recognition to produce state-of-the-art results on various tasks.
Amazon Rekognition is a part of the Amazon AI family of services. Amazon AI services use deep learning to understand images, turn text into lifelike speech, and build intuitive conversational text and speech interfaces.
Q: Do I need any deep learning expertise to use Amazon Rekognition?
No. With Amazon Rekognition, you don’t have to build, maintain or upgrade deep learning pipelines.
To achieve accurate results on complex Computer Vision tasks such as object and scene detection, face analysis, and face recognition, deep learning systems need to be tuned properly and trained with massive amounts of labeled ground truth data. Sourcing, cleaning, and labeling data accurately is a time-consuming and expensive task. Moreover, training a deep neural network is computationally expensive and often requires custom hardware built using Graphics Processing Units (GPU).
Amazon Rekognition is fully managed and comes pre-trained for image recognition tasks, so that you don’t have invest your time and resources on creating a deep learning pipeline. Amazon Rekognition continues to improve the accuracy of its models by building upon the latest research and sourcing new training data. This allows you to focus on high-value application design and development.
Q: How do I get started with Amazon Rekognition?
If you are not already signed up for Amazon Rekognition, you can click the "Try Amazon Rekognition" button on the Amazon Rekognition page and complete the sign-up process. You must have an Amazon Web Services account; if you do not already have one, you will be prompted to create one during the sign-up process. Once you are signed up, try out Amazon Rekognition with your own images using the Amazon Rekognition Management Console or download the Amazon Rekognition SDKs to start creating your own applications. Please refer to our step-by-step Getting Started Guide for more information.
Q. What APIs does Amazon Rekognition offer?
Amazon Rekognition offers APIs to detect objects and scenes, detect and analyze faces, compare faces, and search for similar faces in a collection of faces, along with APIs to manage resources. For details, please refer to the Amazon Rekognition API Reference.
Q: What image formats does Amazon Rekognition support?
Amazon Rekognition currently supports the JPEG and PNG image formats. You can submit images either as an S3 object or as a byte array.
Q: What image file sizes can I use with Amazon Rekognition?
Amazon Rekognition supports image file sizes up to 15MB when passed as an S3 object, and up to 5MB when submitted as an image byte array.
Q: How does image resolution affect the quality of results?
Amazon Rekognition works across a wide range of image resolutions. For best results we recommend using VGA (640x480) resolution or higher. Going below QVGA (320x240) may increase the chances of missing faces, objects, or inappropriate content; although Amazon Rekognition accepts images that are at least 80 pixels in both dimensions.
Q. How small can an object be for Amazon Rekognition to detect and analyze it?
As a rule of thumb, please ensure that the smallest object or face present in the image is at least 5% of the size (in pixels) of the shorter image dimension. For example, if you are working with a 1600x900 image, the smallest face or object should be at least 45 pixels in either dimension.
Q: In which AWS regions is Amazon Rekognition available?
Amazon Rekognition is currently available in US East (Northern Virginia), US West (Oregon) and EU (Ireland) regions.
Q: What is a label?
A label is an object, scene, or concept found in an image based on its contents. For example, a photo of people on a tropical beach may contain labels such as ‘Person’, ‘Water’, ‘Sand’, ‘Palm Tree’, and ‘Swimwear’ (objects), ‘Beach’ (scene), and ‘Outdoors’ (concept).
Q: What is a confidence score and how do I use it?
A confidence score is a number between 0 and 100 that indicates the probability that a given prediction is correct. In the tropical beach example, if the object and scene detection process returns a confidence score of 99 for the label ‘Water’ and 35 for the label ‘Palm Tree’, then it is more likely that the image contains water but not a palm tree.
Applications that are very sensitive to detection errors (false positives) should discard results associated with confidence scores below a certain threshold. The optimum threshold depends on the application. In many cases, you will get the best user experience by setting minimum confidence values higher than the default value.
Q: What is Object and Scene Detection?
Object and Scene Detection refers to the process of analyzing an image to assign labels based on its visual content. Amazon Rekognition does this through the DetectLabels API. This API lets you automatically identify thousands of objects, scenes, and concepts and returns a confidence score for each label. DetectLabels uses a default confidence threshold of 50. Object and Scene detection is ideal for customers who want to search and organize large image libraries, including consumer and lifestyle applications that depend on user-generated content and ad tech companies looking to improve their targeting algorithms.
Q: What types of labels does Amazon Rekognition support?
Rekognition supports thousands of labels belonging to common categories including, but not limited to:
- People and Events: ‘Wedding’, ‘Bride’, ‘Baby’, ‘Birthday Cake’, ‘Guitarist’, etc.
- Food and Drink: ‘Apple’, ‘Sandwich’, ‘Wine’, ‘Cake’, ‘Pizza’, etc.
- Nature and Outdoors: ‘Beach’, ‘Mountains’, ‘Lake’, ‘Sunset’, ‘Rainbow’, etc.
- Animals and Pets: ‘Dog’, ‘Cat’, ‘Horse’, ‘Tiger’, ‘Turtle’, etc.
- Home and Garden: ‘Bed’, ‘Table’, ‘Backyard’, ‘Chandelier’, ‘Bedroom’, etc.
- Sports and Leisure: ‘Golf’, ‘Basketball’, ‘Hockey’, ‘Tennis’, ‘Hiking’, etc.
- Plants and Flowers: ‘Rose’, ‘Tulip’, ‘Palm Tree’, ‘Forest’, ‘Bamboo’, etc.
- Art and Entertainment: ‘Sculpture’, ‘Painting’, ‘Guitar’, ‘Ballet’, ‘Mosaic’, etc.
- Transportation and Vehicles: ‘Airplane’, ‘Car’, ‘Bicycle’, ‘Motorcycle’, ‘Truck’, etc.
- Electronics: ‘Computer’, ‘Mobile Phone’, ‘Video Camera’, ‘TV’, ‘Headphones’, etc.
Q. I can’t find the label I need. How do I request a new label?
Please send us your requests through AWS Customer Support. Amazon Rekognition continuously expands its catalog of labels based on customer feedback.
Q. What is Image Moderation?
Amazon Rekognition’s Image Moderation is a deep-learning based easy to use API for detection of explicit and suggestive adult content in images. Developers can use this additional metadata to filter inappropriate content based on their business needs. Beyond flagging an image based on presence of adult content, Image Moderation also returns a hierarchical list of labels with confidence scores. These labels indicate specific categories of adult content, thus providing more granular control to developers to filter and manage large volumes of user generated content (UGC). This API can be used in moderation workflows for applications such as social and dating sites, photo sharing platforms, blogs and forums, apps for children, e-commerce site, entertainment and online advertising services.
Q. What types of explicit and suggestive adult content does Amazon Rekognition detect?
Amazon Rekognition detects the following types of explicit and suggestive adult content in images:
- Explicit Nudity
- Graphic Male Nudity
- Graphic Female Nudity
- Sexual Activity
- Partial Nudity
- Female Swimwear or Underwear
- Male Swimwear or Underwear
- Revealing Clothes
Amazon Rekognition’s Image Moderation API returns a hierarchy of labels, as well as a confidence score for each detected label. For instance, given an inappropriate image, Rekognition may return “Explicit Nudity” with a confidence score as a top level label. Developers could just use this to flag content. In the same response, Rekognition also returns second level of granularity by providing additional context like “Graphic Male Nudity” with its own confidence score. Developers could use this information to build more complex filtering logic.
Please note that the Image Moderation API is not an authority on, or in any way purports to be an exhaustive filter of, explicit and suggestive adult content. Furthermore, the Image Moderation API does not detect whether an image includes illegal content (such as child pornography) or unnatural adult content.
Q. Can Amazon Rekognition’s Image Moderation API detect other inappropriate content besides explicit and suggestive adult content?
Currently, Rekognition only supports the labels we have outlined above. We will work to continuously add and improve labels based on feedback from our customers.
If you require other types of inappropriate content to be detected in images, please reach out to us using the feedback process outlined later in this section.
Q. How can I ensure that Rekognition meets my adult image moderation use case?
Rekognition’s Image Moderation models have been and tuned and tested extensively, but we recommend that you measure the accuracy on your own data sets to gauge performance.
You can use the ‘MinConfidence’ parameter in your API requests to balance detection of content (recall) vs the accuracy of detection (precision). If you reduce ‘MinConfidence’, you are likely to detect most of the inappropriate content, but are also likely to pick up content that is not actually explicit or suggestive. If you increase ‘MinConfidence’ you are likely to ensure that all your detected content is actually explicit or suggestive but some inappropriate images may not be tagged. For examples on how to use ‘MinConfidence’, please refer to the documentation here.
In case Rekogntion fails to detect adult content in images, please reach out to us using the feedback process outlined below.
Q. How can I give feedback to Rekognition to improve its moderation?
Please send us your requests through AWS Customer Support. Amazon Rekognition continuously expands the types of inappropriate content detected based on customer feedback. It usually takes 6-8 weeks to add new types of explicit or suggestive adult content. Please note that illegal content (such as child pornography) will not be accepted through this process.
Q: What is Facial Analysis?
Facial analysis is the process of detecting a face within an image and extracting relevant face attributes from it. Amazon Rekognition’s DetectFaces API takes in an image and returns the bounding box for each detected face along with attributes such as gender, presence of sunglasses, and face landmark points. You can use facial analysis to analyze demographics and customer sentiment, or use detected face bounding boxes for further processing.
Q: What face attributes can I get from Amazon Rekognition?
Amazon Rekognition returns the following facial attributes for each face detected, along with a bounding box and confidence score for each attribute:
- Eyes open
- Mouth open
- Face landmarks
Q. What is face pose?
Face pose refers to the rotation of a detected face on the pitch, roll, and yaw axes. Each of these parameters is returned as an angle between -180 and +180 degrees. Face pose can be used to find the orientation of the face bounding polygon (as opposed to a rectangular bounding box), to measure deformation, to track faces accurately, and more.
Q. What is face quality?
Face quality describes the quality of the detected face image using two parameters: sharpness and brightness. Both parameters are returned as values between 0 and 1. You can apply a threshold to these parameters to filter well-lit and sharp faces. This is useful for applications that benefit from high-quality face images, such as face comparison and face recognition.
Q: What are face landmarks?
Face landmarks are a set of salient points, usually located on the corners, tips or mid points of key facial components such as the eyes, nose, and mouth. Amazon Rekognition DetectFaces API returns a set of face landmarks that can be used to crop faces, morph one face into another, overlay custom masks to create custom filters, and more.
Q: How many faces can I detect in an image?
You can detect up to 15 faces in an image using Amazon Rekognition.
What is Face Comparison?
Face Comparison is the process of comparing one face to one or more faces to measure similarity. Using the CompareFaces API, Amazon Rekognition lets you measure the likelihood that faces in two images are of the same person. The API compares a face in the source input image with each face detected in the target input image and returns a similarity score for each comparison. You also get a bounding box and confidence score for each face detected. You can use face comparison to verify a person’s identity against their personnel photo on file in near real-time.
Q: Can I use a source image with more than one face?
Yes. If the source image contains multiple faces, CompareFaces detects the largest face and uses it to compare with each face detected in the target image.
Q: How many faces can I compare against?
You can compare one face in the source image with up to 15 detected faces in the target image.
Q: What is Facial Recognition?
Facial recognition is the process of identifying or verifying a person’s identity by searching for their face in a collection of faces. Amazon Rekognition allows you to add facial recognition to your applications using the SearchFaces and SearchFacesByImage APIs. Using facial recognition, you can easily build applications such as multi-factor authentication for bank payments, automated building entry for employees, and more.
Q: What is a face collection and how do I create one?
A face collection is a searchable index of face feature vectors, owned and managed by you. Using the CreateCollection API, you can easily create a collection in a supported AWS region and get back an Amazon Resource Name (ARN). Each face collection has a unique CollectionId associated with it.
Q: How do I add faces to or delete faces from a face collection?
To add a face to an existing face collection, use the IndexFaces API. This API accepts an image in the form of an S3 object or image byte array and adds a vector representation of the faces detected to the face collection. IndexFaces also returns a unique FaceId and face bounding box for each of the faces added.
To delete a face to an existing face collection, use the DeleteFaces API. This API operates on the face collection supplied (using a CollectionId) and removes the entries corresponding to the list of FaceIds. For more information on adding and deleting faces , please refer to our Managing Collections example.
Q. How do I search for a face within a face collection?
Once you have created an indexed collection of faces, you can search for a face within it using either an image (SearchFaceByImage) or a FaceId (SearchFaces). These APIs take in an input face and return a set of faces that match, ordered by similarity score with the highest similarity first. For more details, please refer to our Searching Faces example.
Q. Does Amazon Rekognition use the data stored in a face collection?
No. Amazon Rekognition cannot access or use the data stored in a face collection without your permission. You have complete ownership over your data.
Q: How does Amazon Rekognition count the number of images processed?
For APIs that accept images as inputs, Amazon Rekognition counts the actual number of images analyzed as the number of images processed. DetectLabels, DetectModerationLabels, DetectFaces, IndexFaces, and SearchFaceByImage belong to this category. For the CompareFaces API, where two images are passed as input, only the source image is counted as a unit of images processed.
For API calls that don’t require an image as an input parameter, Amazon Rekognition counts each API call as one image processed. SearchFaces, and ListFaces belong to this category.
The remaining Amazon Rekognition APIs - DeleteFaces, CreateCollection, DeleteCollection, and ListCollections - do not count towards images processed.
Q. Which APIs does Amazon Rekognition charge for?
Amazon Rekognition charges for following APIs: DetectLabels, DetectModerationLabels, DetectFaces, IndexFaces, SearchFaceByImage, CompareFaces, SearchFaces, and ListFaces.
Q. How much does Amazon Rekognition cost?
Please see the Amazon Rekognition Pricing Page for current pricing information.
Q. Will I be charged for the feature vectors I store in my face collections?
Yes. Amazon Rekognition charges $0.01 per 1,000 face vectors per month. For details, please see the pricing page.
Q. Does Amazon Rekognition participate in the AWS Free Tier?
Yes. As part of the AWS Free Usage Tier, you can get started with Amazon Rekognition for free. Upon sign-up, new Amazon Rekognition customers can analyze up to 5,000 images for free each month for the first 12 months. You can use all Amazon Rekognition APIs with this free tier, and also store up to 1,000 faces without any charge.
Q. Do your prices include taxes?
For details on taxes, please see Amazon Web Services Tax Help.
Q: Does Amazon Rekognition work with images stored on Amazon S3?
Yes. You can start analyzing images stored in Amazon S3 by simply pointing the Amazon Rekognition API to your S3 bucket. You don’t need to move your data. For more details of how to use S3 objects with Amazon Rekognition API calls, please see our Detect Labels exercise.
Q: Can I use Amazon Rekognition with images stored in an Amazon S3 bucket in another region?
No. Please ensure that the Amazon S3 bucket you want to use is in the same region as your Amazon Rekognition API endpoint.
Q: How do I process multiple image files in a batch using Amazon Rekognition?
You can process your Amazon S3 images in bulk using the steps described in our Amazon Rekognition Batch Processing example on GitHub.
Q: How can I use AWS Lambda with Amazon Rekognition?
Amazon Rekognition provides seamless access to AWS Lambda and allows you bring trigger-based image analysis to your AWS data stores such as Amazon S3 and Amazon DynamoDB. To use Amazon Rekognition with AWS Lambda, please follow the steps outlined here and select the Amazon Rekognition blueprint.
Q: Does Amazon Rekognition store the images I submit for analysis?
No, not unless you provide us permission to do so. Amazon Rekognition APIs don’t store the images that are submitted for analysis. For face collections, Amazon Rekognition only stores face representations as face metadata in the form of feature vectors, and not as identifiable face image crops. Please note that images submitted through AWS Support for troubleshooting may be stored and used for training purposes, as described when you submit those images.
Q: Does Amazon Rekognition use submitted images for training purposes?
No, not unless you provide us permission to do so. If you would like to share your image datasets and provide training permissions to Amazon Rekognition, please contact AWS Customer Support for further details.
Q: Do I have to provide any training data to Amazon Rekognition?
No. Amazon Rekognition comes pre-trained to detect thousands of labels, to detect faces under a variety of conditions and with the ability to represent a face with a compact set of feature vectors. You don’t need to provide any training data or customize any model settings. Amazon Rekognition continues to improve its models and regularly adds to its catalog of labels.
Q: How do I control user access for Amazon Rekognition?
Amazon Rekognition is integrated with AWS Identity and Access Management (IAM). AWS IAM policies can be used to ensure that only authorized users have access to Amazon Rekognition APIs. For more details, please see the Amazon Rekognition Authentication and Access Control page.