New Mechanical Turk Categorization App

Categorization is one of the more popular use cases for the Amazon Mechanical Turk. A categorization HIT (Human Intelligence Task) asks the Worker to select from a list of options. Our customers use HITs of this type to assign product categories, match URLs to business listings, and to discriminate between line art and photographs.

Using our new Categorization App, you can start categorizing your own items or data in minutes, eliminating the learning curve that has traditionally accompanied this type of activity. The app includes everything that you need to be successful including:

1. Predefined HITs (no HTML editing required).
2. Pre-qualified Master Workers (see Jinesh’s previous blog post on Mechanical Turk Masters).
3. Price recommendations based on complexity and comparable HITs.
4. Analysis tools.

The Categorization App guides you through the four simple steps that are needed to create your categorization project.

First, you create the project, assign a name to it, and enter the question that you want to ask of the workers.

Next, you provide the categories that the Master Workers will use. You simply enter the names and the App will generate the HTML for you. You also have the opportunity to supply instructions as part of the step.

The next step is to upload the data to be categorized. You simply upload a CSV file and select the fields that you’d like the Workers to see. If one of your fields contains links to images to be displayed as part of the HIT, you can also set that up in this step.

Finally, you review the pricing information and the total cost for your HIT. Once everything looks good, you go ahead and Publish the HITs and await the results.

Two Master Workers will handle each HIT. After the Workers have finished all of the HITs in the project, you can use the Result Analysis tools to see a summary of the results. You’ll be able to see how often both of the Master Workers agreed on a categorization, and how often they disagreed. You can also download the actual categorization data for each of your items.

Read more about this feature on the Mechanical Turk Blog.

— Jeff;