AWS News Blog

A Walk Through Hit-Builder.com

Earlier today I spent some time using Hit-Builder.com and found myself very impressed with the site and with what I was able to accomplish.

The site implements a complete, requester-oriented interface to the Amazon Mechanical Turk. Using Hit Builder, it is possible to create and issue a series of HITs (Human Intelligence Tasks) with just one visit to the Amazon site. There’s no need to deal with source code or with raw XML data.

Before getting started, you need to create and fund a Mechanical Turk Requester account. This is done on the Turk site, and would be necessary whether or not the Hit Builder will be used to create and manage HITs. Note that there’s a time delay (usually 3 or 4 business days) between the time when you fund the account and when the funds are available within the account.

Next, you need an AWS Subscription Id and the accompanying Secret Key, which you can get by registering here. Once you’ve done this, you can log in to the AWS Developer Portal, and click on the button labeled “Your Web Services Account”:

Account

Click on “View Access Key Identifiers” and copy down the two keys that you see there (or open up another browser tab).

Moving right along, visit the signup page for the Hit Builder and create your account:

Hb_signup

Once you’ve done this, you can sign in and get started. Use the Manage Account tab to enter your Amazon Subscription Id and Secret Keys:

Hb_keys

By the way, those aren’t my real keys, so don’t strain your eyes trying to figure them out! While you are within the Manage Account tab, you should go to the Account Summary page and verify that your account has been funded —  look for the line labeled “Amazon Payments account balance.”

Ok, time to build the HIT! Visit the Build HITs tab, and select Build HIT Group:

Hb_build_1

From here on out, you need to think of the HIT Builder as a classical desktop application, not as a web site. There are 2 layers of popup windows, and much of the navigation is accomplished by pushing buttons rather than by clicking on links. The UI is smooth and easy to use once you have a good grasp of the elements of a HIT.

At this point it is a good idea to step away from the keyboard and to spend a few minutes designing your first HIT. Good HITs are self-contained, unambiguous, and carefully described. We have seen that new Requesters often interate on the HIT design in order to fine tune the model and to weed out any ambiguities.

My HIT is going to be really simple. I am going to ask people about their favorite vacation activity, which they must select from the following list of choices in a “radio group”:

  • Stay home
  • Sit on the beach
  • Take a cruise
  • Drive somewhere
  • Go skiing
  • Build a web site
  • Other

Start filling out the form by entering a title and a description for your HIT. When it is time to create the overview, click the “Add” link and enter the overview into the popup:

Hb_overview

The next step is definitely the most complex. It is time to actually build the HIT; start by clicking the “Add New HIT” link. This will open up the Add/Edit HIT popup. Enter in some text to introduce the question (in my case I typed “What is your favorite vacation activity?”). Now use the “Add Answer Control” option at the bottom to choose a UI element to collect the answer. I chose “Radio Button – Text”, and then filled in the popup:

Hb_options

For each item in the radio group, I entered two values. The first value is the “pretty” value for display, and the second is the value to be returned as the result value for the HIT. If you are going to be pulling this data into another application, you can use this feature to supply that application with the exact set of values needed for it to proceed; there’s no need for any kind of mapping layer. I also created a free text answer in case the worker chooses “Other.”

The last step is really easy. We need to specify how many workers get to work on this HIT, how much they’ll get paid, and how long the HIT will be active.

Hb_hit_parameter

We could also attach Qualifications to the HIT, but that’s not necessary in this case. The Hit Builder makes it very easy to build, manage, and score Qualifications.

I’m going to ask 50 workers to do this HIT, and I’ll pay each of them 2 cents. I fill in these values and click “Post HITs”, and the Workers will get right to work.

At this point switch over to the Manage HIT Results tab, and choose your HIT Group and your Reviewable HITs:

Hb_manage

As the HITs come in, the Hit Builder interface can be used to review, approve, and even chart them:

Hb_graph

 

Overall, this is a remarkable handy site, with tons of easy to use functionality. Check out the Hit Builder today!

— Jeff;

Jeff Barr

Jeff Barr

Jeff Barr is Chief Evangelist for AWS. He started this blog in 2004 and has been writing posts just about non-stop ever since.