Mechanical Turk Case Study: HIT-Builder

Today, we build complex software applications based on the things computers do well, such as storing and retrieving large amounts of information or rapidly performing calculations. Yet, people still significantly outperform the most powerful computers at completing such simple tasks as identifying objects in photographs. While these tasks call for a human touch, the cost of establishing a group of experienced people to do the work often outweighs the value of completing it.

To combat this problem, Amazon.com created Amazon Mechanical Turk (AMT). AMT provides a programmable interface to a vast network of skilled workers and incorporates the resulting human intelligence into developers’ applications. To the application, the transaction looks very much like any remote procedure call: the application sends the request, and the service returns the results. Behind the scenes, a network of people are checking the AMT site, searching for work, completing tasks, and receiving payment from the project requester via an Amazon.com account. When work is approved by the requester, payment remains in the Amazon.com account to use against future purchases or transfers to a U.S. personal bank account.

Typically, requesters need a basic understanding of how to use APIs or need to hire a software developer to submit work to the AMT Web site. DPA Software is removing this barrier to entry with HIT-Builder. The service helps companies enter their work into the AMT system—without writing a single line of code.

During the Beta test period, there is no charge to use HIT-Builder. Developers just setup a Requester Account at Amazon.com. They can then create HITs (Human Intelligence Tasks) in an Internet Explorer browser and import database and spreadsheet files to setup high-volume applications. To aid beginners, HIT-Builder supports a Wizard mode for the development of simple HITs for AMT. HIT-Builder also supports a Pro mode that provides access to qualification and other advanced features.

Once HITs have been submitted to AMT, the HIT Manager tab enables requesters to review the state of any HIT assignment. Then when HITs have been processed by workers using AMT, requesters can review the results as tabular data, graphed data or export the answers to a spreadsheet. HIT-Builder will be offering one of four plans to pay for HITs submitted to AMT. The most basic plan adds a nominal surcharge to each HIT submitted. High volume plans reduce the per-HIT charge in favor of a monthly subscription amount.

“Instead of humans driving the process by inputting work that a computer does, a computer program is assigning tasks to people and returning the results,” said David Pfeiffer, co-founder, DPA Software. “With Amazon Mechanical Turk, we are able to change the way people interact and receive services from the Web. It is a revolutionary idea with great potential and applications.”

Key applications include transcription, translation, editing/proofreading, optical character recognition/handwriting, content analysis and image recognition. Requesters can rest assured that work will always be handled by a skilled worker. By opting to require a qualification test, requesters have the opportunity to vet the skill level of the people working on their HITs beforehand.

Prior to selecting AMT, DPA Software considered several other solutions that proved reliable for only low volume decisions, but couldn’t scale for large projects. DPA Software chose AMT for its ability to handle high volumes at low cost. The company now offers a complete set of HIT development services including project consulting, HIT development using HIT-Builder, website applications that use AMT and custom AMT web services programming.

For more on HIT-Builder, go to http://www.hit-builder.com/ This link will launch in a new browser window or tab.