Mechanical Turk

Thursday, November 03 2005

Today, by chance, I ran into Amazon's new web service Mechanical Turk that's gotten surprisingly little attention in Internet-land. I don't completely know how it works yet, but that's kind of neat cause I've been thinking about how it could work. What I do know, in short, is that Mechanical Turk (crap you can't really abbreviate MT for this -- it's already taken) is a web service that lets you make calls in your code to humans. They call it Artificial Artificial Intelligence. The idea is that you could write some request that in English would read like "Does the attached picture have a human in it" and submit it to MT (OK, I'm going to call it MT here at least) and then get back a response saying True or False just as if you were doing some simple comparison in a programming language (like 1==1 -> True). This happens because the request is handed over to humans to decipher the content of the image and humans can easily determine if the photo you're curious about has a human in it or not. It's almost like a scriptable Google Answers.

So, what's the motivator for humans to do this? Well, they get paid for it of course. And this is where I start to speculate because I haven't yet had time to look into exactly how the system works beyond what I've already mentioned, but the speculation is kind of fun. Anyway, we I know they get paid for answers. But while some answers are pretty black and white (like the example I just sited about the human being in the picture or not), some answers are subjective and others are just simply approximations. I'm wondering if the system could handle those type of questions/problems through a voting type method. Like maybe you could instead ask if that photo contains an attractive person or ask what that person's weight is judging just from the photo. That starts to be a really cool system with some really cool -- dare I say it -- Wisdom of Crowds type potential.

One of the issues though is the latency of the response. It's an asynchronous system meaning that you put in a request and then you check back for an answer later (presumably through another web service request). However, I would think the amount of money you set for satisfactory answers would determine the latency. If you ask whether or not a human is in a specified photo and offer $20 for an answer I would think you'd get one far faster than if you offered 3 cents. I wonder how close you could get to a realtime system if you used outrageous rewards for simple problems? I wonder if there's an adequate pricing/latency point somewhere in the middle?

So it's some potentially cool new technology (courtesy of a Bookstore let's not forget!). I'll look into it more tomorrow and see if any of my speculations have any component in reality.