Amazon probably hopes that Mechanical Turk will evolve to the point where large companies will jump in, dumping 6- and 7-figure HIT groups into the mix. But for now, it's a cottage industry on both ends of the transaction. Should Amazon do more to foster small-scale Turking? And (more critically for Amazon) are they stuck with it?
Corporate IT departments are strange creatures. As individuals, IT pros love to experiment, learn about new technologies, and use whatever resources are available to get things done quickly.
But group them together in an air-conditioned cube farm, and the collective instinct that results will steer them away trying something new. Your typical IT back office is very much like your father's Oldsmobile - full of sensible choices, industry standards, and minimized risks.
Why? Because they are financially accountable for every mistake they make. They are conservative in this environment because it makes sense to do so.
Big companies are not biting
So, here we have this new web service from Amazon, one which provides a highly-scalable human labor pool that is perfect for large corporations with huge projects and deep pockets. Undoubtedly, this is the cheese cake that Amazon is after, because when you're dealing with 1/2 cent commissions on HITs, you need millions of them make a profit.
But corporate customers are not biting yet - judging from what I see on the HIT lists, and the reactions I've gotten from my own clients.
Why? I think it's due to a couple factors.
Security
Almost any company that possesses millions of pieces of information is absolutely paranoid about their competitors getting access to it. Turk script writers know that fetching this info and reassembling it would be a cinch. Corporate IT knows it, too.
Predictable capacity
Second, large volumes of work require large volumes of throughput, or else your 4 million assignments will take about 4,000 days to complete. Right now, Amazon isn't too keen on releasing workforce volume data, or even a tachometer to see how fast HITs are getting done. Without this data, a corporate manager would be very uneasy about committing a large project to Mturk.
Yes, they could try a small project first - a pilot program to demonstrate the value of the process. But corporations like solutions tailored to their needs, and right now Mturk customization is entirely self-service - download the API docs and off you go. Not very inviting.
Behind door #2...
Fortunately, small businesses do not have these concerns. And thanks to systems like DPA's HIT Builder you don't even have to be technical to use the system, even for complex HITs.
So Amazon should make a genuine effort to cater to the 1- to 1000- HIT requester. Who knows, the repeat business could mount up quickly ;)
6 comments:
mTurk's Requester site also has a non-technical way of posting.
http://requester.mturk.com/
I can say, from my "inside knowledge" that mTurk truly believed "If we build it, they will come". I think they've changed their tune now as they have hired some new people specifically to get mTurk growing. Good news for us all.
As far as workforce data, thousands are registered but at most 500 unique workers are on in any given week. That would definitely go up if there was more work, but I'm sure more work won't come without more workers. (Amazon used to keep track of how many workers there were by hour, day, etc. but the intern they gave the assignment too has other stuff to do and apparently doesn't have time anymore... I bet he's the same guy that's fixing the accounting system. :)
Speaking of mTurk's requester site, you can't create or use qualifications... which, as a requester, I find completely useless. Hopefully they'll make that change soon.
Exactly - the requester console has just enough functionality to give you a taste, but once you see the possibilities of the system you quickly get frustrated with its limits.
DPA's HIT Builder is much more flexible, and gives you access to almost every API feature using a graphical interface. But right now, using it feels like the first time you learned to drive - lots of lurching and grinding of gears.
Yes, Hit Builder is a mess - their HIT cost calculator doesn't work (which is horribly frustrating), it gives errors all the time... it drives me nuts :) But I have no idea about the type of back-end coding needed to run it through the requester API. So I guess we're stuck with it, for now :)
HIT Builder usually gives me grief when I'm trying to build templates for the HIT questions. But I have to say that the developers at DPA are very responsive to support requests, and the system has been getting smoother every passing month.
There was another developer who was working on a simple Mac-based tool (i.e. a desktop app rather than a website) but I don't know how much progress she made. I'll do some poking around...
"Amazon used to keep track of how many workers there were by hour, day, etc. but the intern they gave the assignment too has other stuff to do and apparently doesn't have time anymore..." http://www.vcao.net
Post a Comment