Matt Arnzen

I recently downloaded the amazon mobile application for my blackberry. Part of the application is a service called “Amazon Remembers”.  The Amazon Remembers application can be used to create visual lists of things you want to remember while out and about.  Photos you take from the app are stored  on both the amazon app and the amazon.com site as reminders.  If the item you want is a product, Amazon will try to find and item for sale like the one in the photo. If they find a match, they will send you an email alert and post the result along with the original photo.

How do they do it? At first I thought amazon had figured out how to harness their huge cloud computing infrastucture to visally intrepret images. Well… it turns out it’s not that fancy. It’s ordinary folks around the world, working for pennies, using Amazon’s Mechanical Turk.

What is Mechanical Turk? Wikipedia describes it as as one of the suite of Amazon Web Services, a crowdsourcing marketplace that enables computer programs to co-ordinate the use of human intelligence to perform tasks which computers are unable to do. Requesters, the human beings that write these programs, are able to pose tasks known as HITs (Human Intelligence Tasks), such as choosing the best among several photographs of a storefront, writing product descriptions, or identifying performers on music CDs. Workers (called Providers in Mechanical Turk’s Terms of Service) can then browse among existing tasks and complete them for a monetary payment set by the Requester. To place HITs, the requesting programs use an open Application Programming Interface, or the somewhat limited Mturk Requester site.

How did it do?

I tried 2 tests to see how it works. My first test was to take a picture of the first thing on my desk; of a bottle of Mountain Dew. In no less than 2 minutes, the application returned a product of Mountain Dew flavored lip balm. Not too bad and pretty fast. The next test was to try a book; Amazon’s bread and butter. The picture I took is above.  This test again was very speedy, and 2 minutes later, I had a link to buy the  book from Amazon.

So far I’m really impressed.  I can see myself using this at brick and mortar stores to get pricing from Amazon for comparison shopping. I also frequently use their ratings and reviews so having that data handy would be great.

Nice work Amazon!

Below are some screen shots of the mobile application. If you want it for your blackberry, you can download it from Amazon’s site.

A product description within the Amazon app

Amazon product reviews