word.camera Turns Your Photo Into Paragraphs
word.camera is a curious new website that can magically transform a photograph into words, sentences, and paragraphs that describe what’s seen in your image.
Visit word.camera, and your browser will ask for access to your webcam. Once you provide it, you can frame a shot using the live view box on the screen and then press the red camera button to snap a photo.
The system then spends some time analyzing your shot and turning it into words.
After a minute or two (or up to 5 minutes), the site will direct you to a dedicated page that contains your image and the text that was generated based on it.
The site performs quite admirably on some photographs. In one test it was able to detect a hand holding a phone.
But in another test it had trouble with a printed photo of a camera lying in grass.
Goodwin says the technology behind the website involves an API provided by Clarifai, a company that is using a “convolutional neural network” to translate pixels of photographs into English. The tags generated by the API are then expanded into sentences and paragraphs using ConceptNet.
If you’d like to try the system on existing digital photographs of yours, just sit tight: Goodwin says he’s working on adding the ability to upload any photograph instead of having to take a photo on the spot with your webcam.
Update on 4/9/15: Goodwin tells us he has added a file upload input to the website.
