Category Archives: Machine Learning

Multi-modal analysis and Sentiment Analysis

A colleague pointed me to this Huffington Post article (which has embedded in it the TED talk “Deb Roy: The Birth Of A Word“) in a discussion about Sentiment Analysis.  I guess I have been dismissing all of the sentiment analysis discussions in the past (without really examining the ideas behind it).  I just couldn’t fathom how effective it could be — is it really better than just what a few people with some time on their hands could generate?

The TED talk started with a strange ‘experiment’ which involved wiring Deb Roy’ house with overhead video and audio in each room, and 200 terabytes of video recordings (he guess it’s probably the largest catalog of home movies).  Three years of recording, starting after the birth of his son, he brought the video to MIT and they started the analysis.  The types of analysis they are performing is extremely interesting — using multi-modal analysis to show correlation.  Proximity (spatially), socially (interaction), audio, and video all play into the analysis they performed.  Watch the video and I think you will be interested in what they are producing.

At ~12:30 min in the video, he describes how one of the MIT researchers on his team made the leap from a closed-space, controlled environment to the public-space.  Using public media (e.g. TV) as the video, and social media (e.g. Twitter) as the ‘audio’, they can start showing how the two relate much like they did with the home movies.  Social interconnectedness also was factored into the analysis.

Maybe I need to think a bit more about whether I should be dismissing these analysis ideas..


Leave a comment

Filed under Machine Learning

Awesome Tool which (I am sure) uses Machine Learning

Someone recently threw out a challenge to identify the artist of a particular painting /photograph which was hanging on the wall in the movie Iron Man 2.  What she provided was this image:

Iron Man 2 Office Art
‘unknown’ art from Iron Man 2

This appears to be two frames from the movie?  I can’t recall seeing it in the movie, but ultimately it didn’t matter, since I wasn’t going to go down that path.  (I assume she already tried that avenue, googling ‘iron man 2 office art’ or something similar).

With machine learning on the brain, I realized this was clearly a machine learning problem.  Thankfully, I remembered I had already found an existing tool to do just this — take an input image and find other images which are the same.  Here is where I first talked about TinEye.
All I did was pop the image into photoshop and chop off the right side, since I expected to match to a head-on image of the art and not a combined graphic.  That being done, I uploaded the graphic to TinEye and viola it returned:
From this website, it appears it is a 1997 photograph of Tonnay, France’s Esser River by German photographer Ursula Schulz-Dornburg.  It looks like the photo was in Lichfield Studios in Notting Hill.
I spent a second trying to validate the information, but the artist’s site doesn’t have all her works posted.
Still, I think there is fairly high likelihood that I have found the artist.
 (BTW, it took me way longer to write the blog post than to do the search.. )

Leave a comment

Filed under General, Machine Learning