Multi-modal analysis and Sentiment Analysis

A colleague pointed me to this Huffington Post article (which has embedded in it the TED talk “Deb Roy: The Birth Of A Word“) in a discussion about Sentiment Analysis.  I guess I have been dismissing all of the sentiment analysis discussions in the past (without really examining the ideas behind it).  I just couldn’t fathom how effective it could be — is it really better than just what a few people with some time on their hands could generate?

The TED talk started with a strange ‘experiment’ which involved wiring Deb Roy’ house with overhead video and audio in each room, and 200 terabytes of video recordings (he guess it’s probably the largest catalog of home movies).  Three years of recording, starting after the birth of his son, he brought the video to MIT and they started the analysis.  The types of analysis they are performing is extremely interesting — using multi-modal analysis to show correlation.  Proximity (spatially), socially (interaction), audio, and video all play into the analysis they performed.  Watch the video and I think you will be interested in what they are producing.

At ~12:30 min in the video, he describes how one of the MIT researchers on his team made the leap from a closed-space, controlled environment to the public-space.  Using public media (e.g. TV) as the video, and social media (e.g. Twitter) as the ‘audio’, they can start showing how the two relate much like they did with the home movies.  Social interconnectedness also was factored into the analysis.

Maybe I need to think a bit more about whether I should be dismissing these analysis ideas..


Leave a comment

Filed under Machine Learning

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s