Smart Video Buddy is the Silver Award Winner of the EuroITV Competition Grand Challenge !

Users collect and watch more and more video, and also require new strategies for interacting with the content they consume. We face this challenge by automatically linking videos with semantics: our Smart Video Buddy automatically analyzes video streams in real-time and recognizes semantic concepts. For example, a video scene is identified to show a “tennis match”.

This information is used to link videos with other content, which offers a variety of exciting applications:

• recommendation: The buddy offers an adaptive news feed and points you to other video clips or websites related to what you are just watching
• adaptive advertising: By automatically understanding videos, we can do a better job at recommending products of interest to you.
• search: Alternatively, we can use results of our detection to automatically index videos and make them searchable (see this demo)
• filtering: Finally, we can detect and filter unwanted content (such as “porn” or “violence”).

Overall, our technology enriches videos with semantics and makes them “smarter”.

Our technology performs an analysis of video content by feeding a scenes' motion, color, and texture to a variety of statistical learning algorithms. A key challenge lies in the training of these techniques, which usually requires a time-consuming manual annotation of training data. To overcome this problem, our research focuses on an autonomous learning from web portals such as Flickr or YouTube. This allows us to train highly scalable and flexible visual recognition systems.

You can find more information about our research project MOONVID here. Also, have a look at this flyer outlining our image and video mining research.

Check out other demos of our image and video mining research.

Smart Video Buddy is developed by the Multimedia Analysis & Data Mining (MADM) Research Group at the German Research Center for Artificial Intelligence (DFKI).

For more information, please contact Adrian Ulges.