Large-scale Visual Sentiment Ontology and Detectors Using Adjective Noun Pairs

Damian Borth, Rongrong Ji, Tao Chen, Thomas Breuel, Shih-Fu Chang
ACM MM 2013, Barcelona, Spain, ACM, ACM, 10/2013


We address the challenge of sentiment analysis from visual content. In contrast to existing methods which infer sentiment or emotion directly from low-level features, we propose a novel approach based on understanding the semantics of images. Our key contribution is two-fold: first, we present a psychology theory based method to automatically construct a large-scale Visual Sentiment Ontology (VSO) consisting of more than 3,000 Adjective Noun Pairs (ANP). Second, we propose SentiBank, a novel mid-level representation framework built upon the VSO encoding the concept presence of 1,200 ANPs from visual content. The Visual Sentiment Ontology and SentiBank are distinct from existing works and will open a gate towards various high-level applications. Sentiment analysis experiments on real-world Twitter data covering 2,000 image tweets demonstrate that our joint visual-text approach improves prediction accuracy by 13% (absolute gain) over the state-of-the-art text only methods. The effort will also leads to a large publicly available resource consisting of a concept ontology, a detector library, and the training/testing benchmark for visual sen- timent analysis.



@inproceedings{ BORT2013,
	Title = {Large-scale Visual Sentiment Ontology and Detectors Using Adjective Noun Pairs},
	Author = {Damian Borth and Rongrong Ji and Tao Chen and Thomas Breuel and Shih-Fu Chang},
	BookTitle = {ACM MM 2013},
	Month = {10},
	Year = {2013},
	Publisher = {ACM},
	Organization = {ACM}

Last modified:: 30.08.2016