The Electronic Retailing Association

Article: Social Media Contents Evaluation: what is Consumer Sentiment Analysis? by Dr. Emanuele Frontoni & Marina Paolanti

The advent of Social Media has enabled everyone with a smartphone, tablet or computer to easily create and share their ideas, opinions and contents with millions of other people around the world. Social Media data has challenging characteristics. It is fast and high-volume, but behind every tweet, post, picture and comment there is a customer.

Keeping up with the massive volume of messages, photos, and videos that consumers upload and share on social networks every day is a task that only automated intelligent machines can take on. Artificial Intelligence is helping researchers make sense of all this unstructured data.

Dr. Emanuele Frontoni, Associate Professor, Vice Rector for Digitalization at the Università Politecnica delle Marche, Ancona, Italy.

Emanuele Frontoni received his doctoral degree in Electronic Engineering from the University of Ancona, Italy, in 2003. In the same year he joined the Università Politecnica delle Marche to research "Intelligent Artificial Systems", obtaining his PhD in 2006 with a thesis on Vision Based Robotics. At present he has a Post Doc position in the same Department of Information Engineering. His research focuses on applying computer science, artificial intelligence and computer vision techniques to mobile robots and innovative IT applications. He is a member of IEEE and AI*IA, the Italian Association for Artificial Intelligence.

Dr. Frontoni has several international cooperations with SMEs and companies on innovation and future trends, has published more than 120 papers and has been a speaker at more than 40 international conferences, including the MCMS 2017 in Venice.

In particular, machine learning algorithms, i.e. the study of how computer systems can be programmed to exhibit problem-solving and decision-making capabilities that emulate human intelligence, are helping marketers and advertisers understand insights from this huge amount of unstructured consumer data collected by the world’s largest social networks.

Facebook users upload 350 million photos each day, Snapchat users share 400 million “snaps” (Snapchats term for photos and videos shared over the network) each day, Instagram users upload 55 million photos each day. These numbers give an idea of the enormous amount of data available for analysis.

Advances in deep learning, cutting-edge artificial intelligence research that attempts to programme machines to perform high-level thought and abstractions, are allowing retailers to extract information from the billions of photos, videos, and messages uploaded and shared on social networks each day. In this way, images can be analysed to reveal an abundance of information about the person who uploaded it, as well as the context and objects shown in the photograph.

Like the brain, deep learning systems process information incrementally, beginning with low-level categories, such as letters and words. They can use deductive reasoning to collect, classify, and react to new information, and tease out meaning and arrive at conclusions, without needing humans to get involved and provide labels and category names.

The most advanced systems can also use deductive reasoning to predict outcomes based on large data sets. This is the kind of technology that’s needed to make sense of unstructured data, to parse meaning from contextual cues in text, to classify and recognise objects in pictures, and to identify voices to make sense of speech.

Marina Paolanti, Research Fellow in the Department of Information Engineering (DII) at the Università Politecnica delle Marche, Ancona, Italy.

Marina Paolanti, Università Politecnica delle Marche, received the Master’s degree in Electronic Engineering from Università Politecnica delle Marche, Italy, in 2012. In 2014, she joined the Department of Information Engineering (DII) at the Università Politecnica delle Marche as a Ph.D. student. From January to April 2017, she did an Internship in GfK Verein, Nuremberg, Germany. Currently, she is a Research Fellow in the Department of Information Engineering (DII). Her research focuses on Pattern Recognition applied to several domains. She is the author of scientific research papers in book chapters, journals or conference proceedings in English, speaker of international conferences, and Member of GIRPR - Gruppo Italiano Ricercatori in Pattern Recognition.

Social networking experiences are becoming increasingly centered around photos and videos, but it is extremely difficult to extract information from visual content. Because of this, image and video recognition are two of the more exciting disciplines being worked on in the field of machine learning and deep learning.

Back-propagation is a method for training computer systems to match images with labels or tags that were pre-defined by users. For example, if enough people upload photos tagged “shoes” then a system has a large enough sample size that it can reference to identify new photos of shoes, and tag them appropriately. Moreover, image recognition algorithms are able to identify brand logos in photos and they also identify the environment in which brands appear and whether a brand is accompanied by a smiling face in the photo.

Another challenge with social media data is that they are very fragmentary and cryptic. Social media posts that tend to generate the most interaction on Facebook, Instagram and Twitter are around 10 or 20 characters in length (that’s only few words, but that’s because they are usually accompanied by some form of media such as images or videos).

By evaluating the content of images on social media data, retailers will become more effective and comprehensive in their social listening efforts. Retailers with deep learning and machine learning are able to measure brand sentiment by analysing images that contain their logo or merchandise. Brands will also be able to track how images associated with their brand are being shared across social networks, how often and by what sorts of audiences. Marketers and advertisers will also gain a better understanding of what characteristics of a photo or video influence its virality. 

Brand Related Social Media. From top left clockwise: (1) Picture with overall neutral sentiment. (2) Image with overall positive sentiment. (3) Example of a picture with overall negative sentiment. Picture taken by - Paolanti, M., Kaiser, C., Schallner, R., Frontoni, E., & Zingaretti, P. (2017, September). Visual and textual sentiment analysis of brand-related social media pictures using deep convolutional neural networks. In International Conference on Image Analysis and Processing (pp. 402-413). Springer, Cham.

Sentiment Analysis

Sentiment analysis is the task of evaluating this goldmine of information. Sentiment is pretty simple to understand. It is just a feeling or emotion, an attitude or opinion. On social media, the sentiment of a post can be seen in the tone or emotion conveyed in a brand mention. It retrieves opinions about certain products and classifies them as positive, negative, or neutral. Without sentiment, data can be easily misleading. For example, if a company was launching a new product and it was receiving a huge amount of mentions, it is possible to assume that the product has been well-received.

Existing research works have focused on the sentiment analysis of textual postings such as reviews in shopping platforms and comments in discussion boards. In the last years, the attention has been focused on proposing solutions for the sentiment analysis of visual content. This is due to the fact that it is essential to not only judge the sentiment of the visual elements but also to understand the meaning of the included text. While a picture showing a cosmetic product next to a cute rabbit might be positive, the same picture containing the words “animal testing" might be negative. This information can then provide real insights and learnings on the levels of engagement the content marketing efforts are generating and the general online brand reputation.

Using these powerful social sentiment measuring techniques, it is possible to keep up with the ever-changing conversation about brand and stay relevant well into the future.