Brands Shouldn’t Believe Everything They Read About Themselves Online

“Don’t believe everything you hear” is good advice — especially in an era of fake news and alternative facts. The same goes for managers who often rely on social-sentiment analysis to get a handle on what consumers think of their brands.

Social-sentiment analysis is the process of algorithmically analyzing social posts, comments, and behaviors and categorizing them into positive, negative, or neutral. Many companies use it to understand how their customers are feeling about their brands.

We recently conducted an extensive social-sentiment analysis with a team of researchers at Boston University’s Emerging Media Studies program as part of our Experience Brand Index research this past spring. In that research, we asked 4,000 consumers in the United States and United Kingdom about their actions and interactions with a wide range of brands over the last six months. These experiences were rated across more than a dozen dimensions, and we rolled up the results into a single Brand Experience score from 1 to 100.

The index graded nearly 100 different brands on how well consumers believed they were fulfilling the promises they make, how well they stood out from their competitors, and how likely consumers were to recommend them to friends and to stay loyal. Overall, our top 10-rated brands have a 200% better net promoter score (NPS) than the bottom 10, and have consumers who are 25% more likely to say they’re going to stay loyal.

To round out the research, we enlisted a group of graduate students in Boston University’s Emerging Media Studies program to run social sentiment analysis against the brands, fully expecting to see high-scoring brands receive high levels of positive sentiment and low-scoring brands receive high negatives.

We were wrong.

There appears to be very little predictive power between how people appear to feel online and how consumers who have experiences with those brands rate them.

We think social-sentiment analysis has value as a part of a brand’s consumer intelligence plan, but we have some advice for those using it or about to embark on the journey:

1. React, but don’t over-react. The type of consumers moved to post and share statements about brands (or about anything, for that matter) are not necessarily representative of the entirety of your customer base. Social-media users tend to be younger and more female than overall online audiences, and emerging research into social behavior suggests that people who post on social media tend to hold more extreme positions — they tend to be motivated by strong feelings, either positive or negative.

A recent study by Engagement Labs in the Journal of Advertising Research pointed out that online conversations about brands and offline conversations (as measured by their TalkTrack tracking study) were not strongly related.

In a recent interview, the lead investigator pointed out that online reaction to the Dick’s Sporting Goods decision to stop selling assault rifles and require all gun buyers to be 21 was met with a large degree of negative sentiment online but more positive sentiment offline.

More recently, Forbes did an in-depth analysis of the social reaction to Nike’s decision to feature Colin Kaepernick in an advertising campaign. It found a significant spike in negative sentiment online in the hours after the ad was first released. But, within two days, the sentiment shifted to positive.

So, while it’s important for your brand to react to specific negative customer-service posts immediately and address any specific issues consumers are having, we don’t recommend you react immediately to spikes in sentiment you see on a given day — especially if it’s in reaction to something new, like an ad campaign. If you do, you run the risk of over- or under-correcting for issues that just aren’t there.

2. Drill into specifics. What exactly does the sentiment analysis say and how does the tool you use define sentiment? In our experience, different tools — whether it’s NetBase or Brandmonitor or Hootsuite — will give you vastly different results for the same brand over the same period of time. Every platform defines sentiment differently and scores words and phrases in unique ways. And, despite significant advances in AI and sentiment algorithms, all of the platforms continue to have problems recognizing and correctly categorizing sarcasm, irony, jokes and exaggerations.

For example, a sarcastic post that says, “Great product, right?” and contains a picture of a broken cell phone is likely to be mischaracterized as positive.

As a result, it’s important to use your tool to listen for the right things. Again, the Nike example is instructive here. Rather than just look at the overall sentiment, the company examined tweets that had any purchase-intent statements — either positive (“going to buy”) or negative (“will never buy”) and found that positive outnumbered negative by 5 to 1. And the sales numbers appear to bear that out — with Thomson Reuters reporting a 61% increase in the amount of sold-out merchandise at Nike stores in the 10 days after the campaign launched compared to the 10 days before the ad appeared.

So, specifics matter. Look for spikes in volume and sentiment around specific hashtags to understand what might be going on.

3. Compare to what (and who) you know. The point of sentiment analysis is to give you a quick, directional perspective on what online chatter about your brand is all about. We believe it’s crucial to utilize other ways of tracking how consumers feel about your brand — whether it’s a brand tracker, tracking surveys, or analysis of customer service logs. It’s always best to have a mix of methods that deliver a well-rounded understanding of the voice of your customer.

It’s also best to have a sense of the cultural context during the time you’re measuring sentiment. Online sentiment can be driven by the negative actions of a specific brand — like a large retail bank illegally creating savings and checking accounts without customers’ consent — or it can be influenced by broader conversations in the culture that have little to do with a specific brand. For example, around the time we fielded our survey in the United States and United Kingdom, consumer tech leaders were testifying about privacy practices in the two countries, impacting the online conversation about that entire category of brands.

So, while there’s a ton of discussion about fake news and the role of bots and trolls in political news, we found an equally cautionary tale for brands. When it comes to social sentiment, listener beware.