Sentiment Analysis Purpose and Problems

Opinions abound in tweets, wall posts, blog comments and documents on other web and social media platforms. We looked at the text elements of these documents that comprise opinions in the first part in this series.

Informed by the discussions and presentations at the recent Sentiment Analysis Symposium, let’s examine the business case for sentiment analysis, as well as some issues related to the discovery and analysis processes applied to those documents to mine actionable information for businesses.

Steve Rappaport, ARF Knowledge Solutions Director and author of the just-published Listen First, kicked off the symposium’s Visionary Panel asking why businesses should be using sentiment analysis.

Karla Wachter of communications firm Waggener Edstrom warned that, “Sentiment analysis done in isolation is less likely to solve business problems.” Israel Mirsky of PR firm Porter Novelli expressed that opinion analysis may better describe what businesses seek. They are looking for more than plus, minus and neutral — businesses want to know why and with a high degree of granularity.

Sentiment analysis must have relevance to your business, affirmed Katie Delahaye Paine of research and consulting company KDPaine & Partners, “Show me the data that evidences a certain level of positive sentiment yields sales or translates to purchase intent,” she challenged. “The best uses have to do with business, not with marketing.”

The role of automation in sentiment analysis

Much of the discussion centered on machine versus human sentiment analysis. Jeff Catlin of text analytics engine developer Lexalytics cited the requirement for machines — the enormous amount of data and the speed at which businesses demand information simply cannot be processed by humans. Catlin acknowledged many limitations, such as handling cynicism and sarcasm. One of the areas he sees machine sentiment analysis as improving is extracting edge cases of value for data triage.

While there has been much innovation, Wachter noted, humans are always going to be required. Mirsky observed that the kinds of data being sought, such as indicators of intent, and decisions attempting to made with sentiment analysis are getting more complex. He felt these would remain difficult to accomplish without use of humans until we have stronger artificial intelligence.

“The engines do well on very concrete information,” Catlin commented, “They are not so good on the squirrelly stuff that humans are apt to say.” He warned that the more measures sought, the more machines are asked to act human, increasing the risk of errors.

When the panel was asked to assess how sentiment analysis may be improved, Mirsky called for better feedback loops — users need to have a better understanding of what’s working. Paine supported Wachter’s statement that context of sentiment is essential for analysis by noting that her firm has clients for whom neutral sentiment is a desirable outcome.

In her later presentation, Fiona McNeill of SAS noted that sentiment analysis does not need to do what current business tools and systems do, but does need to integrate with them. “Sentiment is not always impactful to an organization — nor its cousins of emotion, trust, feelings, and alike,” McNeill wrote on her blog, The Text Frontier, after the symposium. “How someone feels does not always affect what they do – but when it has an important impact, it is worthwhile to analyze it.” In this post, she also presents a sentiment analysis framework built on data validation, learning and changing behavior.

Multiple perspectives

Some of the presentations from the Sentiment Analysis Symposium can be found on the event website — look for presentation links next to the listings. In addition, there were a number of excellent blog posts subsequent to the symposium.

Tom H. C. Anderson writes on his Next Gen Market Research Blog, “I’m not at all convinced there is one best way to approach text analytics and strongly believe it’s dependent not just on the domain, but also on the final objectives and needs of the end user. At the risk of using an overly simple example, a carpenter needs different tools depending on the job. A saw, no matter how sharp, will not do the job of a hammer.”

Another wrap-up comes from Mindshare Technologies’ Kurt Williams. He notes, “One popular theme at the Sentiment Symposium was using a sentiment index as a proxy for another metric. For example, after establishing a baseline ‘average daily sentiment’ from news stories or Twitter feeds, it has been found that positive and negative changes in sentiment over time can be a leading indicator of a company’s stock price. However, before you cash in your 401K’s and start trading on sentiment futures, remember that correlation isn’t causation and the technology is still new.”

“Another common theme at the conference was to treat sentiment analysis like a metal detector at the beach,” writes Williams. “It can tell you where something interesting might be buried in the sand, but it can’t tell you how valuable that something is. Similarly, sentiment analysis can indicate something of interest occurring with a correlated metric like a stock price or customer satisfaction index, but it probably can’t tell you what that something is. It’s an early warning system, not a detailed threat assessment.”

On semanticweb.com, Jennifer Saino does a great job of covering several conference topics. She starts Working Out the Kinks In Sentiment Analysis — And Focusing on the Opportunities with, “What’s the most important requirement for sentiment analytics to succeed? Make that question plural, and let’s start our answers with something that the tools in this area themselves have no influence on: Good quality data.”

We’ll pick up the discussion of data in the third part of this series.