Product Feature: Chat Sentiment Analysis

On Twitch, YouTube Live, and other live streaming platforms, the multitude of community chat messages takes the stage as much as the creators themselves. This rich, first-order community feedback provides organic data on audience sentiment. However, it's difficult to extract valuable and actionable signals hidden within the endless stream of emotes.

While many off-the-shelf sentiment analysis products for social media sites like Twitter and Instagram exist, Twitch and similar gaming chats cannot be analyzed with the same conventional methods. Twitch chat acts as a unified collective, characterized by individuals echoing and amplifying popular sentiments through repeated messaging. This phenomenon gives rise to a vast ocean of noise, further clouded by emotes, jokes, "copypastas," and sarcasm, requiring a nuanced understanding of Twitch culture and ever-evolving gaming trends.

Instead of scrolling through millions of messages by hand, Gamesight uses sentiment analysis backed by natural language processing, industry knowledge, and firsthand experience about how gamers chat to offer the best-in-class Twitch and YouTube sentiment analysis product. This allows you to get direct insights on community feedback provided on a stream, decoding those cryptic emojis and memes into actual sentiment.

How Can We Use Chat Sentiment Analysis?

Sentiment analysis offers invaluable insights to both game developers and marketing teams. Instead of quantifying player response based on ambiguous feedback and a few vocal users, you can receive a detailed data-driven overview of live reactions.

Here are some examples of how sentiment analysis can help your campaign and development cycles:

1. Understanding player preferences: Sentiment analysis can be used to understand player preferences for different aspects of a game, such as the gameplay, graphics, and story. Understanding what your fans like about your game can then be used to reaffirm or adjust community analysis, improve the game, and make it more appealing to players.
2. Targeting marketing campaigns: Once developers can take the guesswork out of understanding their player base, this information can then be used to more strategically target and segment the market, increasing return on investment.
3. Adjusting marketing strategies: If there’s a new game trailer, or any form of announcement - sentiment analysis can be used to tangibly identify which aspects of the game generated the most user interest and hype.
4. Identifying bugs and glitches: Identify negative sentiment about a game, which can be a sign of bugs, glitches, or player pain points. This information can then be used to quickly find common issues and improve the player experience.
   

Although our product is centered around analyzing gaming sentiment, sentiment analysis can be used for non-gaming applications. Because we tailored our solution for the Twitch and Youtube communities, nearly all of the above use cases can be translated for movies, shows, and other non-endemic products.

Big Challenges in Twitch Chat

To understand how Gamesight's sentiment model works, we first need to identify the major challenges that Twitch chat presents. Some of the most prominent problems with building a Twitch chat sentiment model include constantly evolving emote usage with new emotes emerging regularly, using emotes to modify the meaning of messages, dealing with sarcasm, and handling very short chats.

To tackle these challenges, we have developed a novel solution that combines several approaches to pull true sentiment from the noise of chat.

The Emote Meta

Let's focus on emotes in particular. Chatters employ emotes to accentuate a message's sentiment and to convey a sentiment on their own when posted independently. This aspect is not unique to Twitch, but custom emotes are. Streamers can create new emotes exclusive to their channel, introducing unknowns that sentiment models cannot recognize. This poses a recurring challenge as new emotes emerge regularly, and certain streamers have a chat that utilizes an entirely different set of emotes, making identification through conventional means nearly impossible. So, this raises the question - how can we attribute sentiment values to these new emotes with unknown sentiment values?

Word Vectorization

Gamesight uses word vectorization to classify new emotes with unknown sentiment values on Twitch. It projects emotes into physical space, treating each emote as a coordinate in a multi-dimensional graph. From there, we observe how unknown and known emotes are used in Twitch chat in conjunction with text. By plotting the known and unknown emotes in this physical space, we can observe that emotes closer in this space have a more similar sentiment.

Calculating which emotes are close within this physical space allows us to extrapolate emote sentiment based on previous chat data, enabling a sentiment model that evolves with Twitch.

B-b-bonus fact!

Because these emotes and words occupy a physical space, we can do intuitive vector addition.

For example: “FeelsGoodMan” - “:)” = “FeelsBadMan”

Data Cleaning

Gamesight's Chat Sentiment Analysis also performs data cleaning, ensuring the removal of noise such as Twitch Prime subscriptions, announcements, moderator messages, and streamer messages from the dataset.

The Importance of N-Grams

Another powerful tool in chat sentiment analysis is the examination of n-grams, which are one to n (where n is any number) phrases commonly repeated in chats. By analyzing these n-grams and their triggers, we gain valuable insights into moments that stand out - whether they involve hype, surprising twists, widely agreed-upon ideas, or even pain points.

Message Classification and Sarcasm

With Gamesight's Chat Sentiment Model, we classify messages as either Positive, Negative, Neutral, or Mixed, with the use of positive and negative text heavily influenced by the emotes employed.

Positive Example: “honestly how funny would it be if we fight banana gators here”

Negative Example: “NOPERS dont play this game”

NOPERS is an emote of the Pepe frog disapprovingly shaking its head.‌ ‌

Neutral Example: “That's the technical term OhISay”

OhISay is an emote of an anime character with a monocle and a mustache.

Mixed Example: “I think I'd like this game better if they removed the gameplay”

This Twitch chat example is a sarcastic joke, since it’s impossible to remove “gameplay.”

Mixed is an example of how we handle sarcasm, as we consider sarcasm to be an incongruence in sentiment (i.e., a message that the algorithm would consider positive, but in reality, it is actually negative - and vice versa).

Putting It All Together

Overall, Gamesight's Chat Sentiment Analysis provides an industry leaning tool for understanding and leveraging the sentiments expressed in Twitch chat for various applications. Quickly understand and react to millions of messages spanning thousands of streams to keep in-touch and understand your community.

If you would like to see Gamesight’s sentiment model in action, check out our 2022 Summer Games Fest Blog Post!

About us

At Gamesight, we use data to help games find commercial success. If you are looking for help getting your game's performance or influencer running and want to talk about this article, identifying the right influencers, or measuring your advertising needs, please reach out to us at gamesight.io.