Flyers Tweets - The Tweets

Part 2 of My Analysis of the Philadelphia Flyers Twitter Activity

The previous post examined the Twitter users that tweet about the Philly Flyers. In this post, we will look at the tweets themselves.

Through my position at Stockton University, I have heard that the Philadelphia Flyers are looking for ways to increase ticket sales for games. I determined several data research questions related to how word can be spread for the Philadelphia Flyers.

Specifically, I asked five questions that I want to answer (or attempt to answer) with my study. The first three questions were addressed in the previous post. Then I determined what Twitter data I wanted to analyze in an attempt to address my questions. Those data are tweets with the hashtags #Flyers, #FlyersNation, and #LETSGOFLYERS. Of course I could have added more hashtags, such as #PhillyFlyers, but the chosen hashtags resulted in sufficient data for an initial analysis. Thirdly, I determined what I wanted to measure in that data, and I divided my data questions into two groups. The first group is concerned mostly with the Twitter users and was the main subject of the previous post. This post will focus on the text of the tweets. Lastly I will present the results from the analysis in an attempt to learn something about using Twitter to attract fans who may then attend the games.

For this post, I asked the following questions: 4. How do events (wins vs losses, opponents) impact the amount of tweets? 5. What are the common characteristics of highly retweeted tweets?

I believe answering these questions will provide insight into when and what to tweet about the Flyers. If I were looking for ways to promote the Flyers, I might try to use what I learn to improve the use of social media.

As this is a R Notebook, all the code is in R, version 3.4.4 (2018-03-15) to be exact.

In March 2018, I scraped Twitter several times in order to gather all tweets that had the hashtags #Flyers, #FlyersNation, or #LETSGOFLYERS. The dates of the collections were March 11, March 20, and March 26, 2018. See the previous post for the code I used to scrape Twitter.

First, several R libraries are needed. Note, I tried to use only high quality libraries, such as those developed by the RStudio group.

library(rtweet) # for users_data()
library(tidyverse) # Instead of just ggplot2 and dplyr
library(tidytext)  # For Twitter text manipulation
library(lubridate)  # for date manipulation
library(reshape2)  # for mutate()
library(scales) # for date_breaks() in the ggplot - scale_x_date()
library(stringr) # for string manipulations
library("RColorBrewer") # Because I want to print with Flyers colors!

Read the three datasets into memory and combine into one master dataset. Then clean the datasets. For more information on the data preparation, see the previous post.

# Load the RData files that were saved after scraping Twitter
load(file="rtweets20180311.RData")
tw11 <- rstats_tweets
users11 <- users_data(rstats_tweets)
load(file="rtweets20180320.RData")
tw20 <- rstats_tweets
users20 <- users_data(rstats_tweets)
load(file="rtweets20180326.RData")
# Combine the two datasets
tw <- bind_rows(tw11, tw20, rstats_tweets)
users <- users_data(rstats_tweets)
users <- bind_rows(users11, users20, users)
rm(tw11, users11, tw20, users20, rstats_tweets)
# Remove duplicates, due to overlapping dates in the individual datasets.
tw <- unique(tw)
users <- unique(users)
### Clean up the data
# Remove users that do not (or should not) contribute value to this study.
users <- users[!(users$user_id %in% c("19618527", "471268712", "154699499", "426029765", "19276719", "493658381", "938072969552826368", "321035743")),]
# Only analyze "local" tweeters - location identified as PA, NJ, or DE
select <- grepl("Phil", users$location, ignore.case = TRUE) | grepl("PA", users$location, ignore.case = FALSE) | grepl("NJ", users$location, ignore.case = FALSE) | grepl("DE", users$location, ignore.case = FALSE)
users <- users[select,]
rm(select)
# Verified accounts include professional radio, TV, and news stations (e.g. NBC), and some names (a spot check identifies the selected as broadcastsers and reporters)
users <- users[!users$verified,] # Save only nonverified accounts
# Now select only the tweets that belong to these user_ids
tw <- tw[tw$user_id %in% users$user_id,]
# Save only the tweets that are in English (at least for now)
tw <- tw[tw$lang=="en",]

Prepare a working dataset that groups the tweets by the hour. In other words, how many tweets are there each hour over the time period.

twperhr <- tw %>%
  group_by(Group.1=format(tw$created_at, "%Y-%m-%d %H")) %>%
  summarise(x=n()) 

The data extends over the time period from 2018-03-04 20:56:42 to 2018-03-26 13:00:21. During that time, the Flyers played ten games, but we may want to consider the game previous to the time period as well as what was expected at the end of the time period.

• Flyers lost to the Panthers (1-4) on March 4 3:00pm
• Flyers lost to the Penguins (2-5) on March 7 8:00pm
• Flyers lost to the Bruins (2-3) on March 8 7:00pm
• Flyers beat the Jets (2-1) on March 10 1:00pm
• Flyers lost to the Golden Knights (2-3) on March 12 7:00pm
• Flyers lost to the Blue Jackets (3-5) on March 15 7:00pm
• Flyers beat the Hurricanes (4-2) on March 17 7:00pm
• Flyers beat the Capitals (6-3) on March 18 5:00pm
• Flyers lost to the Red Wings (4-5) on March 20 7:30pm
• Flyers beat the Rangers (4-3) on March 22 7:00pm
• Flyers lost to the Penguins (4-5) on March 25 12:30pm
• Nothing scheduled for March 26
• Next scheduled game against Dallas on March 27 8:30pm

Load this information into data vectors.

schedule = c("2018-03-07 20", "2018-03-08 19", "2018-03-10 13", "2018-03-12 19", "2018-03-15 19", "2018-03-17 19", "2018-03-18 17", "2018-03-20 19", "2018-03-22 19", "2018-03-25 12")
result <- c("Loss", "Loss", "Win", "Loss", "Loss", "Win", "Win", "Loss", "Win", "Loss")
opponent <- c(" \nPenguins", "Bruins", "Jets", "Golden\nKnights", "Blue\nJackets", "Hurricanes", " \nCapitals", "Red\nWings", "Rangers", "Penguins")
d1 <- format(as.Date(min(tw$created_at),format="%Y-%m-%d"), "%m-%d")
d2 <- format(as.Date(max(tw$created_at),format="%Y-%m-%d"), "%m-%d")

Printing the data as a table

date1 <- data.frame(Date = substr(schedule, 1, 10), Opponent = trimws(sub("\n", " ", opponent)), Result = result)
date1

Using the information gathered, we can create a graph of the time period.

4. How do events (wins vs losses, opponents) impact the amount of tweets?

# Determine the vector locations for game times
matches <- grep(paste(schedule,collapse="|"), twperhr$Group.1)
# Determine the average time from a game official start time that the tweets peak
avPeak <- paste("Average time from game start when tweets peak,", 
  mean(sapply(matches, function(i) 
    which(twperhr$x[(i-12):(i+12)]==max(twperhr$x[(i-12):(i+12)]))-13 )), 
  "hours", sep=" ")
  
# Create a graph of tweets over time and indicate when games occured
ggplot(data=twperhr, aes(x=seq_len(nrow(twperhr)), y=x)) + geom_line() +
    scale_x_continuous(breaks = grep(" 00", twperhr$Group.1), 
      labels = substr(twperhr$Group.1[grepl(" 00", twperhr$Group.1)], 6, 10) ) +
    theme(axis.text.x = element_text(color="darkorange", angle=45), 
          panel.background = element_rect(fill = "white", colour = "orange"),
          panel.grid.minor = element_blank()) +
    annotate("text", x = matches, y = twperhr[matches+6,]$x, 
             label = paste(opponent, "\n(", substr(result,1,1),")",sep=""), 
             colour = gsub("Loss", "red", gsub("Win", "darkblue", result))) +
    labs(title="Tweets Mentioning the Flyers", subtitle=paste(d1,"to",d2,sep=" "), 
         caption= avPeak, x = "Date", y = "Tweets")

# Clean up variables that are no longer needed
rm(matches, avPeak) 

Over the specified time, it does not appear that a win or a loss regularly impacts the number of tweets. The fact that a game is played has a huge impact, but not the outcome. And since the number of tweets peak after the conclusion of the game, the peaks are not the result of anticipation.

While the previous graph was by the hour, perhaps there is something to be gained by looking at the number of tweets per day.

ggplot(data = tw, aes(x = day(created_at))) +
  geom_bar(aes(fill = ..count..)) +
  theme(legend.position = "none") +
#  xlab("March") + ylab("Number of tweets") + 
  labs(title="Tweets Mentioning the Flyers", subtitle=paste(d1,"to",d2,sep=" "), 
         x = "March", y = "Number of Tweets") +
  scale_fill_gradient(low = "orange", high = "orangered2")

There may be something else going on to impact the number of tweets. Perhaps looking at the data by day of the week as well as when games are played would have some pattern?

5. What are the common characteristics of retweeted tweets?

Let us look at the tweets themselves. To do so, we need to clean them up. By that I mean remove references to screen names, hashtags, spaces, numbers, punctuations, and urls.

clean_tweet <- gsub('\\n', '', tw$text) %>% 
  str_replace_all("http\\S+\\s*","") %>%
  str_replace("RT @[a-z,A-Z,0-9]*: ","") %>%
  str_replace_all("#[a-z,A-Z]*","") %>%
  str_replace_all("@[a-z,A-Z]*","") %>%
  str_replace_all("[0-9]","") %>%
  str_replace_all(" "," ")

First we will look at the words used in tweets, and then we will consider the tweets as a whole.

tweets <- data_frame(text=clean_tweet) %>% unnest_tokens(word, text)
data(stop_words)
tweets <- tweets %>% anti_join(stop_words)
tweets %>% count(word, sort = TRUE) 

Graph the words that occur at least 200 times.

tweets %>%
  count(word, sort = TRUE) %>%
  filter(n > 200) %>%
  mutate(word = reorder(word, n)) %>%
  ggplot(aes(word, n)) +
  geom_col(aes(fill = n)) +
  scale_fill_distiller(palette="Oranges") +
  theme(legend.position = "none") +
  xlab("Popular Words") + ylab("Number Occurences") +  
  labs(title="Most Popular Words of Tweets") +
  coord_flip()

Using a sentiment score, analyze each tweet as a whole. To accomplish this, i am determining the sentiment of the words in the tweet, with positive sentiments getting a positive score and negative sentiments getting a negative score. Then I am summing the score for each tweet. Tweets of the form “that is not good” will get a sentiment score of 0, since “not” is negative and “good” is positive, but at least that should not sway the analysis. If “not good” got a score of positive due to the “good”, that would be bad. The sentiment scores will be determined by hour so we can see how the games impact the sentiments of the tweets.

#Determine the sentiments of the tweets
sentiment <- tibble(index = 1:nrow(tw),
                    created = tw$created_at,
                    text = clean_tweet) %>%
  unnest_tokens(word, text) %>%
  inner_join(get_sentiments("nrc")) %>%
  filter(sentiment %in% c("positive", "negative")) %>%
  group_by(index, created) %>%
  count(index, sentiment) %>%
  spread(sentiment, n, fill = 0) %>%
  group_by(Time = round_date(created, unit="hours")) %>%
  mutate(score = positive - negative) %>%
  summarise(score = sum(score))
sentiment$sentiment <- factor(ifelse(sentiment$score > 0, "Negative", "Positive"), labels=c("Positive", "Negative"))

Now we can graph the tweets.

date2 <- format(sentiment$Time, "%Y-%m-%d %H")
for (i in seq_len(10)) { date2 <- sub(schedule[i], opponent[i], date2, ignore.case = TRUE) }
date2[grep("2018-03-", date2, ignore.case = TRUE)] <- ""
ggplot(sentiment, aes(x=as.Date(Time), y=score)) +
  geom_line(size = 1.5, alpha = 0.7, aes(colour = sentiment)) +
  labs(title="Sentiments of Tweets", subtitle=paste(d1,"to",d2,sep=" "), 
       x = "March", y = "Sentiment Score") +
  geom_text(y=rep(c(50, 75, 100), len = nrow(sentiment)), label=date2) +
  theme(legend.position="none")

rm(date2,i)  

The graph may be a little misleading since it appears the Bruins game causes a large spike in tweets. When I manually inspect the data, it appears that the large spike occurs after the conclusion of the March 7 Penguins game and just prior to the start of the March 8 Bruins game.

Moving on, let us now look at the characteristics of retweets. Since retweets, by definition, have multiple occurences, we want just one tweet to represent each retweet set.

clean_tweet <- tw[tw$status_id %in% unique(tw$retweet_status_id),]$text

Clean the text again.

# For removing retweets, references to screen names, hashtags, spaces, numbers, punctuations, urls.
clean_tweet <- gsub('\\n', '', clean_tweet) %>% 
  str_replace_all("http\\S+\\s*","") %>%
  str_replace("RT @[a-z,A-Z,0-9]*: ","") %>%
  str_replace_all("#[a-z,A-Z]*","") %>%
  str_replace_all("@[a-z,A-Z]*","") %>%
  str_replace_all("[0-9]","") %>%
  str_replace_all(" "," ")

Determine the sentiment of each tweet and graph. The top line graphs the positive sentiment, with higher numbers indicating a higher positive sentiment. The lower line graphs the negative sentiment tweets, with lower numbers indicating an increase in negative sentiments.

sentiment <- tibble(index = 1:length(clean_tweet),
  created = tw[tw$status_id %in% unique(tw$retweet_status_id),]$created_at,
              text = clean_tweet) %>%
  unnest_tokens(word, text) %>%
  inner_join(get_sentiments("nrc")) %>%
  filter(sentiment %in% c("positive", "negative")) %>%
  group_by(index, created) %>%
  count(index, sentiment) %>%
  spread(sentiment, n, fill = 0) %>%
  group_by(Time = round_date(created, unit="hours")) %>%
  mutate(score = positive - negative) %>%
  summarise(score = sum(score))
  
sentiment$sentiment <- factor(ifelse(sentiment$score > 0, "Negative", "Positive"), labels=c("Positive", "Negative"))
ggplot(sentiment, aes(x=as.Date(Time), y=score)) +
  geom_line(size = 1.5, alpha = 0.7, aes(colour = sentiment)) +
  labs(title="Sentiments of Tweets", subtitle=paste(d1,"to",d2,sep=" "), 
       x = "March", y = "Sentiment Score") +
  theme(legend.position="none")

date1

Looking at the graph, it appears the loss to the Golden Knights on March 12 was followed by a large spike in positive sentiment with a reduced trend of negative sentiment. Did the loss of the one game cause a high level of encouragement for the upcoming Blue Jackets game? However, after losing to the Blue Jackets, there is a jump in negative sentiment.

I admit that these results do little to address the 5th question, and a more thorough analysis is needed. This further analysis would require more of the same, just in more detail. Maybe looking at what other events occured during the time? Definitley expanding the list of words since the top results are somewhat expected (e.g. game, flyers, win, goal). Another interesting result might be if the sentiment causes more or less retweets? That would be useful information if we wanted to spread the word better.

There is much more I could analyze from this data. For one thing, I should expand on my conclusions from analyzing the data. Since this study was “for fun,” I may or may not return to write a more thorough conclusion, but the reader is free to look at the analysis and form their own conclusions. I guess we will see if I continue with this study or get distracted by another “for fun” project in the next post.