Homework 3: Tidy Text & Sentiment

Introduction

In this assignment you will apply the tokenization skills from code along 03 and TTM chapter 1, and the sentiment analysis skills from code along 04 and TTM Chapter 2 to selected questions from Amy’s research data.

Setup

The data set is called criminal_justice_short.csv and is available in our Jupyterhub Wicked/data folder. Copy this file to your own home directory.

Load the tidyverse, tidytext and sentimentr packages in this code chunk, along with your data.

library(tidyverse)
library(tidytext)
library(sentimentr)

cj_data <- read_csv("data/criminal_justice_short.csv")

The cj_data data has been pre-filtered to drop all non-consenting students, and contains the following 4 variables: Q7, a mark all that apply program of study variable, with follow up questions why_cj, why_pols_legal, why_paralegal_cert that ask “Why did you decide to major or minor in [PLAN]? Please provide as much detail and description as possible.”

Since Q7 is a ‘mark all that apply’, variable, and only those that marked that specific plan were asked follow up “why_” questions, it would be helpful if we had a single indicator variable for each selected major. This way if we wanted to compare responses between programs we can use those variables as filters.

We will use the str_detect() (read: string detect) function for this. It searches for a pattern of characters in a variable, and returns a TRUE if it is found, otherwise it will show FALSE.

Remove the #| eval: false from the top of the provided code chunk to allow them to run. Then remove these instructions.

cj <- cj_data |>
  mutate(
    cj_major = str_detect(Q7, "Criminal Justice major"),
    cj_minor = str_detect(Q7, "Criminal Justice minor"),
    pols_legal = str_detect(Q7, "Political Science major"),
    para_cert = str_detect(Q7, "Paralegal Certificate"),
    un_consider = str_detect(Q7, "STRONGLY CONSIDERING"),
    un_not_consider =str_detect(Q7, "NOT CONSIDERING")
  )

1. Interpret this output in 1-2 sentences.

So for example, the new variable cj_major is TRUE for anyone that chose that major, by itself or in combination with other majors.

cj |> filter(cj_major==TRUE) |> count(Q7)

Part A — Tidy it

2. Tokenize and remove stop words

Let’s explore why students selected the Criminal Justice major. I have started this code for you by writing some data cleaning steps - specifically filtering to only keep rows where the student answered the question, then only selecting the why_cj variable to keep your results simple. Finish this process by tokenizing why_cj and removing stop words. Save the result as cj_words.

cj_words <- cj |>
  filter(!is.na(why_cj)) |>
  select(why_cj) |>
  unnest_tokens(________, ________) |>
  anti_join(________)

head(________)

3. Identify and describe the most common words.

Use count() to count how often each word appears in cj_words, and save the result as cj_word_counts. Then create a barplot of the 10 most frequent words by following the example in the slides from codealong 03. In 2-3 sentences, describe the plot. Which words seem meaningful for understanding students’ reasons, and which words might simply reflect the wording of the survey question?

4. Remove some custom stop words

Criminal, justice, decided, and major are real words, but they’re baked into the question and don’t tell us anything about why a student chose the major. Create a custom stop-word list called custom_stop. Then re-run the anti_join() on cj_words, saving the results into a new data set called cj_words_clean. Next count() the words, and create a barplot that shows the words that show up more than 15 times. Comment on the results.

5. Investigate reasons for choosing another program.

Choose either why_pols_legal or why_paralegal_cert and repeat the above process. That is: tokenize, remove stop words, visualize, remove custom stop words, visualize, interpret.

Part B — Sentiment Analysis

6. Positive and negative words

Using the bing sentiment lexicon, create a barplot grouped by sentiment that shows the 5 most common positive and negative words in cj_word_counts_clean.

7. Look closer at one emotion

Filter the nrc lexicon to only pull out a single emotion category. Then use inner_join() to pull the words in cj_words_clean that fall under that emotion category. Show the count of those words and comment on the results.

8. Sentence-level sentiment

Choose one of the three open-ended response variables:

  • why_cj
  • why_pols_legal
  • why_paralegal_cert

Use sentimentr to score the responses at the sentence level. Then inspect and comment on the top 5 most positive and 5 most negative sentences. IMPORTANT Due to the way long sentences word wrap, you must pipe in knitr::kable(row.names = FALSE) after you pull(sentence) to format the results into a readable table.

9. Reflection

Based on your word-level and sentence-level sentiment results, explain one limitation of using a general-purpose sentiment tool on this kind of qualitative data.