---
title: "Relationships between words"
subtitle: "Week 6 — Codealong Notes"
author: "YOUR NAME HERE"
date: "2026-09-28"
format: pdf
execute:
  echo: true
  warning: false
  message: false
  error: true
---

```{r}
library(tidyverse)
library(tidytext)
library(janeaustenr)
library(igraph)
library(ggraph)
library(widyr)  # install.packages("widyr") before use
```

# 4.1 — Tokenizing by n-grams

## From words to bigrams

```{r}
austen_bigrams <- austen_books() |>
  unnest_tokens(bigram, text, token = "_______", n = ___) |>
  filter(!is.na(bigram))

head(austen_bigrams)
```

## Most common bigrams

```{r}
_______ |>
  count(bigram, sort = TRUE)
```

## Splitting and filtering

```{r}
bigrams_separated <- austen_bigrams |>
  separate(bigram, c("_______", "_______"), sep = " ", remove = FALSE)

bigrams_filtered <- _______ |>
  filter(!word1 %in% stop_words$word) |>
  filter(!word2 %in% stop_words$word)

bigram_counts <- bigrams_filtered |>
  ______(word1, word2, sort = TRUE)

head(bigram_counts)
```


## Streets of Austen

```{r}
bigrams_filtered |>
  filter(word2 == "_______") |>
  count(book, word1, sort = TRUE)
```

## 🎯 Your turn

Pick your own word for `word2` and see what turns up. Try something like `"park"`, or any word you're curious about. You could also choose to filter on `word1` and count the frequency of `word2` — your choice!

```{r}
bigrams_filtered |>
  filter(word2 == "_______") |>
  count(book, word1, sort = TRUE)
```

# 4.2 — Bigrams give context

## Words following "not"

```{r}
bigrams_separated |>
  filter(word1 == "_______") |>
  count(word1, word2, sort = TRUE)
```

## Joining sentiment to bigrams

```{r}
AFINN <- get_sentiments("afinn")

not_words <- bigrams_separated |>
  filter(word1 == "not") |>
  ______(AFINN, by = c(word2 = "word")) |>
  count(word1, word2, value, sort = TRUE)

head(not_words)
```

## Plot: what "not" flips

Weight each word by how often it appears *and* how strong the score:

```{r}
not_words |>
  mutate(contribution = n * value) |>
  arrange(desc(abs(contribution))) |>
  head(20) |>
  mutate(word2 = reorder(word2, contribution)) |>
  ggplot(aes(n * value, word2, fill = n * value > 0)) +
  geom_col(show.legend = FALSE) +
  labs(x = "Sentiment value × number of occurrences",
       y = "Words preceded by \"not\"")
```


## 🎯 Your turn

"Not" isn't the only negator. Repeat the same join-and-count approach for a vector of negation words instead of a single "not".

```{r}
negation_words <- c("not", "no", "never", "without")

negated_words <- bigrams_separated |>
  filter(word1 %in% _______) |>
  ______(AFINN, by = c(word2 = "word")) |>
  count(word1, word2, value, sort = TRUE)

head(negated_words)
```

## Visualize to learn more

```{r}
negated_words |>
  mutate(contribution = n * value) |>
  group_by(word1) |>
  slice_max(abs(contribution), n = 10) |>
  ungroup() |>
  mutate(word2 = reorder_within(word2, contribution, word1)) |>
  ggplot(aes(contribution, word2, fill = contribution > 0)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~word1, scales = "free") +
  scale_y_reordered() +
  labs(x = "Sentiment value × number of occurrences", y = NULL)
```


# 4.3 — Visualizing word networks

## Building the graph object

The `igraph` package turns an edge list into a graph:

```{r}
bigram_graph <- bigram_counts |>
  filter(n > ___) |>
  ______()

bigram_graph
```

## Building the graph

```{r}
ggraph(bigram_graph, layout = "fr") +
  geom_edge_link() +  # draws _______
  geom_node_point() +  # draws _______
  geom_node_text(aes(label = name), vjust = 1, hjust = 1) # adds _______
```

## Make it fancy

```{r}
a <- grid::arrow(type = "closed", length = unit(.15, "inches"))

ggraph(bigram_graph, layout = "fr") +
  geom_edge_link(aes(edge_alpha = n), show.legend = FALSE,
                 arrow = a, end_cap = circle(.07, "inches")) +
  geom_node_point(color = "lightblue", size = 5) +
  geom_node_text(aes(label = name), vjust = 1, hjust = 1) +
  theme_void()
```


# 4.4 — Pairwise counts & correlations

## Splitting into sections

```{r}
austen_section_words <- austen_books() |>
  filter(book == "Pride & Prejudice") |>
  mutate(section = row_number() %/% 10) |>
  filter(section > 0) |>
  unnest_tokens(word, text) |>
  filter(!word %in% stop_words$word)

head(_______)
```

## Count co-occurring words in a 'section'

```{r}
word_pairs <- austen_section_words |>
  ______(word, section, sort = TRUE)

head(word_pairs)
```


## The problem with raw counts

```{r}
word_pairs |>
  filter(item1 == "darcy") |> slice(_______)
```

## Warning: Math! 😱

The phi ($\phi$) coefficient asks: how much more likely are X and Y to appear *together* (or both be absent), compared to appearing separately? It's equivalent to a Pearson correlation on binary data.

* $n_{11}$ = sections where *______* words appear
* $n_{00}$ = sections where *______* appears
* $n_{10}$/$n_{01}$ = sections where ______ does.

## Calculating the $\phi$ coefficient using R

```{r}
word_cors <- austen_section_words |>
  group_by(word) |> filter(n() >= ___) |>
  ______(word, section, sort = TRUE) # this calculates phi

head(word_cors)
```

## 🎯 Your turn

Filter `word_cors` for a word of your choice and look at what it's most correlated with. Does the result make sense?

```{r}
word_cors |>
  filter(item1 == "_______")
```

## Visualizing correlations

```{r}
word_cors |>
  filter(______ > ______) |>
  graph_from_data_frame() |>
  ggraph(layout = "fr") +
  geom_edge_link(aes(edge_alpha = correlation), show.legend = FALSE) +
  geom_node_point(color = "lightblue", size = 5) +
  geom_node_text(aes(label = name), repel = TRUE) +
  theme_void()
```