Working with Factors
In this lesson we will discuss ways to organize and deal with categorical data, also known as factor data types.
Reference: Math 130 - Chapter 5
Learning Objectives
After completing this lesson students will be able to
- Convert a numeric variable to a factor variable.
- Apply and change labels to factor
- Understand and control the ordering of the factor.
- Combine multiple levels of a factor variable into one level
- Learn how to use the
forcatspackage
Also part of the tidyverse!
email data set contains information on emails received by one of the OpenIntro authors for the first three months in 2012. See ?email for more details.The fastfood data set from the openintro package describes nutrition amounts in 515 fast food items. See ?fastfood for more details.The goal of the forcats package is to provide a suite of useful tools that solve common problems with factors.
What is a factor?
Categorical data often are made up of words and letters.
table(ff$restaurant)
Arbys Burger King Chick Fil-A Dairy Queen Mcdonalds Sonic
55 70 27 42 57 53
Subway Taco Bell
96 115 table(email$number)
none small big
549 2827 545 However, they can have different data types.
Factor variables are character data types, but typically with a finite number of possible responses. Think about our class survey.
Code
survey <- googlesheets4::read_sheet("https://docs.google.com/spreadsheets/d/1nUvOJN8z91c3dPN5UrV653qJxlbMCvvj846JDgOscdM")
survey <- survey[,-c(1,2)]
names(survey) <- c("location", "programming_experience", "qual_research_experience", "qual_data_analysis", "data_interests", "learn_best", "hopes")- Character: “How do you learn best?” – open ended response
head(survey$learn_best)[1] "I'm not sure. I like to read about things I'll need to know, but I also try to practice things I'm learning about."
[2] "Solo projects & putting stuff into practice"
[3] "With more smaller assignments rather than fewer larger assignments"
[4] "Hands on activities and learning by observation"
[5] "I learn best from detailed instructions , hands on experience and being able to ask questions if and when in get stuck"
[6] "Visual learner, with detailed explanations spoken slowly" - Factor: “Where were you come from?” – pre-specified number of options
head(survey$location)[1] "Butte County" "Rest of California"
[3] "Outside of the United States" "Butte County"
[5] "Broader North State (Sac & up)" "Butte County" These pre-specified options are called levels. Factor variables have them, character variables do not.
We can use the levels() function to get to know factor variables.
levels(email$number)[1] "none" "small" "big" There are three levels: none, small, and big.
Let’s look at the variable restaurant from the fast food (ff) data set.
levels(ff$restaurant)NULL- An ordinal variable is a categorical variable where order matters: first, second third
- A nominal variable is a categorical variable where order does not matter: red, green, blue
Factor data types allow us to control the ordering of the levels.
Be the boss of your factors!
Convert a character variable to factor
We use the as_factor() function from the forcats package.
Factor (re)naming
What if the variable is already a factor, but has names we don’t prefer.
Use the fct_recode("NEW" = "old") function here.
email$my_forcats_number <- fct_recode(email$number,
"1M+" = "big",
"None" = "none",
"<1M" = "small")
table(email$number, email$my_forcats_number, useNA="always")
None <1M 1M+ <NA>
none 549 0 0 0
small 0 2827 0 0
big 0 0 545 0
<NA> 0 0 0 0The big factor is now labeled 1M+, none is named None, and small is <1M.
Factor ordering
Let’s look back at the variable restaurant from the fast food (ff) data set.
levels(ff$restaurant)[1] "Mcdonalds" "Chick Fil-A" "Sonic" "Arbys" "Burger King"
[6] "Dairy Queen" "Subway" "Taco Bell" R defaults to alphabetical order in other cases, so beware! You may need to correct the ordering for other data sets.
We need to take control of these factors! We can do that by re-factoring the existing factor variable, but this time specifying the levels of the factor (since it already has labels). Say we decide to order the restaurants by putting all the places that sell burgers together.
Original Order: Arbys, Burger King, Chick Fil-A, Dairy Queen, Mcdonalds, Sonic, Subway, Taco Bell
Desired Order: Arbys, Burger King, Dairy Queen, Mcdonalds, Sonic, Chick Fil-A, Subway, Taco Bell
<-) here, these changes were not made to the variable in the ff data set. The examples below demonstrate making an adjustment to a factor variable and saving that adjustment as a new variable in the data set.Using the fct_relevel function, specify the levels that you want to move.
# results not saved
ff$restaurant |>
fct_relevel("Arbys", "Burger King", "Dairy Queen", "Mcdonalds", "Sonic") |>
fct_count() # new function - acts like table# A tibble: 8 × 2
f n
<fct> <int>
1 Arbys 55
2 Burger King 70
3 Dairy Queen 42
4 Mcdonalds 57
5 Sonic 53
6 Chick Fil-A 27
7 Subway 96
8 Taco Bell 115The fct_relevel function has nice shortcuts as well for example to keep the rest of the ordering but only one or two items. See ?fct_relevel for other options.
ff$restaurant |>
fct_relevel("Subway", after = Inf) |> # move to end
fct_relevel("Taco Bell") |> # move to front
levels()[1] "Taco Bell" "Mcdonalds" "Chick Fil-A" "Sonic" "Arbys"
[6] "Burger King" "Dairy Queen" "Subway" Decreasing number of levels
For analysis purposes, sometimes you want to work with a smaller number of factor variables. Let’s look at the restaurants that are included in the fastfood data set.
table(ff$restaurant)
Mcdonalds Chick Fil-A Sonic Arbys Burger King Dairy Queen
57 27 53 55 70 42
Subway Taco Bell
96 115 Combining multiple categories into one
Let’s combine all the sandwich, and burger joints together. I am going to save this new variable as restaurant_new.
The syntax for the fct_collapse function is new level = "old level", where the “old level” is in quotes. As always, it is good practice to create a two way table to make sure the code typed does what we expected it to do.
ff$restaurant_new <- fct_collapse(ff$restaurant,
BurgerJoint = c("Burger King", "Mcdonalds", "Sonic"),
Sandwich = c("Arbys", "Subway"))
table(ff$restaurant, ff$restaurant_new, useNA="always")
BurgerJoint Chick Fil-A Sandwich Dairy Queen Taco Bell <NA>
Mcdonalds 57 0 0 0 0 0
Chick Fil-A 0 27 0 0 0 0
Sonic 53 0 0 0 0 0
Arbys 0 0 55 0 0 0
Burger King 70 0 0 0 0 0
Dairy Queen 0 0 0 42 0 0
Subway 0 0 96 0 0 0
Taco Bell 0 0 0 0 115 0
<NA> 0 0 0 0 0 0Keeping the most frequent categories
Sometimes we only want to keep the most frequent categories and then lump uncommon factor together levels into “other”. The fct_lump_n function
fct_lump_n(ff$restaurant, n=5) |>
fct_count()# A tibble: 6 × 2
f n
<fct> <int>
1 Mcdonalds 57
2 Arbys 55
3 Burger King 70
4 Subway 96
5 Taco Bell 115
6 Other 122Removing categories entirely
Sometimes, you don’t even want to consider certain levels. This often occurs in survey data where the respondent provides an answer of “Refuse to answer” or the data is coded as the word “missing”. The word “missing’ is fundamentally different than the NA code for a missing value.
For demonstration purposes, let’s get rid of the data from DQ. Who eats something other than ice cream at that place anyhow?
ff$restaurant[ff$restaurant == "Dairy Queen"] <- NA
table(ff$restaurant)
Mcdonalds Chick Fil-A Sonic Arbys Burger King Dairy Queen
57 27 53 55 70 0
Subway Taco Bell
96 115 Even though there are no records with the level Dairy Queen, the level itself still is there. R does not assume just because there are no records with that level, that the named level itself should be removed. We use the function fct_drop to drop the levels with no records.