556 lines
18 KiB
Plaintext
556 lines
18 KiB
Plaintext
# Strings
|
||
|
||
## Introduction
|
||
|
||
This chapter introduces you to strings in R.
|
||
You'll learn the basics of how strings work and how to create them by hand.
|
||
Big topic so spread over three chapters.
|
||
|
||
Base R contains many functions to work with strings but we'll generally avoid them here because they can be inconsistent, which makes them hard to remember.
|
||
Instead, we'll use stringr which is designed to be as consistent as possible, and all of its functions start with `str_`.
|
||
The common `str_` prefix is particularly useful if you use RStudio, because typing `str_` will trigger autocomplete, allowing you to see all stringr functions:
|
||
|
||
```{r, echo = FALSE}
|
||
knitr::include_graphics("screenshots/stringr-autocomplete.png")
|
||
```
|
||
|
||
### Prerequisites
|
||
|
||
This chapter will focus on the **stringr** package for string manipulation, which is part of the core tidyverse.
|
||
|
||
```{r setup, message = FALSE}
|
||
library(tidyverse)
|
||
library(babynames)
|
||
```
|
||
|
||
## Creating a string
|
||
|
||
You can create strings with either single quotes or double quotes.
|
||
Unlike other languages, there is no difference in behaviour.
|
||
I recommend always using `"`, unless you want to create a string that contains multiple `"`.
|
||
|
||
```{r}
|
||
string1 <- "This is a string"
|
||
string2 <- 'If I want to include a "quote" inside a string, I use single quotes'
|
||
```
|
||
|
||
If you forget to close a quote, you'll see `+`, the continuation character:
|
||
|
||
> "This is a string without a closing quote
|
||
+
|
||
+
|
||
+ HELP I'M STUCK
|
||
|
||
If this happen to you, press Escape and try again!
|
||
|
||
To include a literal single or double quote in a string you can use `\` to "escape" it:
|
||
|
||
```{r}
|
||
double_quote <- "\"" # or '"'
|
||
single_quote <- '\'' # or "'"
|
||
```
|
||
|
||
That means if you want to include a literal backslash, you'll need to double it up: `"\\"`.
|
||
|
||
Beware that the printed representation of a string is not the same as string itself, because the printed representation shows the escapes.
|
||
To see the raw contents of the string, use `writeLines()`:
|
||
|
||
```{r}
|
||
x <- c("\"", "\\")
|
||
x
|
||
str_view(x)
|
||
```
|
||
|
||
As shown above, you can combine strings into a (character) vector with `c()`:
|
||
|
||
```{r}
|
||
c("one", "two", "three")
|
||
```
|
||
|
||
### Raw strings
|
||
|
||
Creating a string with multiple quotes or backslashes gets confusing quickly.
|
||
For example, lets create a string that contains the contents of the chunk above:
|
||
|
||
```{r}
|
||
tricky <- "double_quote <- \"\\\"\" # or '\"'
|
||
single_quote <- '\\'' # or \"'\""
|
||
str_view(tricky)
|
||
```
|
||
|
||
In R 4.0.0 and above, you can use a **raw** string to reduce the amount of escaping:
|
||
|
||
```{r}
|
||
tricky <- r"(double_quote <- "\"" # or '"'
|
||
single_quote <- '\'' # or "'"
|
||
)"
|
||
str_view(tricky)
|
||
```
|
||
|
||
A raw string starts with `r"(` and finishes with `)"`.
|
||
If your string contains `)"` you can instead use `r"[]"` or `r"{}"`, and if that's still not enough, you can insert any number of dashes to make the opening and closing pairs unique: `` `r"--()--" ``.
|
||
|
||
### Other special characters
|
||
|
||
As well as `\"`, `\'`, and `\\` there are a handful of other special characters that may come in handy. The most common are `"\n"`, newline, and `"\t"`, tab, but you can see the complete list by requesting help on `"` with `?'"'` or `?"'"`.
|
||
|
||
You'll also sometimes see strings containing Unicode escapes like `"\u00b5"`.
|
||
This is a way of writing non-English characters that works on all platforms:
|
||
|
||
```{r}
|
||
x <- "\u00b5"
|
||
x
|
||
```
|
||
|
||
## Combining strings
|
||
|
||
To combine two or more strings, use `str_c()`:
|
||
|
||
```{r}
|
||
str_c("x", "y")
|
||
str_c("x", "y", "z")
|
||
```
|
||
|
||
Use the `sep` argument to control how they're separated:
|
||
|
||
```{r}
|
||
str_c("x", "y", sep = ", ")
|
||
```
|
||
|
||
Like most other functions in R, missing values are contagious.
|
||
As usual, if you want to show a different value, use `coalesce()`:
|
||
|
||
```{r}
|
||
x <- c("abc", NA)
|
||
str_c("|-", x, "-|")
|
||
str_c("|-", coalesce(x, ""), "-|")
|
||
```
|
||
|
||
`mutate()`
|
||
|
||
Another powerful way of combining strings is with the glue package.
|
||
You can either use `glue::glue()` or call it via the `str_glue()` wrapper that string provides for you.
|
||
Glue works a little differently to the other methods: you give it a single string using `{}` to indicate where you want to interpolate in existing variables:
|
||
|
||
```{r}
|
||
str_glue("|-{x}-|")
|
||
```
|
||
|
||
Like `str_c()`, `str_glue()` pairs well with `mutate()`:
|
||
|
||
```{r}
|
||
starwars %>% mutate(
|
||
intro = str_glue("Hi my is {name} and I'm a {species} from {homeworld}"),
|
||
.keep = "none"
|
||
)
|
||
```
|
||
|
||
## Length and subsetting
|
||
|
||
For example, `str_length()` tells you the length of a string:
|
||
|
||
```{r}
|
||
str_length(c("a", "R for data science", NA))
|
||
```
|
||
|
||
You could use this with `count()` to find the distribution of lengths of US babynames:
|
||
|
||
```{r}
|
||
babynames %>%
|
||
count(length = str_length(name))
|
||
```
|
||
|
||
You can extract parts of a string using `str_sub()`.
|
||
As well as the string, `str_sub()` takes `start` and `end` arguments which give the (inclusive) position of the substring:
|
||
|
||
```{r}
|
||
x <- c("Apple", "Banana", "Pear")
|
||
str_sub(x, 1, 3)
|
||
# negative numbers count backwards from end
|
||
str_sub(x, -3, -1)
|
||
```
|
||
|
||
We could use this with `mutate()` to find the first and last letter of each name:
|
||
|
||
```{r}
|
||
babynames %>%
|
||
mutate(
|
||
first = str_sub(name, 1, 1),
|
||
last = str_sub(name, -1, -1)
|
||
)
|
||
```
|
||
|
||
Note that `str_sub()` won't fail if the string is too short: it will just return as much as possible:
|
||
|
||
```{r}
|
||
str_sub("a", 1, 5)
|
||
```
|
||
|
||
Note that the idea of a "letter" isn't a natural fit to every language, so you'll need to take care if you're working with text from other languages.
|
||
We'll briefly talk about some of the issues in Section \@ref(other-languages).
|
||
|
||
TODO: `separate()`
|
||
|
||
### Exercises
|
||
|
||
1. In code that doesn't use stringr, you'll often see `paste()` and `paste0()`.
|
||
What's the difference between the two functions?
|
||
What stringr function are they equivalent to?
|
||
How do the functions differ in their handling of `NA`?
|
||
|
||
2. In your own words, describe the difference between the `sep` and `collapse` arguments to `str_c()`.
|
||
|
||
3. Use `str_length()` and `str_sub()` to extract the middle character from a string.
|
||
What will you do if the string has an even number of characters?
|
||
|
||
4. Write a function that turns (e.g.) a vector `c("a", "b", "c")` into the string `a, b, and c`.
|
||
Think carefully about what it should do if given a vector of length 0, 1, or 2.
|
||
|
||
## String summaries
|
||
|
||
You can perform the opposite operation with `summarise()` and `str_flatten()`:
|
||
|
||
To collapse a vector of strings into a single string, use `collapse`:
|
||
|
||
```{r}
|
||
str_flatten(c("x", "y", "z"), ", ")
|
||
```
|
||
|
||
This is a great tool for `summarise()`ing character data.
|
||
Later we'll come back to the inverse of this, `separate_rows()`.
|
||
|
||
## Long strings
|
||
|
||
`str_wrap()`
|
||
|
||
`str_trunc()`
|
||
|
||
## Introduction to regular expressions
|
||
|
||
Opting out by using `fixed()`
|
||
|
||
## Detect matches
|
||
|
||
To determine if a character vector matches a pattern, use `str_detect()`.
|
||
It returns a logical vector the same length as the input:
|
||
|
||
```{r}
|
||
x <- c("apple", "banana", "pear")
|
||
str_detect(x, "e")
|
||
```
|
||
|
||
Remember that when you use a logical vector in a numeric context, `FALSE` becomes 0 and `TRUE` becomes 1.
|
||
That makes `sum()` and `mean()` useful if you want to answer questions about matches across a larger vector:
|
||
|
||
```{r}
|
||
# How many common words start with t?
|
||
sum(str_detect(words, "^t"))
|
||
# What proportion of common words end with a vowel?
|
||
mean(str_detect(words, "[aeiou]$"))
|
||
```
|
||
|
||
When you have complex logical conditions (e.g. match a or b but not c unless d) it's often easier to combine multiple `str_detect()` calls with logical operators, rather than trying to create a single regular expression.
|
||
For example, here are two ways to find all words that don't contain any vowels:
|
||
|
||
```{r}
|
||
# Find all words containing at least one vowel, and negate
|
||
no_vowels_1 <- !str_detect(words, "[aeiou]")
|
||
# Find all words consisting only of consonants (non-vowels)
|
||
no_vowels_2 <- str_detect(words, "^[^aeiou]+$")
|
||
identical(no_vowels_1, no_vowels_2)
|
||
```
|
||
|
||
The results are identical, but I think the first approach is significantly easier to understand.
|
||
If your regular expression gets overly complicated, try breaking it up into smaller pieces, giving each piece a name, and then combining the pieces with logical operations.
|
||
|
||
A common use of `str_detect()` is to select the elements that match a pattern.
|
||
This makes it a natural pairing with `filter()`.
|
||
The following regexp finds all names with repeated pairs of letters (you'll learn how that regexp works in the next chapter)
|
||
|
||
```{r}
|
||
babynames %>%
|
||
filter(n > 100) %>%
|
||
count(name, wt = n) %>%
|
||
filter(str_detect(name, "(..).*\\1"))
|
||
```
|
||
|
||
A variation on `str_detect()` is `str_count()`: rather than a simple yes or no, it tells you how many matches there are in a string:
|
||
|
||
```{r}
|
||
x <- c("apple", "banana", "pear")
|
||
str_count(x, "a")
|
||
|
||
# On average, how many vowels per word?
|
||
mean(str_count(words, "[aeiou]"))
|
||
```
|
||
|
||
It's natural to use `str_count()` with `mutate()`:
|
||
|
||
```{r}
|
||
babynames %>%
|
||
mutate(
|
||
vowels = str_count(name, "[aeiou]"),
|
||
consonants = str_count(name, "[^aeiou]")
|
||
)
|
||
```
|
||
|
||
### Exercises
|
||
|
||
1. For each of the following challenges, try solving it by using both a single regular expression, and a combination of multiple `str_detect()` calls.
|
||
|
||
a. Find all words that start or end with `x`.
|
||
b. Find all words that start with a vowel and end with a consonant.
|
||
c. Are there any words that contain at least one of each different vowel?
|
||
|
||
2. What word has the highest number of vowels?
|
||
What word has the highest proportion of vowels?
|
||
(Hint: what is the denominator?)
|
||
|
||
## Replacing matches
|
||
|
||
`str_replace_all()` allow you to replace matches with new strings.
|
||
The simplest use is to replace a pattern with a fixed string:
|
||
|
||
```{r}
|
||
x <- c("apple", "pear", "banana")
|
||
str_replace_all(x, "[aeiou]", "-")
|
||
```
|
||
|
||
With `str_replace_all()` you can perform multiple replacements by supplying a named vector.
|
||
The name gives a regular expression to match, and the value gives the replacement.
|
||
|
||
```{r}
|
||
x <- c("1 house", "1 person has 2 cars", "3 people")
|
||
str_replace_all(x, c("1" = "one", "2" = "two", "3" = "three"))
|
||
```
|
||
|
||
Use in `mutate()`
|
||
|
||
#### Exercises
|
||
|
||
1. Replace all forward slashes in a string with backslashes.
|
||
|
||
2. Implement a simple version of `str_to_lower()` using `str_replace_all()`.
|
||
|
||
3. Switch the first and last letters in `words`.
|
||
Which of those strings are still words?
|
||
|
||
## Extract full matches
|
||
|
||
To extract the actual text of a match, use `str_extract()`.
|
||
To show that off, we're going to need a more complicated example.
|
||
I'm going to use the [Harvard sentences](https://en.wikipedia.org/wiki/Harvard_sentences), which were designed to test VOIP systems, but are also useful for practicing regexps.
|
||
These are provided in `stringr::sentences`:
|
||
|
||
```{r}
|
||
length(sentences)
|
||
head(sentences)
|
||
```
|
||
|
||
Imagine we want to find all sentences that contain a colour.
|
||
We first create a vector of colour names, and then turn it into a single regular expression:
|
||
|
||
```{r}
|
||
colours <- c("red", "orange", "yellow", "green", "blue", "purple")
|
||
colour_match <- str_c(colours, collapse = "|")
|
||
colour_match
|
||
```
|
||
|
||
Now we can select the sentences that contain a colour, and then extract the colour to figure out which one it is:
|
||
|
||
```{r}
|
||
has_colour <- str_subset(sentences, colour_match)
|
||
matches <- str_extract(has_colour, colour_match)
|
||
head(matches)
|
||
```
|
||
|
||
Note that `str_extract()` only extracts the first match.
|
||
We can see that most easily by first selecting all the sentences that have more than 1 match:
|
||
|
||
```{r}
|
||
more <- sentences[str_count(sentences, colour_match) > 1]
|
||
str_view_all(more, colour_match)
|
||
|
||
str_extract(more, colour_match)
|
||
```
|
||
|
||
This is a common pattern for stringr functions, because working with a single match allows you to use much simpler data structures.
|
||
To get all matches, use `str_extract_all()`.
|
||
It returns a list, so we'll come back to this later on.
|
||
|
||
### Exercises
|
||
|
||
1. In the previous example, you might have noticed that the regular expression matched "flickered", which is not a colour. Modify the regex to fix the problem.
|
||
|
||
## Extract part of matches
|
||
|
||
If your data is in a tibble, it's often easier to use `tidyr::extract()`.
|
||
It works like `str_match()` but requires you to name the matches, which are then placed in new columns:
|
||
|
||
```{r}
|
||
tibble(sentence = sentences) %>%
|
||
tidyr::extract(
|
||
sentence, c("article", "noun"), "(a|the) ([^ ]+)",
|
||
remove = FALSE
|
||
)
|
||
```
|
||
|
||
#### Exercises
|
||
|
||
1. Find all words that come after a "number" like "one", "two", "three" etc.
|
||
Pull out both the number and the word.
|
||
|
||
2. Find all contractions.
|
||
Separate out the pieces before and after the apostrophe.
|
||
|
||
## Strings -\> Columns
|
||
|
||
## Separate
|
||
|
||
`separate()` pulls apart one column into multiple columns, by splitting wherever a separator character appears.
|
||
Take `table3`:
|
||
|
||
```{r}
|
||
table3
|
||
```
|
||
|
||
The `rate` column contains both `cases` and `population` variables, and we need to split it into two variables.
|
||
`separate()` takes the name of the column to separate, and the names of the columns to separate into, as shown in Figure \@ref(fig:tidy-separate) and the code below.
|
||
|
||
```{r}
|
||
table3 %>%
|
||
separate(rate, into = c("cases", "population"))
|
||
```
|
||
|
||
```{r tidy-separate, echo = FALSE, out.width = "75%", fig.cap = "Separating `rate` into `cases` and `population` to make `table3` tidy", fig.alt = "Two panels, one with a data frame with three columns (country, year, and rate) and the other with a data frame with four columns (country, year, cases, and population). Arrows show how the rate variable is separated into two variables: cases and population."}
|
||
knitr::include_graphics("images/tidy-17.png")
|
||
```
|
||
|
||
By default, `separate()` will split values wherever it sees a non-alphanumeric character (i.e. a character that isn't a number or letter).
|
||
For example, in the code above, `separate()` split the values of `rate` at the forward slash characters.
|
||
If you wish to use a specific character to separate a column, you can pass the character to the `sep` argument of `separate()`.
|
||
For example, we could rewrite the code above as:
|
||
|
||
```{r eval = FALSE}
|
||
table3 %>%
|
||
separate(rate, into = c("cases", "population"), sep = "/")
|
||
```
|
||
|
||
`separate_rows()`
|
||
|
||
## Strings -\> Rows
|
||
|
||
```{r}
|
||
starwars %>%
|
||
select(name, eye_color) %>%
|
||
filter(str_detect(eye_color, ", ")) %>%
|
||
separate_rows(eye_color)
|
||
```
|
||
|
||
### Exercises
|
||
|
||
1. Split up a string like `"apples, pears, and bananas"` into individual components.
|
||
|
||
2. Why is it better to split up by `boundary("word")` than `" "`?
|
||
|
||
3. What does splitting with an empty string (`""`) do?
|
||
Experiment, and then read the documentation.
|
||
|
||
## Other languages {#other-languages}
|
||
|
||
Encoding, and why not to trust `Encoding`.
|
||
As a general rule, we recommend using UTF-8 everywhere, converting as a early as possible (i.e. by using the `encoding` argument to `readr::locale()`).
|
||
|
||
### Length and subsetting
|
||
|
||
This seems like a straightforward computation if you're only familiar with English, but things get complex quick when working with other languages.
|
||
Include some examples from <https://gankra.github.io/blah/text-hates-you/>.
|
||
|
||
This is a problem even with European problem because Unicode provides two ways of representing characters with accents: many common characters have a special codepoint, but others can be built up from individual components.
|
||
|
||
```{r}
|
||
x <- c("\u00e1", "a\u0301")
|
||
x
|
||
str_length(x)
|
||
str_sub(x, 1, 1)
|
||
```
|
||
|
||
### Locales
|
||
|
||
Above I used `str_to_lower()` to change the text to lower case.
|
||
You can also use `str_to_upper()` or `str_to_title()`.
|
||
However, changing case is more complicated than it might at first appear because different languages have different rules for changing case.
|
||
You can pick which set of rules to use by specifying a locale:
|
||
|
||
```{r}
|
||
# Turkish has two i's: with and without a dot, and it
|
||
# has a different rule for capitalising them:
|
||
str_to_upper(c("i", "ı"))
|
||
str_to_upper(c("i", "ı"), locale = "tr")
|
||
```
|
||
|
||
The locale is specified as a ISO 639 language code, which is a two or three letter abbreviation.
|
||
If you don't already know the code for your language, [Wikipedia](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) has a good list.
|
||
If you leave the locale blank, it will use English.
|
||
|
||
Another important operation that's affected by the locale is sorting.
|
||
The base R `order()` and `sort()` functions sort strings using the current locale.
|
||
If you want robust behaviour across different computers, you may want to use `str_sort()` and `str_order()` which take an additional `locale` argument:
|
||
|
||
```{r}
|
||
x <- c("apple", "eggplant", "banana")
|
||
|
||
str_sort(x, locale = "en") # English
|
||
|
||
str_sort(x, locale = "haw") # Hawaiian
|
||
```
|
||
|
||
TODO: add connection to `arrange()`
|
||
|
||
### `coll()`
|
||
|
||
Beware using `fixed()` with non-English data.
|
||
It is problematic because there are often multiple ways of representing the same character.
|
||
For example, there are two ways to define "á": either as a single character or as an "a" plus an accent:
|
||
|
||
```{r}
|
||
a1 <- "\u00e1"
|
||
a2 <- "a\u0301"
|
||
c(a1, a2)
|
||
a1 == a2
|
||
```
|
||
|
||
They render identically, but because they're defined differently, `fixed()` doesn't find a match.
|
||
Instead, you can use `coll()`, defined next, to respect human character comparison rules:
|
||
|
||
```{r}
|
||
str_detect(a1, fixed(a2))
|
||
str_detect(a1, coll(a2))
|
||
```
|
||
|
||
-
|
||
|
||
`coll()`: compare strings using standard **coll**ation rules.
|
||
This is useful for doing case insensitive matching.
|
||
Note that `coll()` takes a `locale` parameter that controls which rules are used for comparing characters.
|
||
Unfortunately different parts of the world use different rules!
|
||
|
||
```{r}
|
||
# That means you also need to be aware of the difference
|
||
# when doing case insensitive matches:
|
||
i <- c("I", "İ", "i", "ı")
|
||
i
|
||
|
||
str_subset(i, coll("i", ignore_case = TRUE))
|
||
str_subset(i, coll("i", ignore_case = TRUE, locale = "tr"))
|
||
```
|
||
|
||
Both `fixed()` and `regex()` have `ignore_case` arguments, but they do not allow you to pick the locale: they always use the default locale.
|
||
You can see what that is with the following code; more on stringi later.
|
||
|
||
```{r}
|
||
stringi::stri_locale_info()
|
||
```
|
||
|
||
The downside of `coll()` is speed; because the rules for recognising which characters are the same are complicated, `coll()` is relatively slow compared to `regex()` and `fixed()`.
|