course
/
r4ds
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

More string re-orgs

This commit is contained in:
Hadley Wickham 2021-12-06 08:41:49 -06:00
parent 915ebf4463
commit a1170ea095
2 changed files with 142 additions and 146 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
prog-strings.Rmd
View File

@ -87,6 +87,17 @@ str_c(
)
```
### `str_dup()`
Closely related to `str_c()` is `str_dup()`.
`str_c(a, a, a)` is like `a + a + a`, what's the equivalent of `3 * a`?
That's `str_dup()`:
```{r}
str_dup(letters[1:3], 3)
str_dup("a", 1:3)
```
## Performance
`fixed()`: matches exactly the specified sequence of bytes.

277
strings.Rmd
View File

@ -6,11 +6,13 @@ status("restructuring")
## Introduction
So far, we've used a bunch of strings without really talking about how they work or the powerful tools you have to work with them.
This chapter begins by diving into the details of creating strings, and from strings, character vectors.
You'll then learn a grab bag of handy string functions before we dive into creating strings from data, then extracting data from strings.
We'll then cover the basics of regular expressions, a powerful, but very concise and sometimes cryptic, language for describing patterns in string.
The chapter concludes with a brief discussion of where your exceptions of English might steer you wrong when working with text from other languages.
So far, you've used a bunch of strings without learning much about the details.
Now it's time to dive into them, learning what makes strings tick, and mastering some of the powerful string manipulation tool you have at your disposal.
We'll begin with the details of creating strings and character vectors.
You'll then learn a grab bag of handy string functions before we dive into creating strings from data.
Next, we'll discuss the basics of regular expressions, a powerful tool for describing patterns in strings, then use those tools to extract data from strings.
The chapter finishes up with a brief discussion where English language expectations might steer you wrong when working with text from other languages.
This chapter is paired with two other chapters.
Regular expression are a big topic, so we'll come back to them again in Chapter \@ref(regular-expressions).
@ -36,12 +38,9 @@ knitr::include_graphics("screenshots/stringr-autocomplete.png")
## Creating a string
To begin, let's discuss the mechanics of creating a string[^strings-1].
We've created strings in passing earlier in the book, but didn't discuss the details.
First, there are two basic ways to create a string: using either single quotes (`'`) or double quotes (`"`).
Unlike other languages, there is no difference in behaviour, but the [tidyverse style guide](https://style.tidyverse.org/syntax.html#character-vectors) recommends using `"`, unless the string contains multiple `"`
[^strings-1]: A string is a length-1 character vector.
Unlike other languages, there is no difference in behaviour, but the [tidyverse style guide](https://style.tidyverse.org/syntax.html#character-vectors) recommends using `"`, unless the string contains multiple `"`.
```{r}
string1 <- "This is a string"
@ -55,7 +54,7 @@ If you forget to close a quote, you'll see `+`, the continuation character:
+
+ HELP I'M STUCK
If this happen to you and you can't figure out which quote you need to close, press Escape to cancel, then try again.
If this happen to you and you can't figure out which quote you need to close, press Escape to cancel, and try again.
### Escapes
@ -66,16 +65,16 @@ double_quote <- "\"" # or '"'
single_quote <- '\'' # or "'"
```
This means if you want to include a literal backslash in your string, you'll need to double it up: `"\\"`:
So if you want to include a literal backslash in your string, you'll need to double it up: `"\\"`:
```{r}
backslash <- "\\"
```
Beware that the printed representation of a string is not the same as string itself, because the printed representation shows the escapes.
To see the raw contents of the string, use `str_view()` [^strings-2]:
To see the raw contents of the string, use `str_view()` [^strings-1]:
[^strings-2]: You can also use the base R function `writeLines()`
[^strings-1]: You can also use the base R function `writeLines()`
```{r}
x <- c(single_quote, double_quote, backslash)
@ -95,15 +94,15 @@ str_view(tricky)
```
That's a lot of backslashes!
(I like the evocative name for this problem: [leaning toothpick syndome](https://en.wikipedia.org/wiki/Leaning_toothpick_syndrome))
To eliminate the escaping you can instead use a **raw string**[^strings-3]:
To eliminate the escaping you can instead use a **raw string**[^strings-2]:
[^strings-3]: Available in R 4.0.0 and above.
[^strings-2]: Available in R 4.0.0 and above.
```{r}
tricky <- r"(double_quote <- "\"" # or '"'
single_quote <- '\'' # or "'"
)"
single_quote <- '\'' # or "'")"
str_view(tricky)
```
@ -122,8 +121,6 @@ x
str_view(x)
```
Now that you've learned the basics of creating strings by "hand", we'll go into the details of creating strings from other strings, starting with combining strings.
### Vectors
You can combine multiple strings into a character vector by using `c()`:
@ -133,8 +130,12 @@ x <- c("first string", "second string", "third string")
x
```
You can create a length zero character vector with `character()`.
This is not usually very useful, but can help you understand the general principle of functions by giving them an unusual input.
Technically, a string is a length-1 character vector, but this doesn't have much bearing on your data analysis live.
If needed, you can create a length zero character vector with `character()`.
This is not generally very useful, but because it's the shortest possible vector, it can sometimes be useful for determining the general pattern of a function by feeding it an extreme.
Now that you've learned the basics of creating strings by "hand", we'll go into the details of creating strings from other strings, starting with a grab bag of small, but useful functions.
### Exercises
@ -142,17 +143,16 @@ This is not usually very useful, but can help you understand the general princip
### Length
It's natural to think about the letters that make up an individual string.
(Not every language uses letters, which we'll talk about more in Section \@ref(other-languages)).
For example, `str_length()` tells you the length of a string in characters:
It's often natural to think about the letters that make up an individual string.
`str_length()` tells you the length of a string:
```{r}
str_length(c("a", "R for data science", NA))
```
You could use this with `count()` to find the distribution of lengths of US babynames, and then with `filter()` to look at the longest names[^strings-4]:
You could use this with `count()` to find the distribution of lengths of US babynames, and then with `filter()` to look at the longest names[^strings-3]:
[^strings-4]: Looking at these entries, I'd say the babynames data removes spaces or hyphens from names and truncates after 15 letters.
[^strings-3]: Looking at these entries, I'd say the babynames data removes spaces or hyphens from names and truncates after 15 letters.
```{r}
babynames %>%
@ -163,6 +163,22 @@ babynames %>%
count(name, wt = n, sort = TRUE)
```
### Long strings
Sometimes the reason you care about the length of a string is because you're trying to fit it into a label on a plot or in a table.
stringr provides two useful tools for cases where your string is too long:
- `str_trunc(x, 20)` ensures that no string is longer than 20 characters, replacing any thing too long with `…`.
- `str_wrap(x, 20)` wraps a string introducing new lines so that each line is at most 20 characters (it doesn't hyphenate, however, so any word longer than 20 characters will make a longer time)
```{r}
x <- "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat."
str_trunc(x, 30)
str_view(str_wrap(x, 30))
```
### Subsetting
You can extract parts of a string using `str_sub(string, start, end)`.
@ -195,113 +211,69 @@ babynames %>%
)
```
Later, we'll come back to the problem of extracting data from strings.
### Long strings
Sometimes the reason you care about the length of a string is because you're trying to fit it into a label.
stringr provides two useful tools for cases where your string is too long:
- `str_trunc(x, 20)` ensures that no string is longer than 20 characters, replacing any thing too long with `…`.
- `str_wrap(x, 20)` wraps a string introducing new lines so that each line is at most 20 characters (it doesn't hyphenate, however, so any word longer than 20 characters will make a longer time)
```{r}
x <- "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat."
str_trunc(x, 30)
str_view(str_wrap(x, 30))
```
### Exercises
1. Use `str_length()` and `str_sub()` to extract the middle letter from each baby name. What will you do if the string has an even number of characters?
2. Are there any major trends in the length of babynames over time? What about the popularity of first and last letters?
## Combining strings
## Creating strings from data
There are two ways in which you might want to combine strings.
You might have a few character vectors which you want to combine together creating a new vector.
Or you might have a single vector that you want to collapse down into a single string.
### str_c()
### `str_c()`
Use `str_c()`[^strings-5] to join together multiple character vectors into a single vector:
Use `str_c()`[^strings-4] to join together multiple character vectors into a single vector:
[^strings-5]: `str_c()` is very similar to the base `paste0()`.
There are two main reasons I use it here: it obeys the usual rules for handling `NA`, and it uses the tidyverse recycling rules.
[^strings-4]: `str_c()` is very similar to the base `paste0()`.
There are two main reasons I recommend: it obeys the usual rules for handling `NA` and it uses the tidyverse recycling rules.
```{r}
str_c("x", "y")
str_c("x", "y", "z")
```
`str_c()` obeys the tidyverse recycling rules so any length-1 vectors (aka strings) will be recycled to the length of the longest vector[^strings-6]:
[^strings-6]: If the other vectors don't have the same length, `str_c()` will error.
`str_c()` is designed to be used with `mutate()` so it obeys the usual tidyverse recycling and missing value rules:
```{r}
names <- c("Timothy", "Dewey", "Mable")
str_c("Hi ", names, "!")
df <- tibble(name = c("Timothy", "Dewey", "Mable", NA))
df %>% mutate(greeting = str_c("Hi ", name, "!"))
```
Like most other functions in R, missing values are contagious, so any missing input will cause the output to be missing.
If you don't want this behaviour, use `coalesce()` to replace missing values with something else:
If you want missing values to display in some other way, use `coalesce()` either inside or outside of `str_c()`:
```{r}
x <- c("abc", NA)
str_c("|-", x, "-|")
str_c("|-", coalesce(x, ""), "-|")
df %>% mutate(
greeting1 = str_c("Hi ", coalesce(name, "you"), "!"),
greeting2 = coalesce(str_c("Hi ", name, "!"), "Hi!")
)
```
Since `str_c()` creates a vector, you'll usually use it with `mutate()`:
### `str_glue()`
One of the downsides of `str_c()` is that you have to constantly open and close the string in order to include variables.
An alternative approach is provided by the glue package with `str_glue()`[^strings-5] .
Glue works a little differently to `str_c()`: you give it a single string that uses `{}` to indicate where variables should be evaluated:
[^strings-5]: If you're not using stringr, you can also access it directly with `glue::glue().`
```{r}
starwars %>%
mutate(greeting = str_c("Hi! I'm ", name, "."), .after = name)
```
### `str_dup()`
`str_c(a, a, a)` is like `a + a + a`, what's the equivalent of `3 * a`?
That's `str_dup()`:
```{r}
str_dup(letters[1:3], 3)
str_dup("a", 1:3)
```
### Glue
Another powerful way of combining strings is with the glue package.
You can either use `glue::glue()` directly or call it via the `str_glue()` wrapper that stringr provides for you.
Glue works a little differently to the other methods: you give it a single string then within the string use `{}` to indicate where existing variables should be evaluated:
```{r}
x <- c("abc", NA)
str_glue("|-{x}-|")
```
Like `str_c()`, `str_glue()` pairs well with `mutate()`:
```{r}
starwars %>%
mutate(
intro = str_glue("Hi! My is {name} and I'm a {species} from {homeworld}"),
.keep = "none"
)
df %>% mutate(greeting = str_glue("Hi {name}!"))
```
You can use any valid R code inside of `{}`, but it's a good idea to pull complex calculations out into their own variables so you can more easily check your work.
Differences with `NA` handling?
Currently, `str_glue()` is slightly inconsistent `str_c()` but we'll hopefully fix that by the time the book is printed: <https://github.com/tidyverse/glue/issues/246>
### `str_flatten()`
So far I've shown you vectorised functions that work will with `mutate()`: the output of these functions is the same length as the input.
There's one last important function that's a summary function: the output is always length 1, regardless of the length of the input.
That's `str_flatten()`:[^strings-7] it takes a character vector and always returns a single string:
`str_c()` and `glue()` are work well with `mutate()` because the output is the same length as the input.
What if you want a function that works well with `summarise()`, a function who's output is always length 1, regardless of the length of the input?
[^strings-7]: The base R equivalent is `paste()` with the `collapse` argument set.
That's the job of `str_flatten()`:[^strings-6] it takes a character vector and always returns a single string:
[^strings-6]: The base R equivalent is `paste()` with the `collapse` argument set.
```{r}
str_flatten(c("x", "y", "z"))
@ -309,8 +281,7 @@ str_flatten(c("x", "y", "z"), ", ")
str_flatten(c("x", "y", "z"), ", ", last = ", and ")
```
Just like `sum()` and `mean()` take a vector of numbers and return a single number, `str_flatten()` takes a character vector and returns a single string.
This makes `str_flatten()` a summary function for strings, so you'll often pair it with `summarise()`:
Which makes it work well with `summarise()`:
```{r}
df <- tribble(
@ -339,15 +310,11 @@ df %>%
2. What does `str_flatten()` return if you give it a length 0 character vector?
## Splitting apart strings
Common for multiple variables worth of data to be stored in a single string.
In this section you'll learn how to use various functions tidyr to extract them.
Waiting on: <https://github.com/tidyverse/tidyups/pull/15>
## Working with patterns
Before we can discuss the opposite problem of extracting data out of strings, we need to take a quick digression to talk about **regular expressions**.
Regular expressions are a very concise language for describing patterns in strings.
### Detect matches
To determine if a character vector matches a pattern, use `str_detect()`.
@ -379,38 +346,6 @@ babynames %>%
(Note that this gives us the proportion of names that contain an x; if you wanted the proportion of babies given a name containing an x, you'd need to perform a weighted mean).
### Count matches
A variation on `str_detect()` is `str_count()`: rather than a simple yes or no, it tells you how many matches there are in a string:
```{r}
str_count(x, "p")
```
It's natural to use `str_count()` with `mutate()`:
```{r}
babynames %>%
distinct(name) %>%
mutate(
vowels = str_count(name, "[aeiou]"),
consonants = str_count(name, "[^aeiou]")
)
```
You also wonder if any names include special characters like periods:
```{r}
babynames %>%
distinct(name) %>%
head() %>%
mutate(
periods = str_count(name, "."),
)
```
That's weird!
### Introduction to regular expressions
To understand what's going on, we need to discuss what the second argument to `str_detect()` really is.
@ -459,7 +394,36 @@ str_view_all("x X xy", regex(".Y", ignore_case = TRUE))
We'll come back to case later, because it's not trivial for many languages.
### Replacing matches
### Count matches
A variation on `str_detect()` is `str_count()`: rather than a simple yes or no, it tells you how many matches there are in a string:
```{r}
str_count(x, "p")
```
It's natural to use `str_count()` with `mutate()`:
```{r}
babynames %>%
count(name) %>%
mutate(
vowels = str_count(name, "[aeiou]"),
consonants = str_count(name, "[^aeiou]")
)
```
```{r}
babynames %>%
count(name, wt = n) %>%
mutate(
vowels = str_count(name, regex("[aeiouy]", ignore_case = TRUE)),
consonants = str_count(name, regex("[^aeiouy]", ignore_case = TRUE)),
ratio = vowels / consonants
)
```
### Replace matches
`str_replace_all()` allow you to replace matches with new strings.
The simplest use is to replace a pattern with a fixed string:
@ -503,6 +467,13 @@ Recommendation to make a function, and think about testing it --- don't need for
5. Switch the first and last letters in `words`.
Which of those strings are still `words`?
## Extract data from strings
Common for multiple variables worth of data to be stored in a single string.
In this section you'll learn how to use various functions tidyr to extract them.
Waiting on: <https://github.com/tidyverse/tidyups/pull/15>
## Locale dependent operations {#other-languages}
So far all of our examples have been using English.
@ -512,16 +483,18 @@ The details of the many ways other languages are different to English are too di
- Words are composed of individual spaces.
- All letters in a word are written down.
The locale is specified as a ISO 639 language code, which is a two or three letter abbreviation.
The locale is usually a ISO 639 language code, which is a two or three letter abbreviation like "en" for English, "fr" for French, and "es" for Spanish.
If you don't already know the code for your language, [Wikipedia](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) has a good list, and you can see which are supported with `stringi::stri_locale_list()`.
Base R string functions automatically use your locale current locale, but stringr functions all default to the English locale.
This ensures that your code works the same way on every system, avoiding subtle bugs.
To choose a different locale you'll need to specify the `locale` argument; seeing that a function has a locale argument tells you that its behaviour will differ from locale to locale.
Here are a few places where locale matter:S
Here are a few places where locale matter:
- Upper and lower case: only relatively few languages have upper and lower case (Latin, Greek, and Cyrillic, plus a handful of lessor known languages). The rules are not te same in every language that uses these alphabets. For example, Turkish has two i's: with and without a dot, and it has a different rule for capitalising them:
- Upper and lower case: only relatively few languages have upper and lower case (Latin, Greek, and Cyrillic, plus a handful of lessor known languages).
The rules are not te same in every language that uses these alphabets.
For example, Turkish has two i's: with and without a dot, and it has a different rule for capitalising them:
```{r}
str_to_upper(c("i", "ı"))
@ -549,11 +522,23 @@ Here are a few places where locale matter:S
str_view(a1, coll(a2))
```
- Another important operation that's affected by the locale is sorting. The base R `order()` and `sort()` functions sort strings using the current locale. If you want robust behaviour across different computers, you may want to use `str_sort()` and `str_order()` which take an additional `locale` argument. Here's an example: in Czech, "ch" is a digraph that appears after `h` in the alphabet.
- Another important operation that's affected by the locale is sorting.
The base R `order()` and `sort()` functions sort strings using the current locale.
If you want robust behaviour across different computers, you may want to use `str_sort()` and `str_order()` which take an additional `locale` argument.
Here's an example: in Czech, "ch" is a digraph that appears after `h` in the alphabet.
```{r}
str_sort(c("a", "ch", "c", "h"))
str_sort(c("a", "ch", "c", "h"), locale = "cs")
str_sort(c("a", "c", "ch", "h", "z"))
str_sort(c("a", "c", "ch", "h", "z"), locale = "cs")
```
Danish has a similar problem.
Normally, characters with diacritic sorts after the plain character.
But in Danish ø and å are letters that come at the end of the alphabet:
```{r}
str_sort(c("a", "å", "o", "ø", "z"))
str_sort(c("a", "å", "o", "ø", "z"), locale = "da")
```
TODO after dplyr 1.1.0: discuss `arrange()`