r4ds/regexps.Rmd

# Regular expressions

```{r, results = "asis", echo = FALSE}
status("restructuring")
```

## Introduction

You learned the basics of regular expressions in Chapter \@ref(strings), but regular expressions really are their own miniature language so it's worth spending some extra time on them.
Regular expressions can be overwhelming at first, and you'll think a cat walked across your keyboard.
Fortunately, as your understanding improves they'll soon start to make sense.

Here we'll focus mostly on pattern language itself, not the functions that use it.
That means we'll mostly work with character vectors, showing the results with `str_view()` and `str_view_all()`.
You'll need to take what you learn and apply it to data frames with tidyr functions or by combining dplyr and stringr functions.

### Prerequisites

This chapter will use regular expressions as provided by the **stringr** package.

```{r setup, message = FALSE}
library(tidyverse)
```

It's worth noting that the regular expressions used by stringr are very slightly different to those of base R.
That's because stringr is built on top of the [stringi package](https://stringi.gagolewski.com), which is in turn built on top of the [ICU engine](https://unicode-org.github.io/icu/userguide/strings/regexp.html), whereas base R functions (like `gsub()` and `grepl()`) use either the [TRE engine](https://github.com/laurikari/tre) or the [PCRE engine](https://www.pcre.org).
Fortunately, the basics of regular expressions are so well established that you're unlikely to encounter any differences when working with the patterns you'll learn in this book.
You only need to be aware of the difference when you start to rely on advanced features like complex Unicode character ranges or special features that use the `(?…)` syntax.
You can learn more about these advanced features in `vignette("regular-expressions", package = "stringr")`.

Another useful reference is [https://www.regular-expressions.info/](https://www.regular-expressions.info/tutorial.html).
It's not R specific, but it includes a lot more information about how regular expressions actually work.

## Escaping {#regexp-escaping}

In Chapter \@ref(strings), you'll learned how to match a literal `.` by using `fixed(".")`.
What if you want to match a literal `.` as part of a regular expression?
You'll need to use an escape, which tells the regular expression you want it to match exactly, not use its special behavior.
Like strings, regexps use the backslash, `\`, to escape special behavior.
So to match a `.`, you need the regexp `\.`.
Unfortunately this creates a problem.
We use strings to represent regular expressions, and `\` is also used as an escape symbol in strings.
So to create the regular expression `\.` we need the string `"\\."`.

```{r}
# To create the regular expression \., we need to use \\.
dot <- "\\."

# But the expression itself only contains one:
str_view(dot)

# And this tells R to look for an explicit .
str_view(c("abc", "a.c", "bef"), "a\\.c")
```

In this book, I'll write regular expression as `\.` and strings that represent the regular expression as `"\\."`.

If `\` is used as an escape character in regular expressions, how do you match a literal `\`?
Well you need to escape it, creating the regular expression `\\`.
To create that regular expression, you need to use a string, which also needs to escape `\`.
That means to match a literal `\` you need to write `"\\\\"` --- you need four backslashes to match one!

```{r}
x <- "a\\b"
str_view(x)
str_view(x, "\\\\")
```

Alternatively, you might find it easier to use the raw strings you learned about in Section \@ref(raw-strings)).
That allows you to avoid one layer of escaping:

```{r}
str_view(x, r"(\\)")
```

The full set of characters with special meanings that need to be escaped is `.^$\|*+?{}[]()`.
In general, look at punctuation character with suspicion; if your regular expression isn't matching what you think it should, check if you've used any of these characters.

As we'll see shortly, escapes can also convert exact matches into special matches.
For example, `s` matches the letter "s", but `\s` matches any whitespace.

### Exercises

1.  Explain why each of these strings don't match a `\`: `"\"`, `"\\"`, `"\\\"`.

2.  How would you match the sequence `"'\`?

3.  What patterns will the regular expression `\..\..\..` match?
    How would you represent it as a string?

## More patterns

With the most important topic of escaping under your belt, now it's time to learn a grab bag of useful patterns.
The following sections will teach you about:

-   Anchors, which allow you to ensure the match is at the start or end of a string.
-   Alternation and parentheses, which allows you to match "this" or "that", and allow you to control which
-   ???
-   Character classes, which allow you to assemble
-   Quantifiers, which controls the number of times a pattern matches
-   Grouping and backreferences

I've tried to the use the technical names for these various components.
They're not always super informative, but they'll usually at least seem somewhat related, and it's helpful to know the correct terms if you later want to google for more information.

### Anchors

By default, regular expressions will match any part of a string.
It's often useful to **anchor** the regular expression so that it matches from the start or to the end of the string.
You can use:

-   `^` to match the start of the string.
-   `$` to match the end of the string.

```{r}
x <- c("apple", "banana", "pear")
str_view(x, "a")  # match "a" anywhere
str_view(x, "^a") # match "a" at start
str_view(x, "a$") # match "a" at end
```

To remember which is which, try this mnemonic which I learned from [Evan Misshula](https://twitter.com/emisshula/status/323863393167613953): if you begin with power (`^`), you end up with money (`$`).

To force a regular expression to only match the full string, anchor it with both `^` and `$`:

```{r}
x <- c("apple pie", "apple", "apple cake")
str_view(x, "apple")
str_view(x, "^apple$")
```

You can also match the boundary between words with `\b`.
I don't often use this in my R code, but I'll sometimes use it when I'm doing a search in RStudio.
It's use to find the name of a function that's a component of other functions.
For example, I'll search for `\bsum\b` to avoid matching `summarise`, `summary`, `rowsum` and so on:

```{r}
x <- c("summary(x)", "summarise(df)", "rowsum(x)", "sum(x)")
str_view(x, "sum")
str_view(x, "\\bsum\\b")
```

### Character classes

You can also create your own collections of characters using `[]`:

-   `[abc]`: matches a, b, or c.
-   `[a-z]`: matches every character between a and z. `[0-9]` matches any number.
-   `[^abc]`: matches anything except a, b, or c.
-   `[\^\-]`: matches `^` or `-`.

A character class containing a single character can be a nice alternative to escapes when you want to include a single special character (i.e. `$` `.` `|` `?` `*` `+` `(` `)` `[` `{`, but not `]` `\` `^`).
This can be more readable because there are fewer slashes, but it also requires a deeper understanding of regular expressions.

```{r}
# Look for a literal character that normally has special meaning in a regex
str_view(c("abc", "a.c", "a*c", "a c"), "a[.]c")
str_view(c("abc", "a.c", "a*c", "a c"), ".[*]c")
str_view(c("abc", "a.c", "a*c", "a c"), "a[ ]")
```

### Shorthand character classes

There are a few character classes that are used so commonly that they get their own shortcut.
You've already seen `.`, which matches any character apart from a newline.
There are three other useful pairs:

-   `\d`: matches any digit; `\D` matches anything that isn't a digit.
-   `\s`: matches any whitespace (e.g. space, tab, newline); `\S` matches anything that isn't whitespace.
-   `\w` matches any "word" character, i.e. letters and numbers; `\W`, matches any non-word character.

Remember, to create a regular expression containing `\d` or `\s`, you'll need to escape the `\` for the string, so you'll type `"\\d"` or `"\\s"`.
The following code demonstrates the different matches with a selection of letters, numbers, and punctuation characters.

```{r}
str_view_all("abcd12345!@#%. ", "\\d+")
str_view_all("abcd12345!@#%. ", "\\D+")
str_view_all("abcd12345!@#%. ", "\\w+")
str_view_all("abcd12345!@#%. ", "\\W+")
str_view_all("abcd12345!@#%. ", "\\s+")
str_view_all("abcd12345!@#%. ", "\\S+")
```

### Quantifiers

The next step up in power involves controlling how many times a pattern matches, the so called **quantifiers**.
We discussed `?` (0 or 1 matches), `+` (1 or more matches), and `*` (0 or more matches) in the last chapter.
Note that the precedence of these operators is high, so you can write: `colou?r` to match either American or British spellings.
That means most uses will need parentheses, like `bana(na)+`.

You can also specify the number of matches precisely:

-   `{n}`: exactly n
-   `{n,}`: n or more
-   `{n,m}`: between n and m

```{r}
x <- "1888 is the longest year in Roman numerals: MDCCCLXXXVIII"
str_view(x, "C{2}")
str_view(x, "C{2,}")
str_view(x, "C{1,3}")
str_view(x, "C{2,3}")
```

By default these matches are **greedy**: they will match the longest string possible.
You can make them **lazy**, matching the shortest string possible by putting a `?` after them.
This is an advanced feature of regular expressions, but it's useful to know that it exists:

```{r}
str_view(x, 'C{2,3}?')
str_view(x, 'C+[LX]+')
str_view(x, 'C+[LX]+?')
```

### Alternation

You can use **alternation** to pick between one or more alternative patterns.
This is a more general form of character classes that's not limited to match single characters.
I recommend always pairing `|` with parentheses, to make it very clear what the alternatives are.
Here are a few examples:

-   Match apple, pear, or banana: `"(apple)|(pear)|(banana)"`
-   Match 3 letters or two digits: `"(\\w{3})|(\\d{3})"`

We'll come back to parentheses very shortly in more detail.

For example, `abc|def` will match either `"abcef"`, or `"abdef"`.
Note that the precedence for `|` is low, so you'll often need to use it with parentheses: `(abc)|(def)` will match either `"abc"`, or `"def"`.
`abc|xyz` matches `abc` or `xyz` not `abcyz` or `abxyz`.
Like with mathematical expressions, if precedence ever gets confusing, use parentheses to make it clear what you want:

```{r}
str_view(c("grey", "gray"), "gr(e|a)y")
```

### Exercises

1.  How would you match the literal string `"$^$"`?

2.  Given the corpus of common words in `stringr::words`, create regular expressions that find all words that:

    a.  Start with "y".
    b.  Don't start with "y".
    c.  End with "x".
    d.  Are exactly three letters long. (Don't cheat by using `str_length()`!)
    e.  Have seven letters or more.

    Since `words` is long, you might want to use the `match` argument to `str_view()` to show only the matching or non-matching words.

3.  Create regular expressions to find all words that:

    a.  Start with a vowel.
    b.  That only contain consonants. (Hint: thinking about matching "not"-vowels.)
    c.  End with `ed`, but not with `eed`.
    d.  End with `ing` or `ise`.

4.  Empirically verify the rule "i before e except after c".

5.  Is "q" always followed by a "u"?

6.  Write a regular expression that matches a `word` if it's probably written in British English, not American English.

7.  Create a regular expression that will match telephone numbers as commonly written in your country.

8.  Describe the equivalents of `?`, `+`, `*` in `{m,n}` form.

9.  Describe in words what these regular expressions match: (read carefully to see if I'm using a regular expression or a string that defines a regular expression.)

    a.  `^.*$`
    b.  `"\\{.+\\}"`
    c.  `\d{4}-\d{2}-\d{2}`
    d.  `"\\\\{4}"`

10. Create regular expressions to find all words that:

    a.  Start with three consonants.
    b.  Have three or more vowels in a row.
    c.  Have two or more vowel-consonant pairs in a row.

11. Solve the beginner regexp crosswords at <https://regexcrossword.com/challenges/beginner>.

## Grouping and capturing

Earlier, you learned about parentheses as a way to disambiguate complex expressions.
Parentheses also create a numbered capturing group (number 1, 2 etc.).
A capturing group stores the part of the string matched by the part of the regular expression inside the parentheses.
You can refer to the same text as previously matched by a capturing group with **backreferences**, like `\1`, `\2` etc.

For example, the following regular expression finds all fruits that have a repeated pair of letters.

```{r}
str_view(fruit, "(..)\\1", match = TRUE)
```

You can also use backreferences when replacing.
The following code will switch the order of the second and third words:

```{r}
sentences %>% 
  str_replace("(\\w+) (\\w+) (\\w+)", "\\1 \\3 \\2") %>% 
  head(5)
```

Names that start and end with the same letter.
Implement with `str_sub()` instead.

### str_match()

```{r}
sentences %>% 
  str_view("the (\\w+) (\\w+)", match = TRUE) %>% 
  head()
```

### Non-capturing groups

Occasionally, you'll want to use parentheses without creating matching groups.
You can create a non-capturing group with `(?:)`.
Typically, however, you'll find it easier to just ignore that result in the output of `str_match()`.

```{r}
x <- c("a gray cat", "a grey dog")
str_match(x, "(gr(e|a)y)")
str_match(x, "(gr(?:e|a)y)")
```

### Exercises

1.  Describe, in words, what these expressions will match:

    a.  `(.)\1\1`
    b.  `"(.)(.)\\2\\1"`
    c.  `(..)\1`
    d.  `"(.).\\1.\\1"`
    e.  `"(.)(.)(.).*\\3\\2\\1"`

2.  Construct regular expressions to match words that:

    a.  Start and end with the same character.
    b.  Contain a repeated pair of letters (e.g. "church" contains "ch" repeated twice.)
    c.  Contain one letter repeated in at least three places (e.g. "eleven" contains three "e"s.)

## Some details

### Overlapping

Matches never overlap, and the regular expression engine only starts looking for a new match after the end of the last match.
For example, in `"abababa"`, how many times will the pattern `"aba"` match?
Regular expressions say two, not three:

```{r}
str_count("abababa", "aba")
str_view_all("abababa", "aba")
```

### Zero width matches

It's possible for a regular expression to match no character, i.e. the space between too characters.
This typically happens when you use a quantifier that allows zero matches:

```{r}
str_view_all("abcdef", "c?")
```

But anchors also create zero-width matches:

```{r}
str_view_all("this is a sentence", "\\b")
str_view_all("this is a sentence", "^")
```

### Greediness

Regular expressions always attempt to match the longest possible string.

### Multi-line strings

-   `dotall = TRUE` allows `.` to match everything, including `\n`.

-   `multiline = TRUE` allows `^` and `$` to match the start and end of each line rather than the start and end of the complete string.

    ```{r}
    x <- "Line 1\nLine 2\nLine 3"
    str_extract_all(x, "^Line")[[1]]
    str_extract_all(x, regex("^Line", multiline = TRUE))[[1]]
    ```

## Flags

When you use a pattern that's a string, it's automatically wrapped into a call to `regex()`:

```{r, eval = FALSE}
# The regular call:
str_view(fruit, "nana")
# Is shorthand for
str_view(fruit, regex("nana"))
```

You can use the other arguments of `regex()` to control details of the match:

-   `ignore_case = TRUE` allows characters to match either their uppercase or lowercase forms.
    This always uses the current locale.

    ```{r}
    bananas <- c("banana", "Banana", "BANANA")
    str_view(bananas, "banana")
    str_view(bananas, regex("banana", ignore_case = TRUE))
    ```

-   `comments = TRUE` allows you to use comments and white space to make complex regular expressions more understandable.
    Spaces are ignored, as is everything after `#`.
    To match a literal space, you'll need to escape it: `"\\ "`.

    ```{r}
    phone <- regex("
      \\(?     # optional opening parens
      (\\d{3}) # area code
      [) -]?   # optional closing parens, space, or dash
      (\\d{3}) # another three numbers
      [ -]?    # optional space or dash
      (\\d{3}) # three more numbers
      ", comments = TRUE)

    str_match("514-791-8141", phone)
    ```

## Strategies

### Using multiple regular expressions

When you have complex logical conditions (e.g. match `a` or `b` but not `c` unless `d`) it's often easier to combine multiple `str_detect()` calls with logical operators instead of trying to create a single regular expression.
For example, here are two ways to find all words that don't contain any vowels:

```{r}
# Find all words containing at least one vowel, and negate
no_vowels_1 <- !str_detect(words, "[aeiou]")
# Find all words consisting only of consonants (non-vowels)
no_vowels_2 <- str_detect(words, "^[^aeiou]+$")
identical(no_vowels_1, no_vowels_2)
```

The results are identical, but I think the first approach is significantly easier to understand.
If your regular expression gets overly complicated, try breaking it up into smaller pieces, giving each piece a name, and then combining the pieces with logical operations.

### Repeated `str_replace()`

### A caution

A word of caution before we finish up this chapter: because regular expressions are so powerful, it's easy to try and solve every problem with a single regular expression.
In the words of Jamie Zawinski:

> Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.

As a cautionary tale, check out this regular expression that checks if a email address is valid:

    (?:(?:\r\n)?[ \t])*(?:(?:(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t]
    )+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:
    \r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(
    ?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ 
    \t]))*"(?:(?:\r\n)?[ \t])*))*@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\0
    31]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\
    ](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+
    (?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:
    (?:\r\n)?[ \t])*))*|(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z
    |(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)
    ?[ \t])*)*\<(?:(?:\r\n)?[ \t])*(?:@(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\
    r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[
     \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)
    ?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t]
    )*))*(?:,@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[
     \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*
    )(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t]
    )+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*)
    *:(?:(?:\r\n)?[ \t])*)?(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+
    |\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r
    \n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:
    \r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t
    ]))*"(?:(?:\r\n)?[ \t])*))*@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031
    ]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](
    ?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?
    :(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?
    :\r\n)?[ \t])*))*\>(?:(?:\r\n)?[ \t])*)|(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?
    :(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?
    [ \t]))*"(?:(?:\r\n)?[ \t])*)*:(?:(?:\r\n)?[ \t])*(?:(?:(?:[^()<>@,;:\\".\[\] 
    \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|
    \\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>
    @,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"
    (?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*))*@(?:(?:\r\n)?[ \t]
    )*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\
    ".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?
    :[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[
    \]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*|(?:[^()<>@,;:\\".\[\] \000-
    \031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(
    ?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)*\<(?:(?:\r\n)?[ \t])*(?:@(?:[^()<>@,;
    :\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([
    ^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\"
    .\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\
    ]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*(?:,@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\
    [\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\
    r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] 
    \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]
    |\\.)*\](?:(?:\r\n)?[ \t])*))*)*:(?:(?:\r\n)?[ \t])*)?(?:[^()<>@,;:\\".\[\] \0
    00-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\
    .|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,
    ;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?
    :[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*))*@(?:(?:\r\n)?[ \t])*
    (?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".
    \[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[
    ^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]
    ]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*\>(?:(?:\r\n)?[ \t])*)(?:,\s*(
    ?:(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\
    ".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)(?:\.(?:(
    ?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[
    \["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t
    ])*))*@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t
    ])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?
    :\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|
    \Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*|(?:
    [^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\
    ]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)*\<(?:(?:\r\n)
    ?[ \t])*(?:@(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["
    ()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)
    ?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>
    @,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*(?:,@(?:(?:\r\n)?[
     \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,
    ;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t]
    )*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\
    ".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*)*:(?:(?:\r\n)?[ \t])*)?
    (?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".
    \[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)(?:\.(?:(?:
    \r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\[
    "()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])
    *))*@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])
    +|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\
    .(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z
    |(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*\>(?:(
    ?:\r\n)?[ \t])*))*)?;\s*)

This is a somewhat pathological example (because email addresses are actually surprisingly complex), but is used in real code.
See the Stack Overflow discussion at <http://stackoverflow.com/a/201378> for more details.

Don't forget that you're in a programming language and you have other tools at your disposal.
Instead of creating one complex regular expression, it's often easier to write a series of simpler regexps.
If you get stuck trying to create a single regexp that solves your problem, take a step back and think if you could break the problem down into smaller pieces, solving each challenge before moving onto the next one.

### Exercises

1.  In the previous example, you might have noticed that the regular expression matched "flickered", which is not a colour. Modify the regex to fix the problem.
2.  Find all words that come after a "number" like "one", "two", "three" etc. Pull out both the number and the word.
3.  Find all contractions. Separate out the pieces before and after the apostrophe.