551 lines
25 KiB
Plaintext
551 lines
25 KiB
Plaintext
# Regular expressions
|
|
|
|
```{r, results = "asis", echo = FALSE}
|
|
status("restructuring")
|
|
```
|
|
|
|
## Introduction
|
|
|
|
You learned the basics of regular expressions in Chapter \@ref(strings), but regular expressions really are their own miniature language so it's worth spending some extra time on them.
|
|
Regular expressions can be overwhelming at first, and you'll think a cat walked across your keyboard.
|
|
Fortunately, as your understanding improves they'll soon start to make sense.
|
|
|
|
Here we'll focus mostly on pattern language itself, not the functions that use it.
|
|
That means we'll mostly work with character vectors, showing the results with `str_view()` and `str_view_all()`.
|
|
You'll need to take what you learn and apply it to data frames with tidyr functions or by combining dplyr and stringr functions.
|
|
|
|
### Prerequisites
|
|
|
|
This chapter will use regular expressions as provided by the **stringr** package.
|
|
|
|
```{r setup, message = FALSE}
|
|
library(tidyverse)
|
|
```
|
|
|
|
It's worth noting that the regular expressions used by stringr are very slightly different to those of base R.
|
|
That's because stringr is built on top of the [stringi package](https://stringi.gagolewski.com), which is in turn built on top of the [ICU engine](https://unicode-org.github.io/icu/userguide/strings/regexp.html), whereas base R functions (like `gsub()` and `grepl()`) use either the [TRE engine](https://github.com/laurikari/tre) or the [PCRE engine](https://www.pcre.org).
|
|
Fortunately, the basics of regular expressions are so well established that you're unlikely to encounter any differences when working with the patterns you'll learn in this book.
|
|
You only need to be aware of the difference when you start to rely on advanced features like complex Unicode character ranges or special features that use the `(?…)` syntax.
|
|
You can learn more about these advanced features in `vignette("regular-expressions", package = "stringr")`.
|
|
|
|
Another useful reference is [https://www.regular-expressions.info/](https://www.regular-expressions.info/tutorial.html).
|
|
It's not R specific, but it includes a lot more information about how regular expressions actually work.
|
|
|
|
## Escaping {#regexp-escaping}
|
|
|
|
In Chapter \@ref(strings), you'll learned how to match a literal `.` by using `fixed(".")`.
|
|
What if you want to match a literal `.` as part of a regular expression?
|
|
You'll need to use an escape, which tells the regular expression you want it to match exactly, not use its special behavior.
|
|
Like strings, regexps use the backslash, `\`, to escape special behavior.
|
|
So to match a `.`, you need the regexp `\.`.
|
|
Unfortunately this creates a problem.
|
|
We use strings to represent regular expressions, and `\` is also used as an escape symbol in strings.
|
|
So to create the regular expression `\.` we need the string `"\\."`.
|
|
|
|
```{r}
|
|
# To create the regular expression \., we need to use \\.
|
|
dot <- "\\."
|
|
|
|
# But the expression itself only contains one:
|
|
str_view(dot)
|
|
|
|
# And this tells R to look for an explicit .
|
|
str_view(c("abc", "a.c", "bef"), "a\\.c")
|
|
```
|
|
|
|
In this book, I'll write regular expression as `\.` and strings that represent the regular expression as `"\\."`.
|
|
|
|
If `\` is used as an escape character in regular expressions, how do you match a literal `\`?
|
|
Well you need to escape it, creating the regular expression `\\`.
|
|
To create that regular expression, you need to use a string, which also needs to escape `\`.
|
|
That means to match a literal `\` you need to write `"\\\\"` --- you need four backslashes to match one!
|
|
|
|
```{r}
|
|
x <- "a\\b"
|
|
str_view(x)
|
|
str_view(x, "\\\\")
|
|
```
|
|
|
|
Alternatively, you might find it easier to use the raw strings you learned about in Section \@ref(raw-strings)).
|
|
That allows you to avoid one layer of escaping:
|
|
|
|
```{r}
|
|
str_view(x, r"(\\)")
|
|
```
|
|
|
|
The full set of characters with special meanings that need to be escaped is `.^$\|*+?{}[]()`.
|
|
In general, look at punctuation character with suspicion; if your regular expression isn't matching what you think it should, check if you've used any of these characters.
|
|
|
|
As we'll see shortly, escapes can also convert exact matches into special matches.
|
|
For example, `s` matches the letter "s", but `\s` matches any whitespace.
|
|
|
|
### Exercises
|
|
|
|
1. Explain why each of these strings don't match a `\`: `"\"`, `"\\"`, `"\\\"`.
|
|
|
|
2. How would you match the sequence `"'\`?
|
|
|
|
3. What patterns will the regular expression `\..\..\..` match?
|
|
How would you represent it as a string?
|
|
|
|
## More patterns
|
|
|
|
With the most important topic of escaping under your belt, now it's time to learn a grab bag of useful patterns.
|
|
The following sections will teach you about:
|
|
|
|
- Anchors, which allow you to ensure the match is at the start or end of a string.
|
|
- Alternation and parentheses, which allows you to match "this" or "that", and allow you to control which
|
|
- ???
|
|
- Character classes, which allow you to assemble
|
|
- Quantifiers, which controls the number of times a pattern matches
|
|
- Grouping and backreferences
|
|
|
|
I've tried to the use the technical names for these various components.
|
|
They're not always super informative, but they'll usually at least seem somewhat related, and it's helpful to know the correct terms if you later want to google for more information.
|
|
|
|
### Anchors
|
|
|
|
By default, regular expressions will match any part of a string.
|
|
It's often useful to **anchor** the regular expression so that it matches from the start or to the end of the string.
|
|
You can use:
|
|
|
|
- `^` to match the start of the string.
|
|
- `$` to match the end of the string.
|
|
|
|
```{r}
|
|
x <- c("apple", "banana", "pear")
|
|
str_view(x, "a") # match "a" anywhere
|
|
str_view(x, "^a") # match "a" at start
|
|
str_view(x, "a$") # match "a" at end
|
|
```
|
|
|
|
To remember which is which, try this mnemonic which I learned from [Evan Misshula](https://twitter.com/emisshula/status/323863393167613953): if you begin with power (`^`), you end up with money (`$`).
|
|
|
|
To force a regular expression to only match the full string, anchor it with both `^` and `$`:
|
|
|
|
```{r}
|
|
x <- c("apple pie", "apple", "apple cake")
|
|
str_view(x, "apple")
|
|
str_view(x, "^apple$")
|
|
```
|
|
|
|
You can also match the boundary between words with `\b`.
|
|
I don't often use this in my R code, but I'll sometimes use it when I'm doing a search in RStudio.
|
|
It's use to find the name of a function that's a component of other functions.
|
|
For example, I'll search for `\bsum\b` to avoid matching `summarise`, `summary`, `rowsum` and so on:
|
|
|
|
```{r}
|
|
x <- c("summary(x)", "summarise(df)", "rowsum(x)", "sum(x)")
|
|
str_view(x, "sum")
|
|
str_view(x, "\\bsum\\b")
|
|
```
|
|
|
|
### Character classes
|
|
|
|
You can also create your own collections of characters using `[]`:
|
|
|
|
- `[abc]`: matches a, b, or c.
|
|
- `[a-z]`: matches every character between a and z. `[0-9]` matches any number.
|
|
- `[^abc]`: matches anything except a, b, or c.
|
|
- `[\^\-]`: matches `^` or `-`.
|
|
|
|
A character class containing a single character can be a nice alternative to escapes when you want to include a single special character (i.e. `$` `.` `|` `?` `*` `+` `(` `)` `[` `{`, but not `]` `\` `^`).
|
|
This can be more readable because there are fewer slashes, but it also requires a deeper understanding of regular expressions.
|
|
|
|
```{r}
|
|
# Look for a literal character that normally has special meaning in a regex
|
|
str_view(c("abc", "a.c", "a*c", "a c"), "a[.]c")
|
|
str_view(c("abc", "a.c", "a*c", "a c"), ".[*]c")
|
|
str_view(c("abc", "a.c", "a*c", "a c"), "a[ ]")
|
|
```
|
|
|
|
### Shorthand character classes
|
|
|
|
There are a few character classes that are used so commonly that they get their own shortcut.
|
|
You've already seen `.`, which matches any character apart from a newline.
|
|
There are three other useful pairs:
|
|
|
|
- `\d`: matches any digit; `\D` matches anything that isn't a digit.
|
|
- `\s`: matches any whitespace (e.g. space, tab, newline); `\S` matches anything that isn't whitespace.
|
|
- `\w` matches any "word" character, i.e. letters and numbers; `\W`, matches any non-word character.
|
|
|
|
Remember, to create a regular expression containing `\d` or `\s`, you'll need to escape the `\` for the string, so you'll type `"\\d"` or `"\\s"`.
|
|
The following code demonstrates the different matches with a selection of letters, numbers, and punctuation characters.
|
|
|
|
```{r}
|
|
str_view_all("abcd12345!@#%. ", "\\d+")
|
|
str_view_all("abcd12345!@#%. ", "\\D+")
|
|
str_view_all("abcd12345!@#%. ", "\\w+")
|
|
str_view_all("abcd12345!@#%. ", "\\W+")
|
|
str_view_all("abcd12345!@#%. ", "\\s+")
|
|
str_view_all("abcd12345!@#%. ", "\\S+")
|
|
```
|
|
|
|
### Quantifiers
|
|
|
|
The next step up in power involves controlling how many times a pattern matches, the so called **quantifiers**.
|
|
We discussed `?` (0 or 1 matches), `+` (1 or more matches), and `*` (0 or more matches) in the last chapter.
|
|
Note that the precedence of these operators is high, so you can write: `colou?r` to match either American or British spellings.
|
|
That means most uses will need parentheses, like `bana(na)+`.
|
|
|
|
You can also specify the number of matches precisely:
|
|
|
|
- `{n}`: exactly n
|
|
- `{n,}`: n or more
|
|
- `{n,m}`: between n and m
|
|
|
|
```{r}
|
|
x <- "1888 is the longest year in Roman numerals: MDCCCLXXXVIII"
|
|
str_view(x, "C{2}")
|
|
str_view(x, "C{2,}")
|
|
str_view(x, "C{1,3}")
|
|
str_view(x, "C{2,3}")
|
|
```
|
|
|
|
By default these matches are **greedy**: they will match the longest string possible.
|
|
You can make them **lazy**, matching the shortest string possible by putting a `?` after them.
|
|
This is an advanced feature of regular expressions, but it's useful to know that it exists:
|
|
|
|
```{r}
|
|
str_view(x, 'C{2,3}?')
|
|
str_view(x, 'C+[LX]+')
|
|
str_view(x, 'C+[LX]+?')
|
|
```
|
|
|
|
### Alternation
|
|
|
|
You can use **alternation** to pick between one or more alternative patterns.
|
|
This is a more general form of character classes that's not limited to match single characters.
|
|
I recommend always pairing `|` with parentheses, to make it very clear what the alternatives are.
|
|
Here are a few examples:
|
|
|
|
- Match apple, pear, or banana: `"(apple)|(pear)|(banana)"`
|
|
- Match 3 letters or two digits: `"(\\w{3})|(\\d{3})"`
|
|
|
|
We'll come back to parentheses very shortly in more detail.
|
|
|
|
For example, `abc|def` will match either `"abcef"`, or `"abdef"`.
|
|
Note that the precedence for `|` is low, so you'll often need to use it with parentheses: `(abc)|(def)` will match either `"abc"`, or `"def"`.
|
|
`abc|xyz` matches `abc` or `xyz` not `abcyz` or `abxyz`.
|
|
Like with mathematical expressions, if precedence ever gets confusing, use parentheses to make it clear what you want:
|
|
|
|
```{r}
|
|
str_view(c("grey", "gray"), "gr(e|a)y")
|
|
```
|
|
|
|
### Exercises
|
|
|
|
1. How would you match the literal string `"$^$"`?
|
|
|
|
2. Given the corpus of common words in `stringr::words`, create regular expressions that find all words that:
|
|
|
|
a. Start with "y".
|
|
b. Don't start with "y".
|
|
c. End with "x".
|
|
d. Are exactly three letters long. (Don't cheat by using `str_length()`!)
|
|
e. Have seven letters or more.
|
|
|
|
Since `words` is long, you might want to use the `match` argument to `str_view()` to show only the matching or non-matching words.
|
|
|
|
3. Create regular expressions to find all words that:
|
|
|
|
a. Start with a vowel.
|
|
b. That only contain consonants. (Hint: thinking about matching "not"-vowels.)
|
|
c. End with `ed`, but not with `eed`.
|
|
d. End with `ing` or `ise`.
|
|
|
|
4. Empirically verify the rule "i before e except after c".
|
|
|
|
5. Is "q" always followed by a "u"?
|
|
|
|
6. Write a regular expression that matches a `word` if it's probably written in British English, not American English.
|
|
|
|
7. Create a regular expression that will match telephone numbers as commonly written in your country.
|
|
|
|
8. Describe the equivalents of `?`, `+`, `*` in `{m,n}` form.
|
|
|
|
9. Describe in words what these regular expressions match: (read carefully to see if I'm using a regular expression or a string that defines a regular expression.)
|
|
|
|
a. `^.*$`
|
|
b. `"\\{.+\\}"`
|
|
c. `\d{4}-\d{2}-\d{2}`
|
|
d. `"\\\\{4}"`
|
|
|
|
10. Create regular expressions to find all words that:
|
|
|
|
a. Start with three consonants.
|
|
b. Have three or more vowels in a row.
|
|
c. Have two or more vowel-consonant pairs in a row.
|
|
|
|
11. Solve the beginner regexp crosswords at <https://regexcrossword.com/challenges/beginner>.
|
|
|
|
## Grouping and capturing
|
|
|
|
Earlier, you learned about parentheses as a way to disambiguate complex expressions.
|
|
Parentheses also create a numbered capturing group (number 1, 2 etc.).
|
|
A capturing group stores the part of the string matched by the part of the regular expression inside the parentheses.
|
|
You can refer to the same text as previously matched by a capturing group with **backreferences**, like `\1`, `\2` etc.
|
|
|
|
For example, the following regular expression finds all fruits that have a repeated pair of letters.
|
|
|
|
```{r}
|
|
str_view(fruit, "(..)\\1", match = TRUE)
|
|
```
|
|
|
|
You can also use backreferences when replacing.
|
|
The following code will switch the order of the second and third words:
|
|
|
|
```{r}
|
|
sentences %>%
|
|
str_replace("(\\w+) (\\w+) (\\w+)", "\\1 \\3 \\2") %>%
|
|
head(5)
|
|
```
|
|
|
|
Names that start and end with the same letter.
|
|
Implement with `str_sub()` instead.
|
|
|
|
### str_match()
|
|
|
|
```{r}
|
|
sentences %>%
|
|
str_view("the (\\w+) (\\w+)", match = TRUE) %>%
|
|
head()
|
|
```
|
|
|
|
### Non-capturing groups
|
|
|
|
Occasionally, you'll want to use parentheses without creating matching groups.
|
|
You can create a non-capturing group with `(?:)`.
|
|
Typically, however, you'll find it easier to just ignore that result in the output of `str_match()`.
|
|
|
|
```{r}
|
|
x <- c("a gray cat", "a grey dog")
|
|
str_match(x, "(gr(e|a)y)")
|
|
str_match(x, "(gr(?:e|a)y)")
|
|
```
|
|
|
|
### Exercises
|
|
|
|
1. Describe, in words, what these expressions will match:
|
|
|
|
a. `(.)\1\1`
|
|
b. `"(.)(.)\\2\\1"`
|
|
c. `(..)\1`
|
|
d. `"(.).\\1.\\1"`
|
|
e. `"(.)(.)(.).*\\3\\2\\1"`
|
|
|
|
2. Construct regular expressions to match words that:
|
|
|
|
a. Start and end with the same character.
|
|
b. Contain a repeated pair of letters (e.g. "church" contains "ch" repeated twice.)
|
|
c. Contain one letter repeated in at least three places (e.g. "eleven" contains three "e"s.)
|
|
|
|
## Some details
|
|
|
|
### Overlapping
|
|
|
|
Matches never overlap, and the regular expression engine only starts looking for a new match after the end of the last match.
|
|
For example, in `"abababa"`, how many times will the pattern `"aba"` match?
|
|
Regular expressions say two, not three:
|
|
|
|
```{r}
|
|
str_count("abababa", "aba")
|
|
str_view_all("abababa", "aba")
|
|
```
|
|
|
|
### Zero width matches
|
|
|
|
It's possible for a regular expression to match no character, i.e. the space between too characters.
|
|
This typically happens when you use a quantifier that allows zero matches:
|
|
|
|
```{r}
|
|
str_view_all("abcdef", "c?")
|
|
```
|
|
|
|
But anchors also create zero-width matches:
|
|
|
|
```{r}
|
|
str_view_all("this is a sentence", "\\b")
|
|
str_view_all("this is a sentence", "^")
|
|
```
|
|
|
|
### Greediness
|
|
|
|
Regular expressions always attempt to match the longest possible string.
|
|
|
|
### Multi-line strings
|
|
|
|
- `dotall = TRUE` allows `.` to match everything, including `\n`.
|
|
|
|
- `multiline = TRUE` allows `^` and `$` to match the start and end of each line rather than the start and end of the complete string.
|
|
|
|
```{r}
|
|
x <- "Line 1\nLine 2\nLine 3"
|
|
str_extract_all(x, "^Line")[[1]]
|
|
str_extract_all(x, regex("^Line", multiline = TRUE))[[1]]
|
|
```
|
|
|
|
## Flags
|
|
|
|
When you use a pattern that's a string, it's automatically wrapped into a call to `regex()`:
|
|
|
|
```{r, eval = FALSE}
|
|
# The regular call:
|
|
str_view(fruit, "nana")
|
|
# Is shorthand for
|
|
str_view(fruit, regex("nana"))
|
|
```
|
|
|
|
You can use the other arguments of `regex()` to control details of the match:
|
|
|
|
- `ignore_case = TRUE` allows characters to match either their uppercase or lowercase forms.
|
|
This always uses the current locale.
|
|
|
|
```{r}
|
|
bananas <- c("banana", "Banana", "BANANA")
|
|
str_view(bananas, "banana")
|
|
str_view(bananas, regex("banana", ignore_case = TRUE))
|
|
```
|
|
|
|
- `comments = TRUE` allows you to use comments and white space to make complex regular expressions more understandable.
|
|
Spaces are ignored, as is everything after `#`.
|
|
To match a literal space, you'll need to escape it: `"\\ "`.
|
|
|
|
```{r}
|
|
phone <- regex("
|
|
\\(? # optional opening parens
|
|
(\\d{3}) # area code
|
|
[) -]? # optional closing parens, space, or dash
|
|
(\\d{3}) # another three numbers
|
|
[ -]? # optional space or dash
|
|
(\\d{3}) # three more numbers
|
|
", comments = TRUE)
|
|
|
|
str_match("514-791-8141", phone)
|
|
```
|
|
|
|
## Strategies
|
|
|
|
### Using multiple regular expressions
|
|
|
|
When you have complex logical conditions (e.g. match `a` or `b` but not `c` unless `d`) it's often easier to combine multiple `str_detect()` calls with logical operators instead of trying to create a single regular expression.
|
|
For example, here are two ways to find all words that don't contain any vowels:
|
|
|
|
```{r}
|
|
# Find all words containing at least one vowel, and negate
|
|
no_vowels_1 <- !str_detect(words, "[aeiou]")
|
|
# Find all words consisting only of consonants (non-vowels)
|
|
no_vowels_2 <- str_detect(words, "^[^aeiou]+$")
|
|
identical(no_vowels_1, no_vowels_2)
|
|
```
|
|
|
|
The results are identical, but I think the first approach is significantly easier to understand.
|
|
If your regular expression gets overly complicated, try breaking it up into smaller pieces, giving each piece a name, and then combining the pieces with logical operations.
|
|
|
|
### Repeated `str_replace()`
|
|
|
|
### A caution
|
|
|
|
A word of caution before we finish up this chapter: because regular expressions are so powerful, it's easy to try and solve every problem with a single regular expression.
|
|
In the words of Jamie Zawinski:
|
|
|
|
> Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.
|
|
|
|
As a cautionary tale, check out this regular expression that checks if a email address is valid:
|
|
|
|
(?:(?:\r\n)?[ \t])*(?:(?:(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t]
|
|
)+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:
|
|
\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(
|
|
?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[
|
|
\t]))*"(?:(?:\r\n)?[ \t])*))*@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\0
|
|
31]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\
|
|
](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+
|
|
(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:
|
|
(?:\r\n)?[ \t])*))*|(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z
|
|
|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)
|
|
?[ \t])*)*\<(?:(?:\r\n)?[ \t])*(?:@(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\
|
|
r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[
|
|
\t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)
|
|
?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t]
|
|
)*))*(?:,@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[
|
|
\t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*
|
|
)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t]
|
|
)+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*)
|
|
*:(?:(?:\r\n)?[ \t])*)?(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+
|
|
|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r
|
|
\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:
|
|
\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t
|
|
]))*"(?:(?:\r\n)?[ \t])*))*@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031
|
|
]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](
|
|
?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?
|
|
:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?
|
|
:\r\n)?[ \t])*))*\>(?:(?:\r\n)?[ \t])*)|(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?
|
|
:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?
|
|
[ \t]))*"(?:(?:\r\n)?[ \t])*)*:(?:(?:\r\n)?[ \t])*(?:(?:(?:[^()<>@,;:\\".\[\]
|
|
\000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|
|
|
\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>
|
|
@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"
|
|
(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*))*@(?:(?:\r\n)?[ \t]
|
|
)*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\
|
|
".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?
|
|
:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[
|
|
\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*|(?:[^()<>@,;:\\".\[\] \000-
|
|
\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(
|
|
?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)*\<(?:(?:\r\n)?[ \t])*(?:@(?:[^()<>@,;
|
|
:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([
|
|
^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\"
|
|
.\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\
|
|
]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*(?:,@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\
|
|
[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\
|
|
r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\]
|
|
\000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]
|
|
|\\.)*\](?:(?:\r\n)?[ \t])*))*)*:(?:(?:\r\n)?[ \t])*)?(?:[^()<>@,;:\\".\[\] \0
|
|
00-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\
|
|
.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,
|
|
;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|"(?
|
|
:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*))*@(?:(?:\r\n)?[ \t])*
|
|
(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".
|
|
\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t])*(?:[
|
|
^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\]
|
|
]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*\>(?:(?:\r\n)?[ \t])*)(?:,\s*(
|
|
?:(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\
|
|
".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)(?:\.(?:(
|
|
?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[
|
|
\["()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t
|
|
])*))*@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t
|
|
])+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?
|
|
:\.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|
|
|
\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*|(?:
|
|
[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".\[\
|
|
]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)*\<(?:(?:\r\n)
|
|
?[ \t])*(?:@(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["
|
|
()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)
|
|
?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>
|
|
@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*(?:,@(?:(?:\r\n)?[
|
|
\t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,
|
|
;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\.(?:(?:\r\n)?[ \t]
|
|
)*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\
|
|
".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*)*:(?:(?:\r\n)?[ \t])*)?
|
|
(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\["()<>@,;:\\".
|
|
\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])*)(?:\.(?:(?:
|
|
\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z|(?=[\[
|
|
"()<>@,;:\\".\[\]]))|"(?:[^\"\r\\]|\\.|(?:(?:\r\n)?[ \t]))*"(?:(?:\r\n)?[ \t])
|
|
*))*@(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])
|
|
+|\Z|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*)(?:\
|
|
.(?:(?:\r\n)?[ \t])*(?:[^()<>@,;:\\".\[\] \000-\031]+(?:(?:(?:\r\n)?[ \t])+|\Z
|
|
|(?=[\["()<>@,;:\\".\[\]]))|\[([^\[\]\r\\]|\\.)*\](?:(?:\r\n)?[ \t])*))*\>(?:(
|
|
?:\r\n)?[ \t])*))*)?;\s*)
|
|
|
|
This is a somewhat pathological example (because email addresses are actually surprisingly complex), but is used in real code.
|
|
See the Stack Overflow discussion at <http://stackoverflow.com/a/201378> for more details.
|
|
|
|
Don't forget that you're in a programming language and you have other tools at your disposal.
|
|
Instead of creating one complex regular expression, it's often easier to write a series of simpler regexps.
|
|
If you get stuck trying to create a single regexp that solves your problem, take a step back and think if you could break the problem down into smaller pieces, solving each challenge before moving onto the next one.
|
|
|
|
### Exercises
|
|
|
|
1. In the previous example, you might have noticed that the regular expression matched "flickered", which is not a colour. Modify the regex to fix the problem.
|
|
2. Find all words that come after a "number" like "one", "two", "three" etc. Pull out both the number and the word.
|
|
3. Find all contractions. Separate out the pieces before and after the apostrophe.
|