257 lines
10 KiB
Plaintext
257 lines
10 KiB
Plaintext
# Pipes
|
|
|
|
```{r, include = FALSE}
|
|
library(dplyr)
|
|
diamonds <- ggplot2::diamonds
|
|
```
|
|
|
|
Pipes let you transform the way you call deeply nested functions. Using a pipe doesn't affect at all what the code does; behind the scenes it is run in (almost) exactly the same way. What the pipe does is change how the code is written and hence how it is read. It tends to transform to a more imperative form (do this, do that, do that other thing, ...) so that it's easier to read.
|
|
|
|
### Piping alternatives
|
|
|
|
To explore how you can write the same code in many different ways, let's use code to tell a story about a little bunny named foo foo:
|
|
|
|
> Little bunny Foo Foo
|
|
> Went hopping through the forest
|
|
> Scooping up the field mice
|
|
> And bopping them on the head
|
|
|
|
We'll start by defining an object to represent little bunny Foo Foo:
|
|
|
|
```{r, eval = FALSE}
|
|
foo_foo <- little_bunny()
|
|
```
|
|
|
|
And then we'll use a function for each key verb `hop()`, `scoop()`, and `bop()`. Using this object and these verbs, there are a number of ways we could retell the story in code:
|
|
|
|
* Save each intermediate step as a new object
|
|
* Rewrite the original object multiple times
|
|
* Compose functions
|
|
* Use the pipe
|
|
|
|
Below we work through each approach, showing you the code and talking about the advantages and disadvantages.
|
|
|
|
#### Intermediate steps
|
|
|
|
The simplest and most robust approach to sequencing multiple function calls is to save each intermediary as a new object:
|
|
|
|
```{r, eval = FALSE}
|
|
foo_foo_1 <- hop(foo_foo, through = forest)
|
|
foo_foo_2 <- scoop(foo_foo_1, up = field_mice)
|
|
foo_foo_3 <- bop(foo_foo_2, on = head)
|
|
```
|
|
|
|
The main downside of this form is that it forces you to name each intermediate element. If there are natural names, this form feels natural, and you should use it. But if you're giving then arbitrary unique names, like this example, I don't think it's that useful. Whenever I write code like this, I invariably write the wrong number somewhere and then spend 10 minutes scratching my head and trying to figure out what went wrong with my code.
|
|
|
|
You may worry that this form creates many intermediate copies of your data and takes up a lot of memory. First, in R, worrying about memory is not a useful way to spend your time: worry about it when it becomes a problem (i.e. you run out of memory), not before. Second, R isn't stupid: it will reuse the shared columns in a pipeline of data frame transformations. Let's take a look at an actual data manipulation pipeline where we add a new column to the `diamonds` dataset from ggplot2:
|
|
|
|
```{r}
|
|
diamonds2 <- mutate(diamonds, price_per_carat = price / carat)
|
|
|
|
library(pryr)
|
|
object_size(diamonds)
|
|
object_size(diamonds2)
|
|
object_size(diamonds, diamonds2)
|
|
```
|
|
|
|
`pryr::object_size()` gives the memory occupied by all of its arguments. The results seem counterintuitive at first:
|
|
|
|
* `diamonds` takes up 3.46 MB,
|
|
* `diamonds2` takes up 3.89 MB,
|
|
* `diamonds` and `diamonds2` together take up 3.89 MB!
|
|
|
|
How can that work? Well, `diamonds2` has 10 columns in common with `diamonds`: there's no need to duplicate all that data so both data frames share the vectors. R will only create a copy of a vector if you modify it. Modifying a single value will mean that the data frames can no longer share as much memory. The individual sizes will be unchanged, but the collective size will increase:
|
|
|
|
```{r}
|
|
diamonds$carat[1] <- NA
|
|
object_size(diamonds)
|
|
object_size(diamonds2)
|
|
object_size(diamonds, diamonds2)
|
|
```
|
|
|
|
(Note that we use `pryr::object_size()` here, not the built-in `object.size()`, because it doesn't have quite enough smarts.)
|
|
|
|
#### Overwrite the original
|
|
|
|
One way to eliminate the intermediate objects is to just overwrite the same object again and again:
|
|
|
|
```{r, eval = FALSE}
|
|
foo_foo <- hop(foo_foo, through = forest)
|
|
foo_foo <- scoop(foo_foo, up = field_mice)
|
|
foo_foo <- bop(foo_foo, on = head)
|
|
```
|
|
|
|
This is less typing (and less thinking), so you're less likely to make mistakes. However, there are two problems:
|
|
|
|
1. It will make debugging painful: if you make a mistake you'll need to start
|
|
again from scratch.
|
|
|
|
1. The repetition of the object being transformed (we've written `foo_foo` six
|
|
times!) obscures what's changing on each line.
|
|
|
|
#### Function composition
|
|
|
|
Another approach is to abandon assignment altogether and just string the function calls together:
|
|
|
|
```{r, eval = FALSE}
|
|
bop(
|
|
scoop(
|
|
hop(foo_foo, through = forest),
|
|
up = field_mice
|
|
),
|
|
on = head
|
|
)
|
|
```
|
|
|
|
Here the disadvantage is that you have to read from inside-out, from right-to-left, and that the arguments end up spread far apart (sometimes called the
|
|
[dagwood sandwhich](https://en.wikipedia.org/wiki/Dagwood_sandwich) problem).
|
|
|
|
#### Use the pipe
|
|
|
|
Finally, we can use the pipe:
|
|
|
|
```{r, eval = FALSE}
|
|
foo_foo %>%
|
|
hop(through = forest) %>%
|
|
scoop(up = field_mouse) %>%
|
|
bop(on = head)
|
|
```
|
|
|
|
This is my favourite form. The downside is that you need to understand what the pipe does, but once you've mastered that idea task, you can read this series of function compositions like it's a set of imperative actions. Foo foo, hops, then scoops, then bops.
|
|
|
|
Behind the scenes magrittr converts this to:
|
|
|
|
```{r, eval = FALSE}
|
|
. <- hop(foo_foo, through = forest)
|
|
. <- scoop(., up = field_mice)
|
|
bop(., on = head)
|
|
```
|
|
|
|
It's useful to know this because if an error is thrown in the middle of the pipe, you'll need to be able to interpret the `traceback()`.
|
|
|
|
### Other tools from magrittr
|
|
|
|
The pipe is provided by the magrittr package, by Stefan Milton Bache. Most of packages you work in this book automatically provide `%>%` for you. You might want to load magrittr yourself if you're using another package, or you want to access some of the other pipe variants that magrittr provides.
|
|
|
|
```{r}
|
|
library(magrittr)
|
|
```
|
|
|
|
* When working with more complex pipes, it's some times useful to call a
|
|
function for its side-effects. Maybe you want to print out the current
|
|
object, or plot it, or save it to disk. Many times, such functions don't
|
|
return anything, effectively terminating the pipe.
|
|
|
|
To work around this problem, you can use the "tee" pipe. `%T>%` works like
|
|
`%>%` except instead it returns the LHS instead of the RHS. It's called
|
|
"tee" because it's like a literal T-shaped pipe.
|
|
|
|
```{r}
|
|
rnorm(100) %>%
|
|
matrix(ncol = 2) %>%
|
|
plot() %>%
|
|
str()
|
|
|
|
rnorm(100) %>%
|
|
matrix(ncol = 2) %T>%
|
|
plot() %>%
|
|
str()
|
|
```
|
|
|
|
* If you're working with functions that don't have a dataframe based API
|
|
(i.e. you pass them individual vectors, not a data frame and expressions
|
|
to be evaluated in the context of that data frame), you might find `%$%`
|
|
useful. It "explodes" out the variables in a data frame so that you can
|
|
refer to them explicitly. This is useful when working with many functions
|
|
in base R:
|
|
|
|
```{r}
|
|
mtcars %$%
|
|
cor(disp, mpg)
|
|
```
|
|
|
|
* For assignment magrittr provides the `%<>%` operator which allows you to
|
|
replace code like:
|
|
|
|
```R
|
|
mtcars <- mtcars %>% transform(cyl = cyl * 2)
|
|
```
|
|
|
|
with
|
|
|
|
```R
|
|
mtcars %<>% transform(cyl = cyl * 2)
|
|
```
|
|
|
|
I'm not a fan of this operator because I think assignment is such a
|
|
special operation that it should always be clear when it's occurring.
|
|
In my opinion, a little bit of duplication (i.e. repeating the
|
|
name of the object twice), is fine in return for making assignment
|
|
more explicit.
|
|
|
|
### When not to use the pipe
|
|
|
|
I also made a slight simplifiation when I said that the `x %>% f(y)` is exactly the same as `f(x, y)`. That's not quite true, which you'll see particularly for two classes of functions:
|
|
|
|
1. Functions that use the current environment. For example, `assign()`
|
|
will create a new variable with the given name in the current environment:
|
|
|
|
```{r}
|
|
assign("x", 10)
|
|
x
|
|
|
|
"x" %>% assign(100)
|
|
x
|
|
```
|
|
|
|
The use of assign with the pipe does not work because it assigns it to
|
|
a temporary environment used by `%>%`. If you do want to use assign with the
|
|
pipe, you can be explicit about the environment:
|
|
|
|
```{r}
|
|
env <- environment()
|
|
"x" %>% assign(100, envir = env)
|
|
x
|
|
```
|
|
|
|
Other functions with this problem are `get()`, and `load()`
|
|
|
|
1. Functions that use effect how their arguments are computed. In R, arguments
|
|
are lazy which means they are only computed when the function uses them,
|
|
not prior to calling the function. This means that the function can affect
|
|
the global environment in various ways. The pipe forces computation of
|
|
each element in series so you can't rely on this behaviour.
|
|
|
|
```{r, error = TRUE}
|
|
tryCatch(stop("!"), error = function(e) "An error")
|
|
|
|
stop("!") %>%
|
|
tryCatch(error = function(e) "An error")
|
|
```
|
|
|
|
There are a relatively wide class of functions with this behaviour including
|
|
`try()`, `supressMessages()`, `suppressWarnings()`, any function from the
|
|
withr package, ...
|
|
|
|
The pipe is a powerful tool, but it's not the only tool at your disposal, and it doesn't solve every problem! Pipes are most useful for rewriting a fairly short linear sequence of operations. I think you should reach for another tool when:
|
|
|
|
* Your pipes get longer than five or six lines. In that case, create
|
|
intermediate objects with meaningful names. That will make debugging easier,
|
|
because you can more easily check the intermediate results. It also helps
|
|
when reading the code, because the variable names can help describe the
|
|
intent of the code.
|
|
|
|
* You have multiple inputs or outputs. If there is not one primary object
|
|
being transformed, write code the regular ways.
|
|
|
|
* You are starting to think about a directed graph with a complex
|
|
dependency structure. Pipes are fundamentally linear and expressing
|
|
complex relationships with them typically does not yield clear code.
|
|
|
|
|
|
### Pipes in production
|
|
|
|
When you run a pipe interactively, it's easy to see if something goes wrong. When you start writing pipes that are used in production, i.e. they're run automatically and a human doesn't immediately look at the output it's a really good idea to include some assertions that verify the data looks like expected. One great way to do this is the ensurer package, written by Stefan Milton Bache (the author of magrittr).
|
|
|
|
<http://www.r-statistics.com/2014/11/the-ensurer-package-validation-inside-pipes/>
|