Merge branch 'master' of github.com:hadley/r4ds

This commit is contained in:
hadley 2016-04-06 09:02:17 -05:00
commit fb5a93d6db
5 changed files with 250 additions and 267 deletions

View File

@ -24,6 +24,7 @@ rmd_files: [
"model-vis.Rmd",
"model-assess.Rmd",
"communicate.Rmd",
"communicate-plots.Rmd",
"rmarkdown.Rmd",
"shiny.Rmd",
]

82
communicate-plots.Rmd Normal file
View File

@ -0,0 +1,82 @@
*Section 2* will show you how to prepare your plots for communication. You'll learn how to make your plots more legible with titles, labels, zooming, and default visual themes.
```{r echo = FALSE, messages = FALSE, warning=FALSE}
library(ggplot2)
```
# Communication with plots
The previous sections showed you how to make plots that you can use as a tools for _exploration_. When you made these plots, you knew---even before you looked at them---which variables the plot would display and which data sets the variables would come from. You might have even known what to look for in the completed plots, assuming that you made each plot with a goal in mind. As a result, it was not very important to put a title or a useful set of labels on your plots.
The importance of titles and labels changes once you use your plots for _communication_. Your audience will not share your background knowledge. In fact, they may not know anything about your plots except what the plots themselves display. If you want your plots to communicate your findings effectively, you will need to make them as self-explanatory as possible.
Luckily, `ggplot2` provides some features that can help you.
### Labels
You can add a title to any `ggplot2` plot by adding the command `labs()` to your plot call. Set the `title` argument of `labs()` to the character string that you would like to appear as the title of your plot. `ggplot2` will place the title at the top of your plot.
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
labs(title = "Fuel efficiency vs. Engine size")
```
You can also use `labs()` to replace the axis and legend labels in your plot, which might be a good idea if your data uses ambiguous or abbreviated variable names. To replace either of the axis labels, set the `x` or `y` arguments to a character string. `ggplot2` will replace the associated axis label with your character string.
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
labs(title = "Fuel efficiency vs. Engine size",
x = "Engine displacement (L)",
y = "Highway fuel efficiency (mpg)")
```
To replace a legend label, set the name of the aesthetic displayed in the legend to the character string that should appear as the title of the legend. For example, the legend in our plot corresponds to the color aesthetic. We can change it's title with the command, `labs(color = "New Title")`, or, more usefully:
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
labs(title = "Fuel efficiency vs. Engine size",
x = "Engine displacement (L)",
y = "Highway fuel efficiency (mpg)",
color = "Type of Car")
```
### Zooming
Often, it can be helpful to zoom in on a specific region of your plot. In `ggplot2` you can do this by adding `coord_cartesian()` to your plot and setting it's `xlim` and `ylim` arguments. Pass each argument a vector of two numbers, the minimum value to display on that axis and the maximum value, e.g.
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
coord_cartesian(xlim = c(5, 7), ylim = c(10, 30))
```
`coord_cartesian()` adds a cartesian coordinate system to your plot (which is the default coordinate system). However, the new coordinate system will use the zoomed in limits.
What if your plot uses a different coordinate system? Most of the other coordinate functions also take `xlim` and `ylim` arguments. You can look up the help pages of the coordinate functions to learn more.
### Themes
Finally, you can also quickly customize the "look" of your plot by adding a theme function to your plot call. This can be a useful thing to do, for example, if you'd like to save ink when you print your plots, or if you wish to ensure that the plots photocopy well.
`ggplot2` contains eight theme functions, listed in the table below. Each applies a different visual theme to your finished plot. You can think of the themes as "skins" for the plot. The themes change how the plot looks without changing the information that the plot displays.
To use any of the theme functions, add the function to your plot all. No arguments are necessary.
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
theme_bw()
```
```{r, echo = FALSE}
knitr::include_graphics("images/visualization-themes.png")
```

View File

@ -1,6 +1,4 @@
---
output: pdf_document
---
# Model
A model is a function that summarizes how the values of one variable vary in relation to the values of other variables. Models play a large role in hypothesis testing and prediction, but for the moment you should think of models just like you think of statistics. A statistic summarizes a *distribution* in a way that is easy to understand; and a model summarizes *covariation* in a way that is easy to understand. In other words, a model is just another way to describe data.
@ -11,7 +9,7 @@ This chapter will explain how to build useful models with R.
*Section 1* will show you how to build linear models, the most commonly used type of model. Along the way, you will learn R's model syntax, a general syntax that you can reuse with most of R's modeling functions.
*Section 2* will show you the best ways to use R's model output, which is often requires additional wrangling.
*Section 2* will show you the best ways to use R's model output, which often requires additional wrangling.
*Section 3* will teach you to build and interpret multivariate linear models, models that use more than one explanatory variable to explain the values of a response variable.
@ -37,7 +35,7 @@ library(broom)
Have you heard that a relationship exists between your height and your income? It sounds far-fetched---and maybe it is---but many people believe that taller people will be promoted faster and valued more for their work, an effect that increases their income. Could this be true?
Luckily, it is easy to measure someone's height, as well as their income (and a swath of other variables besides), which means that we can collect data relevant to the question. In fact, the Bureau of Labor Statistics has been doing this in a controlled way for over 50 years. The BLS [National Longitudinal Surveys (NLS)](https://www.nlsinfo.org/) track the income, education, and life circumstances of a large cohort of Americans across several decades. In case you are wondering, the point of the NLS is not to study the relationship between height and income, that's just a lucky accident.
Luckily, it is easy to measure someone's height, as well as their income, which means that we can collect data relevant to the question. In fact, the Bureau of Labor Statistics has been doing this in a controlled way for over 50 years. The BLS [National Longitudinal Surveys (NLS)](https://www.nlsinfo.org/) track the income, education, and life circumstances of a large cohort of Americans across several decades. In case you are wondering just how your tax dollars are being spent, the point of the NLS is not to study the relationship between height and income, that's just a lucky accident.
You can load the latest cross-section of NLS data, collected in 2013 with the code below.
@ -144,7 +142,7 @@ lm(income ~ 0 + height, data = heights)
## Using model output
R's model output is not very tidy. It is designed to provide a data store from which you can extract information with helper functions.
R's model output is not very tidy. It is designed to provide a data store from which you can extract information with helper functions. You will learn more about tidy data in Tidy Data.
```{r}
coef(h)
@ -213,12 +211,14 @@ What about sex? Many sources have observed that there is a difference in income
### Factors
R stores categorical data as factors or character strings. If you add a string to a model, R will convert it to a factor for the purposes of the model.
R stores categorical data as factors. If you add a string to a model, R will convert it to a factor for the purposes of the model.
A factor is an integer vector with a levels attribute. You can make a factor with `factor()`.
```{r}
fac <- factor(c("c", "a", "b"), levels = c("a", "b", "c"), labels = c("blond", "brunette", "red"))
fac <- factor(c("c", "a", "b"),
levels = c("a", "b", "c"),
labels = c("blond", "brunette", "red"))
fac
unclass(fac)
```
@ -227,19 +227,19 @@ Each level of the factor (i.e. unique value) is encoded as an integer and displa
If you use factors outside of a model, you will notice some limiting behavior:
* You cannot add to a factor values that do not appear in its levels attribute
* factors retain all of their levels attribute when you subset them. To avoid this use `drop = TRUE`.
```{r}
fac[1]
fac[1, drop = TRUE]
```
* If you coerce a factor to a numeric, R will convert the integer vector that underlies the factor, not the level labels that you see when you print the factor.
```{r}
num_fac <- factor(1:3, levels = 1:3, labels = c("100", "200", "300"))
num_fac
as.numeric(num_fac)
```
To coerce these labels to a different data type, first coerce the factor to a character string with `as.character()`
* You cannot add values to a factor that do not appear in its levels.
* Factors retain all of their levels when you subset them. To avoid this use `drop = TRUE`.
```{r}
fac[1]
fac[1, drop = TRUE]
```
* If you coerce a factor to a number with `as.numeric()`, R will convert the integer vector that underlies the factor to a number, not the level labels that you see when you print the factor.
```{r}
num_fac <- factor(1:3, levels = 1:3, labels = c("100", "200", "300"))
num_fac
as.numeric(num_fac)
```
To coerce the labels that you see to a new data type, first coerce the factor to a character string with `as.character()`
```{r}
as.numeric(as.character(num_fac))
```
@ -292,7 +292,7 @@ tidy(she)
## Non-linear models
But what if the relationship between variables is not linear. For example, the relationship between income and education does not seem to be linear.
But what if the relationship between variables is not linear? For example, the relationship between income and education does not seem to be linear.
```{r}
ggplot(data = heights, mapping = aes(x = education, y = income)) +

View File

@ -58,7 +58,7 @@ class(flights)
This is called a `tbl_df` (pronounced "tibble diff") or a `data_frame` (pronounced "data underscore frame"; cf. `data dot frame`). Generally, however, we won't worry about this relatively minor difference and will refer to everything as data frames.
You'll learn more about how that works in data structures. If you want to convert your own data frames to this special case, use `as.data_frame()`. I recommend it for large data frames as it makes interactive exploration much less painful.
You'll learn more about how `data_frame` works in data structures. If you want to convert your own data frames to this special case, use `as.data_frame()`. I recommend it for large data frames as it makes interactive exploration much less painful.
To create your own new tbl\_df from individual vectors, use `data_frame()`:
@ -114,7 +114,7 @@ There are five dplyr functions that you will use to do the vast majority of data
* create new variables with functions of existing variables (`mutate()`), or
* collapse many values down to a single summary (`summarise()`).
These can all be used in conjunction with `group_by()` which changes the scope of each function from operating on the entire dataset to operating on it group-by-group. These six functions the provide the verbs for a language of data manipulation.
These can all be used in conjunction with `group_by()` which changes the scope of each function from operating on the entire dataset to operating on it group-by-group. These six functions provide the verbs for a language of data manipulation.
All verbs work similarly:
@ -200,7 +200,7 @@ Multiple arguments to `filter()` are combined with "and". To get more complicate
filter(flights, month == 11 | month == 12)
```
Note the order isn't like English. The following expression doesn't find on months that equal 11 or 12. Instead it finds all months that equal `11 | 12`, which is `TRUE`. In a numeric context (like here), `TRUE` becomes one, so this finds all flights in January, not November or December.
Note the order isn't like English. The following expression doesn't find on months that equal 11 or 12. Instead it finds all months that equal `11 | 12`, an expression that evaluates to `TRUE`. In a numeric context (like here), `TRUE` becomes one, so this finds all flights in January, not November or December (It is the equivalent of `filter(flights, month == 1)`).
```{r, eval = FALSE}
filter(flights, month == 11 | 12)
@ -225,7 +225,7 @@ filter(flights, !(arr_delay > 120 | dep_delay > 120))
filter(flights, arr_delay <= 120, dep_delay <= 120)
```
Note that R has both `&` and `|` and `&&` and `||`. `&` and `|` are vectorised: you give them two vectors of logical values and they return a vector of logical values. `&&` and `||` are scalar operators: you give them individual `TRUE`s or `FALSE`s. They're used in `if` statements when programming. You'll learn about that later on.
Note that R has both `&` and `|` and `&&` and `||`. `&` and `|` are vectorised: you give them two vectors of logical values and they return a vector of logical values. `&&` and `||` are scalar operators: you give them individual `TRUE`s or `FALSE`s. They're used in `if` statements when programming. You'll learn about that later on in Chapter ?.
Sometimes you want to find all rows after the first `TRUE`, or all rows until the first `FALSE`. The cumulative functions `cumany()` and `cumall()` allow you to find these values:
@ -239,11 +239,11 @@ filter(df, cumany(x)) # all rows after first TRUE
filter(df, cumall(y)) # all rows until first FALSE
```
Whenever you start using multipart expressions in your `filter()`, it's typically a good idea to make them explicit variables with `mutate()` so that you can more easily check your work. You'll learn about `mutate()` in the next section.
Whenever you start using multipart expressions in your `filter()`, it's typically a good idea to make the expressions explicit variables with `mutate()` so that you can more easily check your work. You'll learn about `mutate()` in the next section.
### Missing values
One important feature of R that can make comparison tricky is the missing value, `NA`. `NA` represents an unknown value so missing values are "infectious": any operation involving an unknown value will also be unknown.
One important feature of R that can make comparison tricky is the missing value, `NA`. `NA` represents an unknown value so missing values are "contagious": any operation involving an unknown value will also be unknown.
```{r}
NA > 5
@ -305,7 +305,7 @@ filter(df, is.na(x) | x > 1)
arrange(flights, year, month, day)
```
Use `desc()` to order a column in descending order:
Use `desc()` to re-order by a column in descending order:
```{r}
arrange(flights, desc(arr_delay))
@ -358,7 +358,7 @@ There are a number of helper functions you can use within `select()`:
* `ends_with("xyz")`: matches names that end with "xyz".
* `contains("ijk")`: matches name that contain "ijk".
* `contains("ijk")`: matches names that contain "ijk".
* `matches("(.)\\1")`: selects variables that match a regular expression.
This one matches any variables that contain repeated characters. You'll
@ -382,7 +382,7 @@ rename(flights, tail_num = tailnum)
--------------------------------------------------------------------------------
The `select()` function works similarly to the `select` argument in `base::subset()`. Because the dplyr philosophy is to have small functions that do one thing well, it is its own function in dplyr.
The `select()` function works similarly to the `select` argument in `base::subset()`. `select()` is its own function in dplyr because the dplyr philosophy is to have small functions that each do one thing well.
--------------------------------------------------------------------------------
@ -395,7 +395,7 @@ The `select()` function works similarly to the `select` argument in `base::subse
Besides selecting sets of existing columns, it's often useful to add new columns that are functions of existing columns. This is the job of `mutate()`.
`mutate()` always adds new columns at the end so we'll start by creating a narrower dataset so we can see the new variables. Remember that when you're in RStudio, the easiest way to see all the columns is `View()`
`mutate()` always adds new columns at the end of your dataset so we'll start by creating a narrower dataset so we can see the new variables. Remember that when you're in RStudio, the easiest way to see all the columns is `View()`
```{r}
flights_sml <- select(flights,
@ -410,7 +410,7 @@ mutate(flights_sml,
)
```
Note that you can refer to columns that you've just created:
Note that you can refer to columns in `mutate()` that you've just created:
```{r}
mutate(flights_sml,
@ -432,13 +432,13 @@ transmute(flights,
--------------------------------------------------------------------------------
`mutate()` is similar to `transform()` in base R, but in `mutate()` you can refer to variables you've just created; in `transform()` you can not.
`mutate()` is similar to `transform()` in base R, but in `mutate()` you can refer to variables you've just created; in `transform()` you cannot.
--------------------------------------------------------------------------------
### Useful functions
There are many functions for creating new variables. The key property is that the function must be vectorised: it needs to return the same number of outputs as inputs. There's no way to list every possible function that you might use, but here's a selection of functions that are frequently useful:
There are many functions for creating new variables that you can use with `mutate()`. The key property is that the function must be vectorised: it needs to return the same number of outputs as inputs. There's no way to list every possible function that you might use, but here's a selection of functions that are frequently useful:
* Arithmetic operators: `+`, `-`, `*`, `/`, `^`. These are all vectorised, so
you can work with multiple columns. These operations use "recycling rules"
@ -477,12 +477,25 @@ There are many functions for creating new variables. The key property is that th
values. This allows you to compute running differences (e.g. `x - lag(x)`)
or find when values change (`x != lag(x))`. They are most useful in
conjunction with `group_by()`, which you'll learn about shortly.
```{r}
x <- 1:10
x
lag(x)
lead(x)
```
* Cumulative and rolling aggregates: R provides functions for running sums,
products, mins and maxes: `cumsum()`, `cumprod()`, `cummin()`, `cummax()`.
dplyr provides `cummean()` for cumulative means. If you need rolling
aggregates (i.e. a sum computed over a rolling window), try the RcppRoll
package.
```{r}
x
cumsum(x)
cummean(x)
```
* Logical comparisons, `<`, `<=`, `>`, `>=`, `!=`, which you learned about
earlier. If you're doing a complex sequence of logical operations it's
@ -495,13 +508,13 @@ There are many functions for creating new variables. The key property is that th
ranks; use `desc(x)` to give the largest values the smallest ranks.
```{r}
x <- c(1, 2, 2, NA, 3, 4)
y <- c(1, 2, 2, NA, 3, 4)
data_frame(
row_number(x),
min_rank(x),
dense_rank(x),
percent_rank(x),
cume_dist(x)
row_number(y),
min_rank(y),
dense_rank(y),
percent_rank(y),
cume_dist(y)
) %>% knitr::kable()
```
@ -548,18 +561,18 @@ The last verb is `summarise()`. It collapses a data frame to a single row:
summarise(flights, delay = mean(dep_delay, na.rm = TRUE))
```
That's not terribly useful unless we pair it with `group_by()`. This changes the unit of analysis from the complete dataset to individual groups. When you the dplyr verbs on a grouped data frame they'll be automatically applied "by group". For example, if we applied exactly the same code to a data frame grouped by day, we get the average delay per day:
That's not terribly useful unless we pair it with `group_by()`. This changes the unit of analysis from the complete dataset to individual groups. When you use the dplyr verbs on a grouped data frame they'll be automatically applied "by group". For example, if we applied exactly the same code to a data frame grouped by date, we get the average delay per date:
```{r}
by_day <- group_by(flights, year, month, day)
summarise(by_day, delay = mean(dep_delay, na.rm = TRUE))
```
Together `group_by()` and `summarise()` provide one of tools that you'll use most commonly when working with dplyr: grouped summaries. But before we go any further with this, we need to introduce a powerful new idea: the pipe.
Together `group_by()` and `summarise()` provide one of the tools that you'll use most commonly when working with dplyr: grouped summaries. But before we go any further with this, we need to introduce a powerful new idea: the pipe.
### Combining multiple operations with the pipe
Imagine we want to explore the relationship between the distance and average delay for each location. Using what you already know about dplyr, you might write code like this:
Imagine that we want to explore the relationship between the distance and average delay for each location. Using what you already know about dplyr, you might write code like this:
```{r, fig.width = 6}
by_dest <- group_by(flights, dest)
@ -577,13 +590,13 @@ ggplot(delay, aes(dist, delay)) +
geom_smooth(se = FALSE)
```
There are three steps:
There are three steps to prepare this data:
* Group flights by destination
1. Group flights by destination
* Summarise to compute distance, average delay, and number of flights.
2. Summarise to compute distance, average delay, and number of flights.
* Filter to remove noisy points and Honolulu airport which is almost
3. Filter to remove noisy points and Honolulu airport, which is almost
twice as far away as the next closest airport.
This code is a little frustrating to write because we have to give each intermediate data frame a name, even though we don't care about it. Naming things well is hard, so this slows us down.
@ -603,9 +616,9 @@ delays <- flights %>%
This focuses on the transformations, not what's being transformed, which makes the code easier to read. You can read it as a series of imperative statements: group, then summarise, then filter. As suggested by this reading, a good way to pronounce `%>%` when reading code is "then".
Behind the scenes, `x %>% f(y)` turns into `f(x, y)` so you can use it to rewrite multiple operations that you can read left-to-right, top-to-bottom. We'll use piping frequently from now on because it considerably improves the readability of code, and we'll come back to it in more detail in Chapter XYZ.
Behind the scenes, `x %>% f(y)` turns into `f(x, y)`, and `x %>% f(y) %>% g(z)` turns into `g(f(x, y), z)` and so on. You can use the pipe to rewrite multiple operations in a way that you can read left-to-right, top-to-bottom. We'll use piping frequently from now on because it considerably improves the readability of code, and we'll come back to it in more detail in Chapter XYZ.
Most of the packages you'll learn through this book have been designed to work with the pipe (tidyr, dplyr, stringr, purrr, ...). The only exception is ggplot2: it was developed considerably before the discovery of the pipe. Unfortunately the next iteration of ggplot2, ggvis, which does use the pipe, isn't ready for prime time yet.
Most of the packages you'll learn through this book have been designed to work with the pipe (tidyr, dplyr, stringr, purrr, ...). The only exception is ggplot2: it was developed considerably before the pipe was discovered. Unfortunately the next iteration of ggplot2, ggvis, which does use the pipe, isn't ready for prime time yet.
### Missing values
@ -617,7 +630,7 @@ flights %>%
summarise(mean = mean(dep_delay))
```
We get a lot of missing values! That's because aggregation functions obey the usual rule of missing values: if there's any missing value in the input, the output will be a missing value. Fortunately, all aggregation functions have an `na.rm` argument which removes the missing values prior to computation:
We get a lot of missing values! That's because aggregation functions obey the usual rule of missing values: if there's any missing value in the input, the output will be a missing value. `x %>% f(y)` turns into `f(x, y)`ou'll learn more about aggregation functions in Section 5.7.4. Fortunately, all aggregation functions have an `na.rm` argument which removes the missing values prior to computation:
```{r}
flights %>%
@ -637,7 +650,7 @@ not_cancelled %>%
### Counts
Whenever you do any aggregation, it's always a good idea to include either a count (`n()`), or a count of non-missing values (`sum(!is.na(x))`). That way you can check that you're not drawing conclusions based on very small amounts of data amount of non-missing data.
Whenever you do any aggregation, it's always a good idea to include either a count (`n()`), or a count of non-missing values (`sum(!is.na(x))`). That way you can check that you're not drawing conclusions based on very small amounts of non-missing data.
For example, let's look at the planes (identified by their tail number) that have the highest average delays:
@ -670,7 +683,7 @@ ggplot(delays, aes(n, delay)) +
Not suprisingly, there is much more variation in the average delay when there are few flights. The shape of this plot is very characteristic: whenever you plot a mean (or many other summaries) vs number of observations, you'll see that the variation decreases as the sample size increases.
When looking at this sort of plot, it's often useful to filter out the groups with the smallest numbers of observations, so you can see more of the pattern and less of the extreme variation in the smallest groups. This what the following code does, and also shows you a handy pattern for integrating ggplot2 into dplyr flows. It's a bit painful that you have to switch from `%>%` to `+`, but once you get the hang of it, it's quite convenient.
When looking at this sort of plot, it's often useful to filter out the groups with the smallest numbers of observations, so you can see more of the pattern and less of the extreme variation in the smallest groups. This is what the following code does, and also shows you a handy pattern for integrating ggplot2 into dplyr flows. It's a bit painful that you have to switch from `%>%` to `+`, but once you get the hang of it, it's quite convenient.
```{r}
delays %>%
@ -723,7 +736,7 @@ You can find a good explanation of this problem at <http://varianceexplained.org
Just using means, counts, and sum can get you a long way, but R provides many other useful summary functions:
* Measure of location: we've used `mean(x)`, but `median(x)` is also
useful.The mean is the sum divided by the length; the median is a value
useful. The mean is the sum divided by the length; the median is a value
where 50% of `x` is above, and 50% is below.
It's sometimes useful to combine aggregation with logical subsetting:
@ -733,7 +746,7 @@ Just using means, counts, and sum can get you a long way, but R provides many ot
group_by(year, month, day) %>%
summarise(
avg_delay1 = mean(arr_delay),
avg_delay2 = mean(arr_delay[arr_delay > 0])
avg_delay2 = mean(arr_delay[arr_delay > 0]) # the average positive delay
)
```
@ -743,14 +756,14 @@ Just using means, counts, and sum can get you a long way, but R provides many ot
are robust equivalents that maybe more useful if you have outliers.
```{r}
# Why is distance to some destinations more variable than others?
# Why is distance to some destinations more variable than to others?
not_cancelled %>%
group_by(dest) %>%
summarise(distance_sd = sd(distance)) %>%
arrange(desc(distance_sd))
```
* By rank: `min(x)`, `quantile(x, 0.25)`, `max(x)`.
* Measures of rank: `min(x)`, `quantile(x, 0.25)`, `max(x)`.
```{r}
# When do the first and last flights leave each day?
@ -762,8 +775,8 @@ Just using means, counts, and sum can get you a long way, but R provides many ot
)
```
* By position: `first(x)`, `nth(x, 2)`, `last(x)`. These work similarly to
`x[1]`, x[n], and `x[length(x)]` but let you set a default value if that
* Measures of position: `first(x)`, `nth(x, 2)`, `last(x)`. These work similarly to
`x[1]`, `n <- 2; x[n]`, and `x[length(x)]` but let you set a default value if that
position does not exist (i.e. you're trying to get the 3rd element from a
group that only has two elements).
@ -810,7 +823,7 @@ Just using means, counts, and sum can get you a long way, but R provides many ot
count(tailnum, wt = distance)
```
* Counts and proportions of logical values: `sum(x > 10)`, `mean(y == 0)`
* Counts and proportions of logical values: `sum(x > 10)`, `mean(y == 0)`.
When used with numeric functions, `TRUE` is converted to 1 and `FALSE` to 0.
This makes `sum()` and `mean()` particularly useful: `sum(x)` gives the
number of `TRUE`s in `x`, and `mean(x)` gives the proportion.
@ -845,6 +858,12 @@ Be careful when progressively rolling up summaries: it's OK for sums and counts,
If you need to remove grouping, and return to operations on ungrouped data, use `ungroup()`.
```{r}
daily %>%
ungroup() %>% # no longer grouped by date
summarise(flights = n()) # all flights
```
### Exercises
1. Brainstorm at least 5 different ways to assess the typical delay

View File

@ -2,7 +2,7 @@
> "The simple graph has brought more information to the data analysts mind than any other device."---John Tukey
If you are like most humans, your brain is not designed to work with raw data. Your brain can only attend to a few values at a time, which makes it difficult to discover patterns in raw data, like in the table below. Can you spot the striking relationship between $X$ and $Y$?
If you are like most humans, your brain is not designed to work with raw data. The working memory can only attend to a few values at a time, which makes it difficult to discover patterns in raw data. For example, can you spot the striking relationship between $X$ and $Y$ in the table below?
```{r data, echo=FALSE}
x <- rep(seq(0.2, 1.8, length = 5), 2) + runif(10, -0.15, 0.15)
@ -14,9 +14,7 @@ order <- sample(1:10)
knitr::kable(round(data.frame(X = X[order], Y = Y[order]), 2))
```
While you may stumble over raw data, you can easily process visual information. Within your mind is a visual processing system that has been fine-tuned by thousands of years of evolution.
As a result, the quickest way to understand your data is to visualize it. Once you plot your data, you can instantly see the relationships between values. Here, we see that the values above fall on a circle.
While we may stumble over raw data, we can easily process visual information. Within your mind is a visual processing system that has been fine-tuned by thousands of years of evolution. As a result, the quickest way to understand your data is to visualize it. Once you plot your data, you can instantly see the relationships between values. Here, we see that the values above fall on a circle.
```{r echo=FALSE, dependson=data}
ggplot2::qplot(X, Y) + ggplot2::coord_fixed(ylim = c(-2.5, 2.5), xlim = c(-2.5, 2.5))
@ -24,12 +22,6 @@ ggplot2::qplot(X, Y) + ggplot2::coord_fixed(ylim = c(-2.5, 2.5), xlim = c(-2.5,
This chapter will teach you how to visualize your data with R and the `ggplot2` package. R contains several systems for making graphs, but the `ggplot2` system is one of the most beautiful and most versatile. `ggplot2` implements the *grammar of graphics*, a coherent system for describing and building graphs. With `ggplot2`, you can do more faster by learning one system and applying it in many places.
### Outline
*Section 1* will get you started making graphs right away. You'll learn how to use the grammar of graphics to make any type of plot.
*Section 2* will show you how to prepare your plots for communication. You'll learn how to make your plots more legible with titles, labels, zooming, and default visual themes.
### Prerequisites
To access the data sets and functions that we will use in this chapter, load the `ggplot2` package:
@ -43,13 +35,7 @@ install.packages("ggplot2")
library(ggplot2)
```
## The Layered Grammar of Graphics
The grammar of graphics is a language for describing graphs. Once you learn the language, you can use it to build graphs with `ggplot2`; but how should you learn the language?
Have you ever tried to learn a language by only studying its rules, vocabulary, and syntax? That's how I tried to learn Spanish in college, and now I speak _un muy, muy, poquito_.
It is far better to learn a language by actually speaking it! And that's what we'll do here; we'll learn the grammar of graphics by making a series of plots. Don't worry if things seem confusing at first, by the end of the section everything will come together in a clear way.
## A code template
Let's use our first graph to answer a question: Do cars with big engines use more fuel than cars with small engines? You probably already have an answer, but try to make your answer precise. What does the relationship between engine size and fuel efficieny look like? Is it positive? Negative? Linear? Nonlinear? Strong? Weak?
@ -70,24 +56,14 @@ library(ggplot2)
***
### Scatterplots
The easiest way to understand the `mpg` data set is to visualize it, which means that it is time to make our first plot. To do this, open an R session and run the code below. The code plots the `displ` variable of `mpg` against the `hwy` variable to make the plot below. Does the plot confirm your hypothesis about fuel efficiency and engine size?
To plot `mpg`, open an R session and run the code below. The code plots the `displ` variable of `mpg` against the `hwy` variable.
```{r}
ggplot(data = mpg) +
geom_point(mapping = aes(x = displ, y = hwy))
```
The plot shows a negative relationship between engine size (`displ`) and fuel efficiency (`hwy`). In other words, cars with big engines use more fuel. But the plot shows us something else as well.
One group of points seems to fall outside of the linear trend. These cars have a higher mileage than you might expect. Can you tell why? Before we examine these cars, let's review the code that made our plot.
```{r, echo = FALSE}
knitr::include_graphics("images/visualization-1.png")
```
#### Template
The plot shows a negative relationship between engine size (`displ`) and fuel efficiency (`hwy`). In other words, cars with big engines use more fuel. Does this confirm your hypothesis about fuel efficiency and engine size?
Our code is almost a template for making plots with `ggplot2`.
@ -96,56 +72,60 @@ ggplot(data = mpg) +
geom_point(mapping = aes(x = displ, y = hwy))
```
With `ggplot2`, you begin a plot with the function `ggplot()`. `ggplot()` doesn't create a plot by itself; instead it initializes a new plot that you can add layers to.
With `ggplot2`, you begin a plot with the function `ggplot()`. `ggplot()` creates a coordinate system that you can add layers to. The first argument of `ggplot()` is the data set to use in the graph. So `ggplot(data = mpg)` creates an empty graph that will use the `mpg` data set.
The first argument of `ggplot()` is the data set to use in the graph. So `ggplot(data = mpg)` initializes a graph that will use the `mpg` data set.
You complete your graph by adding one or more layers to `ggplot()`. Here, the function `geom_point()` adds a layer of points to your plot, which creates a scatterplot. `ggplot2` comes with many geom functions that each add a different type of layer to a plot. Each geom function in `ggplot2` takes a mapping argument.
You complete your graph by adding one or more layers to `ggplot()`. Here, the function `geom_point()` adds a layer of points to the plot, which creates a scatterplot. `ggplot2` comes with other geom functions that you can use as well. Each function creates a different type of layer, and each function takes a mapping argument.
The mapping argument of your geom function explains where your points should go. You must set `mapping` to a call to `aes()`. The `x` and `y` arguments of `aes()` explain which variables to map to the x and y axes of your plot. `ggplot()` will look for those variables in your data set, `mpg`.
The mapping argument of your geom function explains where your points should go. You must set `mapping` to a call to `aes()`. The `x` and `y` arguments of `aes()` explain which variables to map to the x and y axes of the graph. `ggplot()` will look for those variables in your data set, `mpg`.
This code suggests a minimal template for making graphs with `ggplot2`. To make a graph, replace the bracketed sections in the code below with a data set, a geom function, or a set of mappings.
This code suggests a reusable template for making graphs with `ggplot2`. To make a graph, replace the bracketed sections in the code below with a data set, a geom function, or a set of mappings.
```{r eval = FALSE}
ggplot(data = <DATA>) +
<GEOM_FUNCTION>(mapping = aes(<MAPPINGS>))
```
#### Aesthetic Mappings
The sections below will show you how to complete and extend this template to make a graph. We will begin with `<MAPPINGS>`.
## Aesthetic mappings
> "The greatest value of a picture is when it forces us to notice what we never expected to see."---John Tukey
Our plot above revealed a group of cars that had better than expected mileage. How can you explain these cars?
In the plot above, one group of points seems to fall outside of the linear trend. These cars have a higher mileage than you might expect. How can you explain these cars?
```{r, echo = FALSE}
knitr::include_graphics("images/visualization-1.png")
```
Let's hypothesize that the cars are hybrids. One way to test this hypothesis is to look at the `class` value for each car. The `class` variable of the `mpg` data set classifies cars into groups such as compact, midsize, and suv. If the outlying points are hybrids, they should be classified as compact cars or, perhaps, subcompact cars (keep in mind that this data was collected before hybrid trucks and suvs became popular).
You can add a third value, like `class`, to a two dimensional scatterplot by mapping it to an _aesthetic_.
You can add a third variable, like `class`, to a two dimensional scatterplot by mapping it to an _aesthetic_.
An aesthetic is a visual property of the points in your plot. Aesthetics include things like the size, the shape, or the color of your points. You can display a point (like the one below) in different ways by changing the values of its aesthetic properties. Since we already use the word "value" to describe data, let's use the word "level" to describe aesthetic properties. Here we change the levels of a point's size, shape, and color to make the point small, trianglular, or blue.
An aesthetic is a visual property of the objects in your plot. Aesthetics include things like the size, the shape, or the color of your points. You can display a point (like the one below) in different ways by changing the values of its aesthetic properties. Since we already use the word "value" to describe data, let's use the word "level" to describe aesthetic properties. Here we change the levels of a point's size, shape, and color to make the point small, trianglular, or blue.
```{r, echo = FALSE}
knitr::include_graphics("images/visualization-2.png")
```
You can convey information about your data by mapping the aesthetics in your plot to the variables in your data set. For example, we can map the colors of our points to the `class` variable. Then the color of each point will reveal its class affiliation.
You can convey information about your data by mapping the aesthetics in your plot to the variables in your data set. For example, you can map the colors of your points to the `class` variable to reveal the class of each car.
```{r}
ggplot(data = mpg) +
geom_point(mapping = aes(x = displ, y = hwy, color = class))
```
To map an aesthetic to a variable, set the name of the aesthetic to the name of the variable, _and do this in your plot's `aes()` call_. `ggplot2` will automatically assign a unique level of the aesthetic (here a unique color) to each unique value of the variable. `ggplot2` will also add a legend that explains which levels correspond to which values.
To map an aesthetic to a variable, set the name of the aesthetic to the name of the variable, _and do this in your plot's `aes()` call_. `ggplot2` will automatically assign a unique level of the aesthetic (here a unique color) to each unique value of the variable, a process known as _mapping_. `ggplot2` will also add a legend that explains which levels correspond to which values.
The colors reveal that many of the unusual points are two seater cars. These cars don't seem like hybrids. In fact, they seem like sports cars---and that's what they are. Sports cars have large engines like suvs and pickup trucks, but small bodies like midsize and compact cars, which improves their gas mileage. In hindsight, these cars were unlikely to be hybrids since they have large engines.
In the above example, we mapped `class` to the color aesthetic, but we could have mapped `class` to the size aesthetic in the same way. In this case, the exact size of each point reveals its class affiliation.
In the above example, we mapped `class` to the color aesthetic, but we could have mapped `class` to the size aesthetic in the same way. In this case, the exact size of each point would reveal its class affiliation.
```{r}
ggplot(data = mpg) +
geom_point(mapping = aes(x = displ, y = hwy, size = class))
```
Or we could have mapped `class` to the _alpha_ aesthetic, which controls the transparency of the points. Now the transparency of each point corresponds with its class affiliation.
Or we could have mapped `class` to the _alpha_ aesthetic, which controls the transparency of the points. Now the transparency of each point corresponds to its class affiliation.
```{r}
ggplot(data = mpg) +
@ -165,9 +145,9 @@ ggplot(data = mpg) +
***
In each case, you set the name of the aesthetic to the variable to display, and you do this within the `aes()` function. The syntax highlights a useful insight because you also set `x` and `y` to variables within `aes()`. The insight is that the x and y locations of a point are themselves aesthetics, visual properties that you can map to variables to display information about the data.
In each case, you set the name of the aesthetic to the variable to display, and you do this within the `aes()` function. The `aes()` function gathers together each of the aesthetic mappings used by a layer and passes them to the layer's mapping argument. The syntax highlights a useful insight because you also set `x` and `y` to variables within `aes()`. The insight is that the x and y locations of a point are themselves aesthetics, visual properties that you can map to variables to display information about the data.
Once you set an aesthetic, `ggplot2` takes care of the rest. It selects a pleasing set of levels to use for the aesthetic, and it constructs a legend that explains the mapping. For x and y aesthetics, `ggplot2` does not create a legend, but it creates an axis line with tick marks and a label. The axis line acts as a legend; it explains the mapping between locations and values.
Once you set an aesthetic, `ggplot2` takes care of the rest. It selects a pleasing set of levels to use for the aesthetic, and it constructs a legend that explains the mapping between levels and values. For x and y aesthetics, `ggplot2` does not create a legend, but it creates an axis line with tick marks and a label. The axis line acts as a legend; it explains the mapping between locations and values.
You can also set the aesthetic properties of your geom manually. For example, we can make all of the points in our plot blue.
@ -176,7 +156,7 @@ ggplot(data = mpg) +
geom_point(mapping = aes(x = displ, y = hwy), color = "blue")
```
Here, the color doesn't convey information about a variable. It just changes the appearance of the plot. To set an aesthetic manually, call the aesthetic as an argument of your geom function. Then pass the aesthetic a value that R will recognize, such as
Here, the color doesn't convey information about a variable. It only changes the appearance of the plot. To set an aesthetic manually, do not plce it in the `aes()` function. Call the aesthetic by name as an argument of your geom function. Then pass the aesthetic a value that R will recognize, such as
* the name of a color as a character string
* the size of a point as a cex expansion factor (see `?par`)
@ -213,13 +193,13 @@ pchShow <-
pchShow()
```
If you get an odd result, double check that you are calling the aesthetic as its own argument (and not calling it from inside of `mapping = aes()`. I like to think of aesthetics like this, if you set the aesthetic:
If you get an odd result, double check that you are calling the aesthetic as its own argument (and not calling it from inside of `mapping = aes()`. We like to think of aesthetics like this, if you set the aesthetic:
* _inside_ of the `aes()` function, `ggplot2` will map the aesthetic to data values and build a legend.
* _outside_ of the `aes()` function, `ggplot2` will directly set the aesthetic to your input.
#### Exercises
### Exercises
Now that you know how to use aesthetics, take a moment to experiment with the `mpg` data set.
@ -231,13 +211,13 @@ Now that you know how to use aesthetics, take a moment to experiment with the `m
***
**Tip** - See the help page for `geom_point()` (`?geom_point`) to learn which aesthetics are available to use in a scatterplot. See the help page for the `mpg` data set (`?mpg`) to learn which variables are in the data set.
**Tip** - See the help page for `geom_point()` (by running `?geom_point`) to learn which aesthetics are available to use in a scatterplot. See the help page for the `mpg` data set (`?mpg`) to learn which variables are in the data set.
***
#### Geoms
## Geoms
You can further enhance your scatterplots by adding summary information to them. But before you can do that with `ggplot2`, you'll need to solve this riddle: how are these two plots similar?
How are these two plots similar?
```{r echo = FALSE, message = FALSE, fig.show='hold', fig.width=3, fig.height=3}
ggplot(data = mpg) +
@ -247,29 +227,25 @@ ggplot(data = mpg) +
geom_smooth(mapping = aes(x = displ, y = hwy))
```
Both plots contain the same x variable, the same y variable, and if you look closely, you can see that they both describe the same data. But the plots are not identical.
Both plots contain the same x variable, the same y variable, and both describe the same data. But the plots are not identical. Each plot uses a different visual object to represent the data. In `ggplot2` syntax, we say that they use different _geoms_.
Each plot uses a different visual object to represent the data. You could say that these two graphs are different "types" of plots, or that they "draw" different things. In `ggplot2` syntax, we say that they use different _geoms_.
A _geom_ is the geometrical object that a plot uses to represent data. People often describe plots by the type of geom that the plot uses. For example, bar charts use bar geoms, line charts use line geoms, boxplots use boxplot geoms, and so on. Scatterplots break the trend; they use the point geom. As we see above, you can use different geoms to plot the same data. The plot on the left uses the point geom, and the plot on the right uses the smooth geom, a smooth line fitted to the data.
A _geom_ is the geometrical object that a plot uses to represent data. People often describe plots by the type of geom that the plot uses. For example, bar charts use bar geoms, line charts use line geoms, boxplots use boxplot geoms, and so on. Scatterplots use the point geom.
As we see above, you can use different geoms to plot the same data. The plot on the left uses the point geom, and the plot on the right uses the smooth geom, a smooth line fitted to the data.
To change the geom in your plot, change the geom function that you add to `ggplot()`. For instance, you can make the plot on the left with `geom_point()`:
To change the geom in your plot, change the geom function that you add to `ggplot()`. For instance, to make the plot on the left, use `geom_point()`:
```{r eval=FALSE}
ggplot(data = mpg) +
geom_point(mapping = aes(x = displ, y = hwy))
```
And you can make the plot on the right with `geom_smooth()`:
To make the plot on the right use `geom_smooth()`:
```{r eval=FALSE, message = FALSE}
ggplot(data = mpg) +
geom_smooth(mapping = aes(x = displ, y = hwy))
```
Every geom function in `ggplot2` takes a `mapping` argument. However, you'll want to think about which aesthetics you use with which geom. You could set the shape of a point, but you couldn't set the "shape" of a line. On the other hand, you _could_ set the linetype of a line. When you do that `geom_smooth()` will draw a different line, with a different linetype, for each group of points defined by whichever variable you map to linetype.
Every geom function in `ggplot2` takes a `mapping` argument. However, not every aesthetic works with every geom. You could set the shape of a point, but you couldn't set the "shape" of a line. On the other hand, you _could_ set the linetype of a line. `geom_smooth()` will draw a different line, with a different linetype, for each unique value of the variable that you map to linetype.
```{r message = FALSE}
ggplot(data = mpg) +
@ -278,7 +254,7 @@ ggplot(data = mpg) +
Here `geom_smooth()` separates the cars into three lines based on their `drv` value, which describes a car's drive train. One line describes all of the points with a `4` value, one line describes all of the points with an `f` value, and one line describes all of the points with an `r` value. Here, `4` stands for four wheel drive, `f` for front wheel drive, and `r` for rear wheel drive.
If this sounds strange to you, we can make it more clear by overlaying the lines on top of the raw data and then coloring everything according to `drv`.
If this sounds strange, we can make it more clear by overlaying the lines on top of the raw data and then coloring everything according to `drv`.
```{r message = FALSE, echo = FALSE}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy, color = drv)) +
@ -286,19 +262,19 @@ ggplot(data = mpg, mapping = aes(x = displ, y = hwy, color = drv)) +
geom_smooth(mapping = aes( linetype = drv))
```
What?! Two geoms in the same graph! If this makes you excited, buckle up. It's time to learn how to place multiple geoms in the same plot.
Notice that this plot contains two geoms in the same graph! If this makes you excited, buckle up. In the next section, we will learn how to place multiple geoms in the same plot.
`ggplot2` provides 37 geom functions that you can use to visualize your data. Each geom is particularly well suited for visualizing a certain type of data or a certain type of relationship. The table below lists the geoms in `ggplot2`, loosely organized by the type of relationship that they describe.
Next to each geom is a visual representation of the geom. Beneath the geom is a list of aesthetics that apply to the geom. Required aesthetics are listed in bold. Many geoms have very useful arguments that help them do their job. For these geoms, we've listed those arguments in the example code.
To learn more about any single geom, open it's help page in R by running the command `?` followed by the name of the geom function, e.g. `?geom_smooth()`.
To learn more about any single geom, open it's help page in R by running the command `?` followed by the name of the geom function, e.g. `?geom_smooth`.
***
**Tip** - Many geoms use a single object to describe all of the data, e.g. `geom_smooth()`. For these geoms, you can ask `ggplot2` to draw a separate object for each group of observations by setting the `group` aesthetic to a discrete variable.
**Tip** - Many geoms use a single object to describe all of the data. For example, `geom_smooth()` uses a single line. For these geoms, you can set the group aesthetic to a discrete variable to draw multiple objects. `ggplot2` will draw a separate object for each unique value of the grouping variable.
In practice, `ggplot2` will automatically group the data for these geoms whenever you map an aesthetic to a discrete variable. (as in the `linetype` example). It is convenient to rely on this feature because the group aesthetic by itself does not add a legend or distinguishing features to the geoms.
In practice, `ggplot2` will automatically group the data for these geoms whenever you map an aesthetic to a discrete variable (as in the `linetype` example). It is convenient to rely on this feature because the group aesthetic by itself does not add a legend or distinguishing features to the geoms.
```{r, echo = FALSE}
knitr::include_graphics("images/visualization-geoms-1.png")
@ -307,9 +283,9 @@ knitr::include_graphics("images/visualization-geoms-3.png")
knitr::include_graphics("images/visualization-geoms-4.png")
```
#### Layers
## Layers
Smooth lines are especially useful when you plot them _on top_ of raw data. The raw data provides a context for the smooth line, and the smooth line provides a summary of the raw data. To plot a smooth line on top of a scatterplot, add a call to `geom_smooth()` _after_ a call to `geom_point()`.
To display multiple geoms in the same plot, add multiple geom functions to `ggplot()`. `ggplot2` will add each new geom as a new layer on top of the previous geoms.
```{r, message = FALSE}
ggplot(data = mpg) +
@ -317,11 +293,7 @@ ggplot(data = mpg) +
geom_smooth(mapping= aes(x = displ, y = hwy))
```
Why does this work? You can think of each geom function in `ggplot2` as a layer. When you add multiple geoms to your plot call, `ggplot2` will add multiple layers to your plot. This let's you build sophisticated, multi-layer plots; `ggplot2` will place each new geom on top of the preceeding geoms.
Pay attention to your coding habits whenever you use multiple geoms. Our call now contains some redundant code. We call `mapping = aes(x = displ, y = hwy)` twice. As a general rule, it is unwise to repeat code because each repetition creates a chance to make a typo or error. Repetitions also make your code harder to read and write.
You can avoid repetition by passing a set of mappings to `ggplot()`. `ggplot2` will treat these mappings as global mappings that apply to each geom in the graph. You can then remove the mapping arguments in the individual layers.
To avoid redundancy, pay attention to your code when you use multiple geoms. Our code now calls `mapping = aes(x = displ, y = hwy)` twice. You can avoid this type of repetition by passing a set of mappings to `ggplot()`. `ggplot2` will treat these mappings as global mappings that apply to each geom in the graph. You can then remove the mapping arguments in the individual layers.
```{r, message = FALSE}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
@ -337,7 +309,7 @@ ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_smooth()
```
You can use the same system to specify individual data sets for each layer. For example, we can apply our smooth line to just a subset of the `mpg` data set, the cars with eight cylinder engines.
You can use the same system to specify individual data sets for each layer. Here, our smooth line displays just a subset of the `mpg` data set, the cars with eight cylinder engines. The local data argument in `geom_smooth()` overrides the global data argument in `ggplot()` for the smooth layer only.
```{r, message = FALSE, warning = FALSE}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
@ -345,7 +317,7 @@ ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_smooth(data = subset(mpg, cyl == 8))
```
##### Exercises
### Exercises
1. What would this graph look like?
@ -368,9 +340,7 @@ ggplot(mapping = aes(x = displ, y = hwy)) +
```
### Bar Charts
You now know how to make useful scatterplots with `ggplot2`, but there are many different types of plots that you can use to visualize your data. After scatterplots, one of the most used types of plot is the bar chart.
## Position adjustments
To make a bar chart with `ggplot2` use the function `geom_bar()`. `geom_bar()` does not require a $y$ aesthetic.
@ -395,24 +365,20 @@ ggplot(data = diamonds) +
geom_bar(mapping = aes(x = cut, color = cut))
```
The effect is interesting, sort of psychedelic, but not what we had in mind.
To control the interior fill of a bar, you must call the _fill_ aesthetic.
The effect is interesting, sort of psychedelic, but not what we had in mind. To control the interior fill of a bar, you must call the _fill_ aesthetic. The same pattern applies for all geoms that contain "substance." _Color_ controls the outline of the geom and _fill_ controls the interior fill.
```{r}
ggplot(data = diamonds) +
geom_bar(mapping = aes(x = cut, fill = cut))
```
If you map the fill aesthetic to a third variable, like `clarity`, you get a stacked bar chart.
If you map the fill aesthetic to a third variable, like `clarity`, you get a stacked bar chart. Each colored rectangle represents a combination of `cut` and `clarity`.
```{r}
ggplot(data = diamonds) +
geom_bar(mapping = aes(x = cut, fill = clarity))
```
#### Positions
But what if you don't want a stacked bar chart? What if you want the chart below? Could you make it?
```{r echo = FALSE}
@ -420,31 +386,32 @@ ggplot(data = diamonds) +
geom_bar(mapping = aes(x = cut, fill = clarity), position = "dodge")
```
The chart displays the same 40 color coded rectangles as the stacked bar chart above. Each bar represents a combination of `cut` and `clarity`. However, the position of the bars within the two charts is different. In the stacked bar chart, `ggplot2` stacked bars that have the same cut on top of each other. In this plot, `ggplot2` places bars that have the same cut beside each other.
The chart displays the same 40 color coded rectangles as the stacked bar chart above. However, the position of the bars within the two charts is different. In the stacked bar chart, `ggplot2` stacked bars that have the same `cut` value on top of each other. In this plot, `ggplot2` places bars that have the same `cut` value beside each other.
You can control this behavior by adding a _position adjustment_ to your geom. A position adjustment tells `ggplot2` what to do when two or more objects appear at the same spot in the coordinate system. To set a position adjustment, set the `position` argument of your geom function to one of `"identity"`, `"stack"`, `"dodge"`, `"fill"`, or `"jitter"`.
##### Position = "identity"
### Position = "identity"
When `position = "identity"`, `ggplot2` will place each object exactly where it falls in the context of the graph.
For our bar chart, this would mean that each bar would start at `y = 0` and would appear directly above the `cut` value that it describes. Since there are eight bars for each value of `cut`, many bars would overlap. The plot will look suspiciously like a stacked bar chart, but the stacked heights will be inaccurate, as each bar actually descends to `y = 0`. Some bars would not appear at all because they would be completely overlapped by other bars.
`position = "identity"` is a poor choice for a bar chart, but is the sensible default position adjustment for many geoms, such as `geom_point()`.
```{r}
ggplot(data = diamonds) +
geom_bar(mapping = aes(x = cut, fill = clarity), position = "identity") +
ggtitle('Position = "identity"')
```
`position = "identity"` is a poor choice for a bar chart, but is the sensible default position adjustment for many geoms, such as `geom_point()`.
***
**Tip** - You can add a title to your plot by adding `+ ggtitle("<Your Title>")` to your plot call.
***
##### Position = "stack"
### Position = "stack"
`position = "stack"` places overlapping objects directly _above_ one another. This is the default position adjustment for bar charts in `ggplot2`. Here each bar begins exactly where the bar below it ends.
@ -454,7 +421,7 @@ ggplot(data = diamonds) +
ggtitle('Position = "stack"')
```
##### Position = "fill"
### Position = "fill"
`position = "fill"` places overlapping objects above one another. However, it scales the objects to take up all of the available vertical space. As a result, `position = "fill"` makes it easy to compare relative frequencies across groups.
@ -464,7 +431,7 @@ ggplot(data = diamonds) +
ggtitle('Position = "fill"')
```
##### Position = "dodge"
### Position = "dodge"
`position = "dodge"` places overlapping objects directly _beside_ one another. This is how I created the graph at the start of the section.
@ -474,22 +441,18 @@ ggplot(data = diamonds) +
ggtitle('Position = "dodge"')
```
##### Position = "jitter"
### Position = "jitter"
The last type of position adjustment does not make sense for bar charts, but it can be very useful for scatterplots. Recall our first scatterplot.
The last type of position adjustment does not make sense for bar charts, but it can be very useful for scatterplots. Recall our first scatterplot. Did you notice that the plot displays only 126 points, even though there are 234 observations in the data set?
```{r echo = FALSE}
ggplot(data = mpg) +
geom_point(mapping = aes(x = displ, y = hwy))
```
Did you notice that the plot displays only 126 points, even though there are 234 observations in the data set? Did you also notice that the points appear to fall on a grid. Why should this be?
The values of `hwy` and `displ` are rounded to the nearest integer and tenths values. As a result, the points appear on a grid and many points overlap each other. This arrangement makes it hard to see where the mass of the data is. Are the data points spread equally throughout the graph, or is there one special combination of `hwy` and `displ` that contains 109 values?
This is common behavior in scatterplots. The points appear in a grid because the `hwy` and `displ` measurements were rounded to the nearest integer and tenths values. As a result, many points overlap each other because they've been rounded to the same values of `hwy` and `displ`. The rounding also explains why our graph appears to contain only 126 points. 108 points are hidden on top of other points located at the same value.
This arrangement can cause problems because it makes it hard to see where the mass of the data is. Is there one special combination of `hwy` and `displ` that contains 109 values? Or are the data points more or less equally spread throughout the graph?
You can avoid this overplotting problem by setting the position adjustment to "jitter". `position = "jitter"` adds a small amount of random noise to each point, which spreads the points out because no two points are likely to receive the same amount of random noise.
You can avoid this gridding by setting the position adjustment to "jitter". `position = "jitter"` adds a small amount of random noise to each point. This spreads the points out because no two points are likely to receive the same amount of random noise.
```{r}
ggplot(data = mpg) +
@ -511,9 +474,9 @@ But isn't random noise, you know, bad? It *is* true that jittering your data wil
***
#### Stats
## Stats
Let's turn our attention back to bar charts for a moment. Bar charts are interesting because they reveal something subtle about plots. Consider our basic bar chart.
Bar charts are interesting because they reveal something subtle about plots. Consider our basic bar chart.
```{r echo = FALSE}
ggplot(data = diamonds) +
@ -541,7 +504,7 @@ Some graphs, like scatterplots, plot the raw values of your data set. Other grap
knitr::include_graphics("images/visualization-stat-bar.png")
```
A few geoms, like `geom_point()`, plot your raw data as it is. To keep things simple, let's imagine that these geoms also transform the data. They just use a very lame transformation, the identity transformation, which returns the data in its original state. Now we can say that _every_ geom uses a stat.
A few geoms, like `geom_point()`, plot your raw data as it is. These geoms also apply a transformation to your data, the identity transformation, which returns the data in its original state. Now we can say that _every_ geom uses a stat.
```{r, echo = FALSE}
@ -552,7 +515,7 @@ You can learn which stat a geom uses, as well as what variables it computes by v
Stats are the most subtle part of plotting because you do not see them in action. `ggplot2` applies the transformation and stores the results behind the scenes. You only see the finished plot. Moreover, `ggplot2` applies stats automatically, with a very intuitive set of defaults. So why bother thinking about stats? Because you can use stats to do three very useful things.
First, you can change the stat that a geom uses. To do this, set the geom's stat argument. For example, you can map the heights of your bars to raw values---not counts---if you change the stat of `geom_bar()` from "count" to "identity". This works best if your data contains one value per bar, as in the demo data set below. Map the $y$ aesthetic to the variable that contains the bar heights.
First, you can change the stat that a geom uses. To do this, set the geom's stat argument. For example, you can map the heights of your bars to raw values---not counts---if you change the stat of `geom_bar()` from "count" to "identity". This works best if your data contains one value per bar, as in the demo data set below. Add a $y$ aesthetic, and map it to the variable that contains the bar heights.
```{r}
demo <- data.frame(
@ -566,7 +529,7 @@ ggplot(data = demo) +
geom_bar(mapping = aes(x = a, y = b), stat = "identity")
```
Use consideration when you change a geom's stat. Many combinations of geoms and stats will create incompatible results. In practice, I almost always use a geom's default stat.
Use consideration when you change a geom's stat. Many combinations of geoms and stats will create incompatible results. In practice, you will almost always use a geom's default stat.
Second, you can customize how a stat does its job. For example, the count stat takes a width parameter that it uses to set the widths of the bars in a bar plot. To pass a width value to the stat, provide a width argument to your geom function. `width = 1` will make the bars wide enough to touch each other.
@ -581,35 +544,33 @@ ggplot(data = diamonds) +
***
Finally, you can tell `ggplot2` how to use the stat. Many stats in `ggplot2` create multiple variables, some of which go unused. For example, the count stat creates two variables, `count` and `prop`, but `geom_bar()` only uses the `count` variable by default. It doesn't make sense to use the `prop` variable for `geom_bar()`, but consider `geom_count()`. `geom_count()` uses the "sum" stat to create bubble charts. Each bubble represents a group, and the size of the bubble displays how many points are in the group (e.g. the count of the group).
Finally, you can tell `ggplot2` how to use the stat. Many stats in `ggplot2` create multiple variables, some of which go unused. For example, `geom_count()` uses the "sum" stat to create bubble charts. Each bubble represents a group of data points, and the size of the bubble displays how many points are in the group (e.g. the count of the group).
```{r}
ggplot(data = diamonds) +
geom_count(mapping = aes(x = cut, y = clarity))
```
The sum stat is similar to the count stat because it creates two variables, `n` (count) and `prop`. With `geom_count()`, you can use the prop variable to display the proportion of observations in each row (or column) that appear in the groups. To tell `geom_count()` to use the prop variable, map $size$ to `..prop..`. The two dots that surround prop notify `ggplot2` that the prop variable appears in the transformed data set that is created by the stat, and not the raw data set. Be sure to include these dots whenever you refer to a variable that is created by a stat.
The sum stat creates two variables, `n` (count) and `prop`. By default, `geom_count()` uses the `n` variable to create the size of each bubble. You can use the prop variable in combination with a _group_ aesthetic to display the proportion of observations in each row (or column) that appear in the bubbles. To tell `geom_count()` to use the prop variable, map $size$ to `..prop..`. The two dots that surround prop notify `ggplot2` that the prop variable appears in the transformed data set that is created by the stat, and not in the raw data set. Be sure to include these dots whenever you refer to a variable that is created by a stat.
```{r}
ggplot(data = diamonds) +
geom_count(mapping = aes(x = cut, y = clarity, size = ..prop.., group = clarity))
```
`geom_count()` also requires you to set the group aesthetic to either the $x$ or $y$ variable when you use `..prop..`. If you set group to the $x$ variable, `..prop..` will show proportions across columns. If you set it to the $y$ variable, `..prop..` will show proportions across rows, as in the plot above. Here, the proportions in each row sum to one.
If you set _group_ to the $x$ variable, `..prop..` will show proportions across columns. If you set it to the $y$ variable, `..prop..` will show proportions across rows, as in the plot above. Here, the proportions in each row sum to one.
The best way to discover which variables are created by a stat is to visit the stat's help page.
The best way to discover which variables are created by a stat is to visit the stat's help page. `ggplot2` provides 22 stats for you to use. Each stat is saved as a function, which provides a convenient way to access a stat's help page, e.g. `?stat_identity`.
`ggplot2` provides 22 stats for you to use. It saves the stats as functions that begin with the suffix `stat_`. Although you call a stat as a character string surrounded by quotes, e.g. `stat = "identity"`, you would use the full function name to look up the stat's help page, e.g. `?stat_identity`.
The table below describes each stat in `ggplot2` and lists the parameters that the stat takes, as well as the variables that the stat makes. These are the variables that you can call with the `..` syntax, e.g. `..prop..`.
The table below describes each stat in `ggplot2` and lists the parameters that the stat takes, as well as the variables that the stat makes.
```{r, echo = FALSE}
knitr::include_graphics("images/visualization-stats.png")
```
### Polar charts
## Coordinate systems
Let's leave the cartesian coordinate system and examine the polar coordinate system for a moment. We will begin with a riddle: how is a bar chart similar to a coxcomb plot, like the one below?
Let's leave the cartesian coordinate system and examine the polar coordinate system. We will begin with a riddle: how is a bar chart similar to a coxcomb plot, like the one below?
```{r echo = FALSE, message = FALSE, fig.show='hold', fig.width=3, fig.height=4}
ggplot(data = diamonds) +
@ -621,9 +582,7 @@ ggplot(data = diamonds) +
Answer: A coxcomb plot is a bar chart plotted in polar coordinates. If this seems surprising, consider how you would make a coxcomb plot with `ggplot2`.
#### Coordinate systems
To make a coxcomb plot with `ggplot2`, first build a bar chart and then add `coord_polar()` to your plot call. Polar bar charts will look better if you also set the width parameter of `geom_bar()` to 1. This will ensure that no space appears between the bars.
To make a coxcomb plot, first build a bar chart and then add `coord_polar()` to your plot call. Polar bar charts will look better if you also set the width parameter of `geom_bar()` to 1. This will ensure that no space appears between the bars.
```{r}
ggplot(data = diamonds) +
@ -633,9 +592,7 @@ ggplot(data = diamonds) +
You can use `coord_polar()` to turn any plot in `ggplot2` into a polar chart. Whenever you add `coord_polar()` to a plot's call, `ggplot2` will draw the plot on a polar coordinate system. It will map the plot's $y$ variable to $r$ and the plot's $x$ variable to $\theta$. You can reverse this behavior by passing `coord_polar()` the argument `theta = "y"`.
Polar coordinates unlock another riddle as well. You may have noticed that `ggplot2` does not come with a pie chart geom. Why would that be?
In practice, a pie chart is just a stacked bar chart plotted in polar coordinates. To make a pie chart in `ggplot2`, create a stacked bar chart and:
Polar coordinates unlock another riddle as well. You may have noticed that `ggplot2` does not come with a pie chart geom. Why would that be? In practice, a pie chart is just a stacked bar chart plotted in polar coordinates. To make a pie chart in `ggplot2`, create a stacked bar chart and:
1. ensure that the x axis only has one value. An easy way to do this is to set `x = factor(1)`.
2. set the width of the bar to one, e.g. `width = 1`
@ -662,11 +619,11 @@ knitr::include_graphics("images/visualization-coordinate-systems.png")
***
#### Facets
## Facets
Coxcomb plots are especially useful when you make many coxcomb plots to compare side-by-side. Each coxcomb will act as a glyph that you can use to compare subsets of your data. The quickest way to draw separate coxcombs for subsets of your data is to facet your plot. When you _facet_ a plot, you split it into subplots that each describe a subset of the data.
Coxcomb plots are especially useful when you split your plot into _facets_, subplots that each display a subset of the data. Each subplot will act as a glyph that you can use to compare groups of data.
To facet your plot on a single discrete variable, add `facet_wrap()` to your plot call. The first argument of `facet_wrap()` is a formula, always a `~` followed by a variable name (here "formula" is the name of a data structure in R, not a synonym for "equation"). For example, here we create a separate subplot for each level of the `clarity` variable. The first subplot displays the group of points that have the `clarity` value `I1`. The second subplot displays the group of points that have the `clarity` value `SI2`. And so on.
To facet your plot, add `facet_wrap()` to your plot call. The first argument of `facet_wrap()` is a formula, always a `~` followed by a variable name (here "formula" is the name of a data structure in R, not a synonym for "equation"). The variable that you pass to `facet_wrap()` should be discrete. Here we create a separate subplot for each level of the `clarity` variable. The first subplot displays the group of points that have the `clarity` value `I1`. The second subplot displays the group of points that have the `clarity` value `SI2`. And so on.
```{r fig.height = 7, fig.width = 7}
ggplot(data = diamonds) +
@ -696,7 +653,7 @@ ggplot(data = mpg) +
facet_wrap(~ class)
```
##### Exercises
### Exercises
1. What graph will this code make?
@ -715,11 +672,11 @@ ggplot(data = mpg) +
```
### The layered grammar of graphics
## The layered grammar of graphics
You may not have realized it, but you learned much more in the previous sections than how to make scatterplots, bar charts, and coxcomb plots. You learned a foundation that you can use to make _any_ type of plot with `ggplot2`.
In the previous sections, you learned much more than how to make scatterplots, bar charts, and coxcomb plots. You learned a foundation that you can use to make _any_ type of plot with `ggplot2`.
To see this, let's add position adjustments, stats, coordinate systems, and faceting to our code template. In `ggplot2`, each of these parameters will work with every plot and every geom.
To see this, add position adjustments, stats, coordinate systems, and faceting to our code template. In `ggplot2`, each of these parameters will work with every plot and every geom.
```{r eval = FALSE}
ggplot(data = <DATA>) +
@ -732,9 +689,9 @@ ggplot(data = <DATA>) +
<FACET_FUNCTION>
```
Our new template takes seven parameters, the bracketed words that appear in the template. In practice, you rarely need to supply all seven parameters because `ggplot2` will provide useful defaults for everything except the data, the mappings, and the geom function.
Our new template takes seven parameters, the bracketed words that appear in the template. In practice, you rarely need to supply all seven parameters to make a graph because `ggplot2` will provide useful defaults for everything except the data, the mappings, and the geom function.
The seven parameters in the template compose the grammar of graphics, a formal system for building plots. The grammar of graphics is based on the insight that you can uniquely describe _any_ plot as a combination of---you guessed it: a data set, a geom, a set of mappings, a stat, a position adjustment, a coordinate system, and a faceting scheme.
The seven parameters in the template compose the grammar of graphics, a formal system for building plots. The grammar of graphics is based on the insight that you can uniquely describe _any_ plot as a combination of a data set, a geom, a set of mappings, a stat, a position adjustment, a coordinate system, and a faceting scheme.
To see how this works, consider how you could build a basic plot from scratch: you could start with a data set and then transform it into the information that you want to display (with a stat).
@ -743,94 +700,18 @@ knitr::include_graphics("images/visualization-grammar-1.png")
```
***
Next, you could choose a geometric object to represent each observation in the transformed data. You could then map aesthetic properties of the objects to variables in the data to display the values of the observations in a visual manner.
Next, you could choose a geometric object to represent each observation in the transformed data. You could then use the aesthetic properties of the geoms to represent variables in the data. You would map the values of each variable to the levels of an aesthetic.
```{r, echo = FALSE}
knitr::include_graphics("images/visualization-grammar-2.png")
```
You'd then select a coordinate system to place the objects into. You'd use the location of the objects (which is itself an aesthetic property) to display the values of the x and y variables. At that point, you would have a complete graph, but you could further adjust the positions of the objects or facet the graph if you like. You could also extend the plot by adding one or more additional layers, where each additional layer contains a data set, a geom, a set of mappings, a stat, and a position adjustment.
You'd then select a coordinate system to place the geoms into. You'd use the location of the objects (which is itself an aesthetic property) to display the values of the x and y variables. At that point, you would have a complete graph, but you could further adjust the positions of the geoms within the coordinate system (a position adjustment) or split the graph into subplots (facetting). You could also extend the plot by adding one or more additional layers, where each additional layer uses a data set, a geom, a set of mappings, a stat, and a position adjustment.
```{r, echo = FALSE}
knitr::include_graphics("images/visualization-grammar-3.png")
```
Although this method may seem complicated, you could use it to build _any_ plot that you imagine. In other words, you can use the code template that you've learned in this chapter to build hundreds of thousands of unique plots.
You could use this method to build _any_ plot that you imagine. In other words, you can use the code template that you've learned in this chapter to build hundreds of thousands of unique plots.
You could use these plots to explore and understand your data. Or you could use them to share insights about your data with others. Graphs make an excellent communication tool, especially when they are self explanatory. The final section will provide some tips on how to prepare your graphs to share with others.
## Communication with plots
The previous sections showed you how to make plots that you can use as a tools for _exploration_. When you made these plots, you knew---even before you looked at them---which variables the plot would display and which data sets the variables would come from. You might have even known what to look for in the completed plots, assuming that you made each plot with a goal in mind. As a result, it was not very important to put a title or a useful set of labels on your plots.
The importance of titles and labels changes once you use your plots for _communication_. Your audience will not share your background knowledge. In fact, they may not know anything about your plots except what the plots themselves display. If you want your plots to communicate your findings effectively, you will need to make them as self-explanatory as possible.
Luckily, `ggplot2` provides some features that can help you.
### Labels
You can add a title to any `ggplot2` plot by adding the command `labs()` to your plot call. Set the `title` argument of `labs()` to the character string that you would like to appear as the title of your plot. `ggplot2` will place the title at the top of your plot.
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
labs(title = "Fuel efficiency vs. Engine size")
```
You can also use `labs()` to replace the axis and legend labels in your plot, which might be a good idea if your data uses ambiguous or abbreviated variable names. To replace either of the axis labels, set the `x` or `y` arguments to a character string. `ggplot2` will replace the associated axis label with your character string.
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
labs(title = "Fuel efficiency vs. Engine size",
x = "Engine displacement (L)",
y = "Highway fuel efficiency (mpg)")
```
To replace a legend label, set the name of the aesthetic displayed in the legend to the character string that should appear as the title of the legend. For example, the legend in our plot corresponds to the color aesthetic. We can change it's title with the command, `labs(color = "New Title")`, or, more usefully:
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
labs(title = "Fuel efficiency vs. Engine size",
x = "Engine displacement (L)",
y = "Highway fuel efficiency (mpg)",
color = "Type of Car")
```
### Zooming
Often, it can be helpful to zoom in on a specific region of your plot. In `ggplot2` you can do this by adding `coord_cartesian()` to your plot and setting it's `xlim` and `ylim` arguments. Pass each argument a vector of two numbers, the minimum value to display on that axis and the maximum value, e.g.
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
coord_cartesian(xlim = c(5, 7), ylim = c(10, 30))
```
`coord_cartesian()` adds a cartesian coordinate system to your plot (which is the default coordinate system). However, the new coordinate system will use the zoomed in limits.
What if your plot uses a different coordinate system? Most of the other coordinate functions also take `xlim` and `ylim` arguments. You can look up the help pages of the coordinate functions to learn more.
### Themes
Finally, you can also quickly customize the "look" of your plot by adding a theme function to your plot call. This can be a useful thing to do, for example, if you'd like to save ink when you print your plots, or if you wish to ensure that the plots photocopy well.
`ggplot2` contains eight theme functions, listed in the table below. Each applies a different visual theme to your finished plot. You can think of the themes as "skins" for the plot. The themes change how the plot looks without changing the information that the plot displays.
To use any of the theme functions, add the function to your plot all. No arguments are necessary.
```{r}
ggplot(data = mpg, mapping = aes(x = displ, y = hwy)) +
geom_point(mapping = aes(color = class)) +
geom_smooth() +
theme_bw()
```
```{r, echo = FALSE}
knitr::include_graphics("images/visualization-themes.png")
```
You could use these plots to explore and understand your data. Or you could use them to share insights about your data with others. Chapter ? will provide some tips on how to use graphs to discover insights in your data. Chapter ? will provide some tips on how to prepare your graphs to share with others. Graphs make an excellent communication tool, especially when they are self explanatory.