282 lines
9.6 KiB
Plaintext
282 lines
9.6 KiB
Plaintext
```{r setup, include = FALSE}
|
|
library(broom)
|
|
library(ggplot2)
|
|
library(dplyr)
|
|
```
|
|
|
|
# Model visualisation
|
|
|
|
In this chapter we will explore model visualisation from two different sides:
|
|
|
|
1. Use a model to make it easier to see important patterns in our data.
|
|
|
|
1. Use visualisation to understand what a model is telling us about our data.
|
|
|
|
We're going to give you a basic strategy, and point you to places to learn more. The key is to think about data generated from your model as regular data - you're going to want to manipulate it and visualise it in many different ways.
|
|
|
|
Being good at modelling is a mixture of having some good general principles and having a big toolbox of techniques. Here we'll focus on general techniques to help you undertand what your model is telling you.
|
|
|
|
Focus on constructing models that help you better understand the data. This will generally lead to models that predict better. But you have to beware of overfitting the data - in the next section we'll discuss some formal methods. But a healthy dose of scepticism is also a powerful: do you believe that a pattern you see in your sample is going to generalise to a wider population?
|
|
|
|
Transition from implicit knowledge in your head and in data to explicit knowledge in the model. In other words, you want to make explicit your knowledge of the data and capture it explicitly in a model. This makes it easier to apply to new domains, and easier for others to use. But you must always remember that your knowledge is incomplete.
|
|
|
|
For very large and complex datasets this is going to be a lot of
|
|
|
|
In the next chapter, you'll also learn about how to visualisation the model-level summaries, and the model parameters.
|
|
|
|
```{r}
|
|
# Helper functions
|
|
add_predictions <- function(data, ...) {
|
|
models <- list(...)
|
|
for (nm in names(models)) {
|
|
data[[nm]] <- predict(models[[nm]], data)
|
|
}
|
|
data
|
|
}
|
|
|
|
add_residuals <- function(data, ...) {
|
|
models <- list(...)
|
|
|
|
for (nm in names(models)) {
|
|
y <- eval(predictor(models[[nm]]), data)
|
|
yhat <- predict(models[[nm]], data)
|
|
|
|
data[[nm]] <- y - yhat
|
|
}
|
|
data
|
|
}
|
|
|
|
predictor <- function(model) {
|
|
terms(model)[[2]]
|
|
}
|
|
```
|
|
|
|
|
|
## Residuals
|
|
|
|
To motivate the use of models we're going to start with an interesting pattern from the NYC flights dataset: the number of flights per day.
|
|
|
|
```{r}
|
|
library(nycflights13)
|
|
library(lubridate)
|
|
library(dplyr)
|
|
|
|
daily <- flights %>%
|
|
mutate(date = make_datetime(year, month, day)) %>%
|
|
group_by(date) %>%
|
|
summarise(n = n())
|
|
|
|
ggplot(daily, aes(date, n)) +
|
|
geom_line()
|
|
```
|
|
|
|
Understand this pattern is challenging because there's a very strong day-of-week effect that dominates the subtler patterns:
|
|
|
|
```{r}
|
|
daily <- daily %>%
|
|
mutate(wday = wday(date, label = TRUE))
|
|
library(lvplot)
|
|
ggplot(daily, aes(wday, n)) +
|
|
geom_boxplot()
|
|
```
|
|
|
|
Why are there so few flights on Saturdays? My hypthosis is that most travel is for business, and you generally don't want to spend all of Sunday away from home. Sunday is in between Saturday and Monday because sometimes you have to leave Sunday night in order to arrive in time for a meeting on Monday morning.
|
|
|
|
One way to remove this strong pattern is to fit a model that "explains" the day of week effect, and then look at the residuals:
|
|
|
|
```{r}
|
|
mod <- lm(n ~ wday, data = daily)
|
|
daily <- daily %>% add_residuals(n_resid = mod)
|
|
|
|
daily %>%
|
|
ggplot(aes(date, n_resid)) +
|
|
geom_hline(yintercept = 0, size = 2, colour = "white") +
|
|
geom_line()
|
|
```
|
|
|
|
Note the change in the y-axis: now we are seeing the deviation from the expected number of flights, given the day of week. This plot is interesting because now that we've removed the very day-of-week effect, we can see some of the subtler patterns that remain
|
|
|
|
1. There are some with very few flights. If you're familiar with American
|
|
public holidays, you might spot New Year's day, July 4th, Thanksgiving
|
|
and Christmas. There are some others that dont' seem to correspond to
|
|
|
|
```{r}
|
|
daily %>% filter(n_resid < -100)
|
|
```
|
|
|
|
1. There seems to be some smoother long term trend over the course of a year:
|
|
there are fewer flights in January, and more in summer (May-Sep). We can't
|
|
do much more with this trend than note it because we only have a single
|
|
year of data.
|
|
|
|
1. Our day of week adjustment seems to fail starting around June: you can
|
|
still see a strong regular pattern that our model hasn't removed.
|
|
|
|
We'll tackle the day of week effect first. Let's start by tweaking our plot drawing one line for each day of the week.
|
|
|
|
```{r}
|
|
ggplot(daily, aes(date, n_resid, colour = wday)) +
|
|
geom_hline(yintercept = 0, size = 2, colour = "white") +
|
|
geom_line()
|
|
```
|
|
|
|
This makes it clear that the problem with our model is mostly Saturdays: it seems like during some there are more flights on Saturdays than we expect, and during Fall there are fewer. I suspect this is because of summer holidays: many people going on holiday in the summer, and people don't mind travelling on Saturdays for vacation.
|
|
|
|
Let's zoom in on that pattern, this time looking at the raw numbers:
|
|
|
|
```{r}
|
|
daily %>%
|
|
filter(wday == "Sat") %>%
|
|
ggplot(aes(date, n)) +
|
|
geom_line() +
|
|
scale_x_datetime(date_breaks = "1 month", date_labels = "%d-%b")
|
|
```
|
|
|
|
So it looks like summer holidays is from early June to late August. And that seems to line up fairly well with the state's school holidays <http://schools.nyc.gov/Calendar/2013-2014+School+Year+Calendars.htm>: Jun 26 - Sep 9. So lets add a "school" variable to attemp to control for that.
|
|
|
|
```{r}
|
|
daily <- daily %>%
|
|
mutate(school = cut(date,
|
|
breaks = as.POSIXct(ymd(20130101, 20130605, 20130825, 20140101)),
|
|
labels = c("spring", "summer", "fall")
|
|
))
|
|
|
|
daily %>%
|
|
filter(wday == "Sat") %>%
|
|
ggplot(aes(date, n, colour = school)) +
|
|
geom_line() +
|
|
scale_x_datetime(date_breaks = "1 month", date_labels = "%d-%b")
|
|
```
|
|
|
|
It's useful to see how this new variable affects the other days of the week:
|
|
|
|
```{r}
|
|
daily %>%
|
|
ggplot(aes(wday, n, colour = school)) +
|
|
geom_boxplot()
|
|
```
|
|
|
|
It looks like there is significant variation, so fitting a separate day of week effect for each term is reasonable. This improves our model, but not as much as we might hope:
|
|
|
|
```{r}
|
|
mod2 <- lm(n ~ wday * school, data = daily)
|
|
daily$n_resid2 <- resid(mod2)
|
|
|
|
ggplot(daily, aes(date, n_resid2)) +
|
|
geom_line()
|
|
```
|
|
|
|
That's because this model is basically calculating an average for each combination of wday and school term. We have a lot of big outliers, so they tend to drag the mean far away from the typical value.
|
|
|
|
```{r}
|
|
mean <- daily %>%
|
|
group_by(wday, school) %>%
|
|
summarise(n = mean(n))
|
|
|
|
|
|
daily %>%
|
|
ggplot(aes(wday, n, colour = school)) +
|
|
geom_boxplot() +
|
|
geom_point(data = mean, size = 3, shape = 17, position = position_dodge(width = 0.75))
|
|
|
|
```
|
|
|
|
We can reduce this problem by switch to a robust model, fit by `MASS::rlm()`. A robust model is a variation of the linear which you can think of a fitting medians, instead of means. This greatly reduces the impact of the outliers on our estimates, and gives a result that does a good job of removing the day of week pattern:
|
|
|
|
```{r}
|
|
mod2 <- MASS::rlm(n ~ wday * school, data = daily)
|
|
daily$n_resid2 <- resid(mod2)
|
|
|
|
ggplot(daily, aes(date, n_resid2)) +
|
|
geom_hline(yintercept = 0, size = 2, colour = "white") +
|
|
geom_line()
|
|
```
|
|
|
|
It's now much easier to see the long term trend, and the positive and negative outliers.
|
|
|
|
### Exercises
|
|
|
|
1. Use your google sleuthing skills to brainstorm why there were fewer than
|
|
expected flights on Jan 20, May 26, and Sep 9. (Hint: they all have the
|
|
same explanation.) How would these days generalise to another year?
|
|
|
|
1. What do the days with high positive residuals represent?
|
|
|
|
1. What happens if you fit a day of week effect that varies by month?
|
|
Why is this not very helpful?
|
|
|
|
1. Above we made the hypothesis that people leaving on Sundays are more
|
|
likely to be business travellers who need to be somewhere on Monday.
|
|
Explore that hypothesis by seeing how it breaks down based on distance:
|
|
if it's true, you'd expect to see more Sunday flights to places that
|
|
are far away.
|
|
|
|
1. It's a little frustrating that Sunday and Saturday are on separate ends
|
|
of the plot. Write a small function to set the levels of the factor so
|
|
that the week starts on Monday.
|
|
|
|
|
|
## Predictions
|
|
|
|
Focus on predictions from a model because this works for any type of model. Visualising parameters can also be useful, but tends to be most useful when you have many similar models. Visualising predictions works regardless of the model family.
|
|
|
|
```{r}
|
|
|
|
```
|
|
|
|
Visualising high-dimensional models is challenging. You'll need to partition off a useable slice at a time.
|
|
|
|
|
|
```{r}
|
|
library(tidyr)
|
|
|
|
date_vars <- function(df) {
|
|
df %>% mutate(
|
|
school = cut(date,
|
|
breaks = as.POSIXct(ymd(20130101, 20130605, 20130825, 20140101)),
|
|
labels = c("spring", "summer", "fall")
|
|
),
|
|
wday = wday(date, label = TRUE)
|
|
)
|
|
}
|
|
|
|
daily %>%
|
|
expand(date) %>%
|
|
date_vars() %>%
|
|
add_predictions(pred = mod2) %>%
|
|
ggplot(aes(date, pred)) +
|
|
geom_line()
|
|
|
|
daily %>%
|
|
expand(date, wday = "Sat", school = "spring") %>%
|
|
add_predictions(pred = mod2) %>%
|
|
ggplot(aes(date, pred)) +
|
|
geom_line()
|
|
|
|
|
|
daily %>%
|
|
expand(wday, school) %>%
|
|
add_predictions(pred = mod2) %>%
|
|
ggplot(aes(wday, pred, colour = school)) +
|
|
geom_point() +
|
|
geom_line(aes(group = school))
|
|
|
|
```
|
|
|
|
## Delays and weather
|
|
|
|
```{r}
|
|
hourly <- flights %>%
|
|
group_by(origin, time_hour) %>%
|
|
summarise(
|
|
delay = mean(dep_delay, na.rm = TRUE)
|
|
) %>%
|
|
inner_join(weather, by = c("origin", "time_hour"))
|
|
|
|
ggplot(hourly, aes(time_hour, delay)) +
|
|
geom_point()
|
|
|
|
ggplot(hourly, aes(hour(time_hour), delay)) +
|
|
geom_boxplot(aes(group = hour(time_hour)))
|
|
```
|