Subsetting vectors

There are five main types of things that you can subset a vector with, i.e. that can be the i in x[i]:

1. A vector of positive integers. Subsetting with positive integers keeps the elements at those positions:

   x <- c("one", "two", "three", "four", "five")
   x[c(3, 2, 5)]
   #> [1] "three" "two"   "five"

   By repeating a position, you can actually make a longer output than input, making the term “subsetting” a bit of a misnomer.

   x[c(1, 1, 5, 5, 5, 2)]
   #> [1] "one"  "one"  "five" "five" "five" "two"

2. A vector of negative integers. Negative values drop the elements at the specified positions:

   x[c(-1, -3, -5)]
   #> [1] "two"  "four"

3. A logical vector. Subsetting with a logical vector keeps all values corresponding to a TRUE value. This is most often useful in conjunction with the comparison functions.

   x <- c(10, 3, NA, 5, 8, 1, NA)
   
   # All non-missing values of x
   !is.na(x)
   #> [1]  TRUE  TRUE FALSE  TRUE  TRUE  TRUE FALSE
   x[!is.na(x)]
   #> [1] 10  3  5  8  1
   
   # All even (or missing!) values of x
   x %% 2 == 0
   #> [1]  TRUE FALSE    NA FALSE  TRUE FALSE    NA
   x[x %% 2 == 0]
   #> [1] 10 NA  8 NA

   Note that, unlike filter(), NA indices will be included in the output as NAs.

4. A character vector. If you have a named vector, you can subset it with a character vector:

   x <- c(abc = 1, def = 2, xyz = 5)
   x[c("xyz", "def")]
   #> xyz def 
   #>   5   2

   As with subsetting with positive integers, you can use a character vector to duplicate individual entries.

5. Nothing. The final type of subsetting is nothing, x[], which returns the complete x. This is not useful for subsetting vectors, but as we’ll see shortly it is useful when subsetting 2d structures like tibbles.

Subsetting data frames

There are quite a few different waysRead https://adv-r.hadley.nz/subsetting.html#subset-multiple to see how you can also subset a data frame like it is a 1d object and how you can subset it with a matrix. that you can use [ with a data frame, but the most important way is to selecting rows and columns independently with df[rows, cols]. Here rows and cols are vectors as described above. For example, df[rows, ] and df[, cols] select just rows or just columns, using the empty subset to preserve the other dimension.

Here are a couple of examples:

df <- tibble(
  x = 1:3, 
  y = c("a", "e", "f"), 
  z = runif(3)
)

# Select first row and second column
df[1, 2]
#> # A tibble: 1 × 1
#>   y    
#>   <chr>
#> 1 a

# Select all rows and columns x and y
df[, c("x" , "y")]
#> # A tibble: 3 × 2
#>       x y    
#>   <int> <chr>
#> 1     1 a    
#> 2     2 e    
#> 3     3 f

# Select rows where `x` is greater than 1 and all columns
df[df$x > 1, ]
#> # A tibble: 2 × 3
#>       x y         z
#>   <int> <chr> <dbl>
#> 1     2 e     0.834
#> 2     3 f     0.601

We’ll come back to $ shortly, but you should be able to guess what df$x does from the context: it extracts the x variable from df. We need to use it here because [ doesn’t use tidy evaluation, so you need to be explicit about the source of the x variable.

There’s an important difference between tibbles and data frames when it comes to [. In this book we’ve mostly used tibbles, which are data frames, but they tweak some older behaviors to make your life a little easier. In most places, you can use tibbles and data frame interchangeably, so when we want to draw particular attention to R’s built-in data frame, we’ll write data.frames. So if df is a data.frame, then df[, cols] will return a vector if col selects a single column and a data frame if it selects more than one column. If df is a tibble, then [ will always return a tibble.

df1 <- data.frame(x = 1:3)
df1[, "x"]
#> [1] 1 2 3

df2 <- tibble(x = 1:3)
df2[, "x"]
#> # A tibble: 3 × 1
#>       x
#>   <int>
#> 1     1
#> 2     2
#> 3     3

One way to avoid this ambiguity with data.frames is to explicitly specify drop = FALSE:

df1[, "x", drop = FALSE]
#>   x
#> 1 1
#> 2 2
#> 3 3

dplyr equivalents

A number of dplyr verbs are special cases of [:

• filter() is equivalent to subsetting the rows with a logical vector, taking care to exclude missing values:

  df <- tibble(
    x = c(2, 3, 1, 1, NA), 
    y = letters[1:5], 
    z = runif(5)
  )
  df |> filter(x > 1)
  
  # same as
  df[!is.na(df$x) & df$x > 1, ]

  Another common technique in the wild is to use which() for its side-effect of dropping missing values: df[which(df$x > 1), ].

• arrange() is equivalent to subsetting the rows with an integer vector, usually created with order():

  df |> arrange(x, y)
  
  # same as
  df[order(df$x, df$y), ]

  You can use order(decreasing = TRUE) to sort all columns in descending order or -rank(col) to individual sort columns in decreasing order.

• Both select() and relocate() are similar to subsetting the columns with a character vector:

  df |> select(x, z)
  
  # same as
  df[, c("x", "z")]

Base R also provides a function that combines the features of filter() and select()But it doesn’t handle grouped data frames differently and it doesn’t support selection helper functions like starts_with(). called subset():

df |> 
  filter(x > 1) |> 
  select(y, z)
#> # A tibble: 2 × 2
#>   y           z
#>   <chr>   <dbl>
#> 1 a     0.157  
#> 2 b     0.00740

# same as
df |> subset(x > 1, c(y, z))
#> # A tibble: 2 × 2
#>   y           z
#>   <chr>   <dbl>
#> 1 a     0.157  
#> 2 b     0.00740

This function was the inspiration for much of dplyr’s syntax.

Exercises

1. Create functions that take a vector as input and return:

   1. The elements at even numbered positions.
   2. Every element except the last value.
   3. Only even values (and no missing values).

2. Why is x[-which(x > 0)] not the same as x[x <= 0]? Read the documentation for which() and do some experiments to figure it out.

Data frames

[[ and $ can be used like pull() to extract columns out of a data frame. [[ can access by position or by name, and $ is specialized for access by name:

tb <- tibble(
  x = 1:4,
  y = c(10, 4, 1, 21)
)

# by position
tb[[1]]
#> [1] 1 2 3 4

# by name
tb[["x"]]
#> [1] 1 2 3 4
tb$x
#> [1] 1 2 3 4

They can also be used to create new columns, the base R equivalent of mutate():

tb$z <- tb$x + tb$y
tb
#> # A tibble: 4 × 3
#>       x     y     z
#>   <int> <dbl> <dbl>
#> 1     1    10    11
#> 2     2     4     6
#> 3     3     1     4
#> 4     4    21    25

There are a number other base approaches to creating new columns including with transform(), with(), and within(). Hadley collected a few examples at https://gist.github.com/hadley/1986a273e384fb2d4d752c18ed71bedf.

Using $ directly is convenient when performing quick summaries. For example, if you just want find the size of the biggest diamond or the possible values of cut, there’s no need to use summarise():

max(diamonds$carat)
#> [1] 5.01

levels(diamonds$cut)
#> [1] "Fair"      "Good"      "Very Good" "Premium"   "Ideal"

Tibbles

There are a couple of important differences between tibbles and base data.frames when it comes to $. Data frames match the prefix of any variable names (so-called partial matching) and don’t complain if a column doesn’t exist:

df <- data.frame(x1 = 1)
df$x
#> Warning in df$x: partial match of 'x' to 'x1'
#> [1] 1
df$z
#> NULL

Tibbles are more strict: they only ever match variable names exactly and they will generate a warning if the column you are trying to access doesn’t exist:

tb <- tibble(x1 = 1)

tb$x
#> Warning: Unknown or uninitialised column: `x`.
#> NULL
tb$z
#> Warning: Unknown or uninitialised column: `z`.
#> NULL

For this reason we sometimes joke that tibbles are lazy and surly: they do less and complain more.

Lists

[[ and $ are also really important for working with lists, and it’s important to understand how they differ to [. Lets illustrate the differences with a list named l:

l <- list(
  a = 1:3, 
  b = "a string", 
  c = pi, 
  d = list(-1, -5)
)

• [ extracts a sub-list. It doesn’t matter how many elements you extract, the result will always be a list.

  str(l[1:2])
  #> List of 2
  #>  $ a: int [1:3] 1 2 3
  #>  $ b: chr "a string"
  str(l[4])
  #> List of 1
  #>  $ d:List of 2
  #>   ..$ : num -1
  #>   ..$ : num -5

  Like with vectors, you can subset with a logical, integer, or character vector.

• [[ and $ extract a single component from a list. They remove a level of hierarchy from the list.

  str(l[[1]])
  #>  int [1:3] 1 2 3
  str(l[[4]])
  #> List of 2
  #>  $ : num -1
  #>  $ : num -5
  
  str(l$a)
  #>  int [1:3] 1 2 3

The difference between [ and [[ is particularly important for lists because [[ drills down into the list while [ returns a new, smaller list. To help you remember the difference, take a look at the an unusual pepper shaker shown in #fig-pepper-1. If this pepper shaker is your list pepper, then, pepper[1] is a pepper shaker containing a single pepper packet, as in #fig-pepper-2. If we suppose this pepper shaker is a list pepper, then, pepper[1] is a pepper shaker containing a single pepper packet, as in #fig-pepper-2. pepper[2] would look the same, but would contain the second packet. pepper[1:2] would be a pepper shaker containing two pepper packets. pepper[[1]] would extract the pepper packet itself, as in #fig-pepper-3.

This same principle applies when you use 1d [ with a data frame:

df <- tibble(x = 1:3, y = 3:5)

# returns a one-column data frame
df["x"]
#> # A tibble: 3 × 1
#>       x
#>   <int>
#> 1     1
#> 2     2
#> 3     3

# returns the contents of x
df[["x"]]
#> [1] 1 2 3

Exercises

1. What happens when you use [[ with a positive integer that’s bigger than the length of the vector? What happens when you subset with a name that doesn’t exist?

2. What would pepper[[1]][1] be? What about pepper[[1]][[1]]?