r4ds/oreilly/spreadsheets.html

533 lines
35 KiB
HTML
Raw Normal View History

<section data-type="chapter" id="chp-spreadsheets">
2022-11-19 01:55:22 +08:00
<h1><span id="sec-import-spreadsheets" class="quarto-section-identifier d-none d-lg-block"><span class="chapter-title">Spreadsheets</span></span></h1>
<section id="introduction" data-type="sect1">
<h1>
Introduction</h1>
<p>So far you have learned about importing data from plain text files, e.g. <code>.csv</code> and <code>.tsv</code> files. Sometimes you need to analyze data that lives in a spreadsheet. In this chapter we will introduce you to tools for working with data in Excel spreadsheets and Google Sheets. This will build on much of what youve learned in <a href="#chp-data-import" data-type="xref">#chp-data-import</a> but we will also discuss additional considerations and complexities when working with data from spreadsheets.</p>
<p>If you or your collaborators are using spreadsheets for organizing data, we strongly recommend reading the paper “Data Organization in Spreadsheets” by Karl Broman and Kara Woo: <a href="https://doi.org/10.1080/00031305.2017.1375989" class="uri">https://doi.org/10.1080/00031305.2017.1375989</a>. The best practices presented in this paper will save you much headache down the line when you import the data from a spreadsheet into R to analyse and visualise.</p>
</section>
<section id="excel" data-type="sect1">
<h1>
Excel</h1>
<section id="prerequisites" data-type="sect2">
<h2>
Prerequisites</h2>
<p>In this chapter, youll learn how to load data from Excel spreadsheets in R with the <strong>readxl</strong> package. This package is non-core tidyverse, so you need to load it explicitly but it is installed automatically when you install the tidyverse package.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">library(readxl)
library(tidyverse)</pre>
</div>
<p><strong>xlsx</strong> and <strong>XLConnect</strong> can be used for reading data from and writing data to Excel spreadsheets. However, these two packages require Java installed on your machine and the rJava package. Due to potential challenges with installation, we recommend using alternative packages weve introduced in this chapter.</p>
</section>
<section id="getting-started" data-type="sect2">
<h2>
Getting started</h2>
<p>Most of readxls functions allow you to load Excel spreadsheets into R:</p>
<ul><li>
2022-11-19 00:30:32 +08:00
<code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_xls()</a></code> reads Excel files with <code>xls</code> format.</li>
<li>
2022-11-19 00:30:32 +08:00
<code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_xlsx()</a></code> read Excel files with <code>xlsx</code> format.</li>
<li>
2022-11-19 00:30:32 +08:00
<code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_excel()</a></code> can read files with both <code>xls</code> and <code>xlsx</code> format. It guesses the file type based on the input.</li>
</ul><p>These functions all have similar syntax just like other functions we have previously introduced for reading other types of files, e.g. <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code>, <code><a href="https://readr.tidyverse.org/reference/read_table.html">read_table()</a></code>, etc. For the rest of the chapter we will focus on using <code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_excel()</a></code>.</p>
</section>
<section id="sec-reading-spreadsheets" data-type="sect2">
<h2>
Reading spreadsheets</h2>
<p><a href="#fig-students-excel" data-type="xref">#fig-students-excel</a> shows what the spreadsheet were going to read into R looks like in Excel.</p>
<div class="cell">
<div class="cell-output-display">
2022-11-19 00:42:43 +08:00
<figure id="fig-students-excel"><p><img src="images/import-spreadsheets-students.png" alt="A look at the students spreadsheet in Excel. The spreadsheet contains information on 6 students, their ID, full name, favourite food, meal plan, and age." width="1200"/></p>
<figcaption>Spreadsheet called students.xlsx in Excel.</figcaption>
</figure>
</div>
</div>
2022-11-19 00:30:32 +08:00
<p>The first argument to <code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_excel()</a></code> is the path to the file to read.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">students &lt;- read_excel("data/students.xlsx")</pre>
</div>
2022-11-19 00:30:32 +08:00
<p><code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_excel()</a></code> will read the file in as a tibble.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">students
#&gt; # A tibble: 6 × 5
#&gt; `Student ID` `Full Name` favourite.food mealPlan AGE
#&gt; &lt;dbl&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt;
#&gt; 1 1 Sunil Huffmann Strawberry yoghurt Lunch only 4
#&gt; 2 2 Barclay Lynn French fries Lunch only 5
#&gt; 3 3 Jayendra Lyne N/A Breakfast and lunch 7
#&gt; 4 4 Leon Rossini Anchovies Lunch only &lt;NA&gt;
#&gt; 5 5 Chidiegwu Dunkel Pizza Breakfast and lunch five
#&gt; 6 6 Güvenç Attila Ice cream Lunch only 6</pre>
</div>
<p>We have six students in the data and five variables on each student. However there are a few things we might want to address in this dataset:</p>
<ol type="1"><li>
<p>The column names are all over the place. You can provide column names that follow a consistent format; we recommend <code>snake_case</code> using the <code>col_names</code> argument.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel(
"data/students.xlsx",
col_names = c("student_id", "full_name", "favourite_food", "meal_plan", "age")
)
#&gt; # A tibble: 7 × 5
#&gt; student_id full_name favourite_food meal_plan age
#&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt;
#&gt; 1 Student ID Full Name favourite.food mealPlan AGE
#&gt; 2 1 Sunil Huffmann Strawberry yoghurt Lunch only 4
#&gt; 3 2 Barclay Lynn French fries Lunch only 5
#&gt; 4 3 Jayendra Lyne N/A Breakfast and lunch 7
#&gt; 5 4 Leon Rossini Anchovies Lunch only &lt;NA&gt;
#&gt; 6 5 Chidiegwu Dunkel Pizza Breakfast and lunch five
#&gt; # … with 1 more row</pre>
</div>
<p>Unfortunately, this didnt quite do the trick. You now have the variable names we want, but what was previously the header row now shows up as the first observation in the data. You can explicitly skip that row using the <code>skip</code> argument.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel(
"data/students.xlsx",
col_names = c("student_id", "full_name", "favourite_food", "meal_plan", "age"),
skip = 1
)
#&gt; # A tibble: 6 × 5
#&gt; student_id full_name favourite_food meal_plan age
#&gt; &lt;dbl&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt;
#&gt; 1 1 Sunil Huffmann Strawberry yoghurt Lunch only 4
#&gt; 2 2 Barclay Lynn French fries Lunch only 5
#&gt; 3 3 Jayendra Lyne N/A Breakfast and lunch 7
#&gt; 4 4 Leon Rossini Anchovies Lunch only &lt;NA&gt;
#&gt; 5 5 Chidiegwu Dunkel Pizza Breakfast and lunch five
#&gt; 6 6 Güvenç Attila Ice cream Lunch only 6</pre>
</div>
</li>
<li>
<p>In the <code>favourite_food</code> column, one of the observations is <code>N/A</code>, which stands for “not available” but its currently not recognized as an <code>NA</code> (note the contrast between this <code>N/A</code> and the age of the fourth student in the list). You can specify which character strings should be recognized as <code>NA</code>s with the <code>na</code> argument. By default, only <code>""</code> (empty string, or, in the case of reading from a spreadsheet, an empty cell) is recognized as an <code>NA</code>.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel(
"data/students.xlsx",
col_names = c("student_id", "full_name", "favourite_food", "meal_plan", "age"),
skip = 1,
na = c("", "N/A")
)
#&gt; # A tibble: 6 × 5
#&gt; student_id full_name favourite_food meal_plan age
#&gt; &lt;dbl&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt;
#&gt; 1 1 Sunil Huffmann Strawberry yoghurt Lunch only 4
#&gt; 2 2 Barclay Lynn French fries Lunch only 5
#&gt; 3 3 Jayendra Lyne &lt;NA&gt; Breakfast and lunch 7
#&gt; 4 4 Leon Rossini Anchovies Lunch only &lt;NA&gt;
#&gt; 5 5 Chidiegwu Dunkel Pizza Breakfast and lunch five
#&gt; 6 6 Güvenç Attila Ice cream Lunch only 6</pre>
</div>
</li>
<li>
2022-11-19 00:30:32 +08:00
<p>One other remaining issue is that <code>age</code> is read in as a character variable, but it really should be numeric. Just like with <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code> and friends for reading data from flat files, you can supply a <code>col_types</code> argument to <code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_excel()</a></code> and specify the column types for the variables you read in. The syntax is a bit different, though. Your options are <code>"skip"</code>, <code>"guess"</code>, <code>"logical"</code>, <code>"numeric"</code>, <code>"date"</code>, <code>"text"</code> or <code>"list"</code>.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel(
"data/students.xlsx",
col_names = c("student_id", "full_name", "favourite_food", "meal_plan", "age"),
skip = 1,
na = c("", "N/A"),
col_types = c("numeric", "text", "text", "text", "numeric")
)
#&gt; Warning: Expecting numeric in E6 / R6C5: got 'five'
#&gt; # A tibble: 6 × 5
#&gt; student_id full_name favourite_food meal_plan age
#&gt; &lt;dbl&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt;
#&gt; 1 1 Sunil Huffmann Strawberry yoghurt Lunch only 4
#&gt; 2 2 Barclay Lynn French fries Lunch only 5
#&gt; 3 3 Jayendra Lyne &lt;NA&gt; Breakfast and lunch 7
#&gt; 4 4 Leon Rossini Anchovies Lunch only NA
#&gt; 5 5 Chidiegwu Dunkel Pizza Breakfast and lunch NA
#&gt; 6 6 Güvenç Attila Ice cream Lunch only 6</pre>
</div>
<p>However, this didnt quite produce the desired result either. By specifying that <code>age</code> should be numeric, we have turned the one cell with the non-numeric entry (which had the value <code>five</code>) into an <code>NA</code>. In this case, we should read age in as <code>"text"</code> and then make the change once the data is loaded in R.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">students &lt;- read_excel(
"data/students.xlsx",
col_names = c("student_id", "full_name", "favourite_food", "meal_plan", "age"),
skip = 1,
na = c("", "N/A"),
col_types = c("numeric", "text", "text", "text", "text")
)
students &lt;- students |&gt;
mutate(
age = if_else(age == "five", "5", age),
age = parse_number(age)
)
students
#&gt; # A tibble: 6 × 5
#&gt; student_id full_name favourite_food meal_plan age
#&gt; &lt;dbl&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt;
#&gt; 1 1 Sunil Huffmann Strawberry yoghurt Lunch only 4
#&gt; 2 2 Barclay Lynn French fries Lunch only 5
#&gt; 3 3 Jayendra Lyne &lt;NA&gt; Breakfast and lunch 7
#&gt; 4 4 Leon Rossini Anchovies Lunch only NA
#&gt; 5 5 Chidiegwu Dunkel Pizza Breakfast and lunch 5
#&gt; 6 6 Güvenç Attila Ice cream Lunch only 6</pre>
</div>
</li>
</ol><p>It took us multiple steps and trial-and-error to load the data in exactly the format we want, and this is not unexpected. Data science is an iterative process. There is no way to know exactly what the data will look like until you load it and take a look at it. Well, there is one way, actually. You can open the file in Excel and take a peek. That might be tempting, but its strongly not recommended. <!--# TO DO: Provide reason why it's not recommended. --> Instead, you should not be afraid of doing what we did here: load the data, take a peek, make adjustments to your code, load it again, and repeat until youre happy with the result.</p>
</section>
<section id="reading-individual-sheets" data-type="sect2">
<h2>
Reading individual sheets</h2>
<p>An important feature that distinguishes spreadsheets from flat files is the notion of multiple sheets. <a href="#fig-penguins-islands" data-type="xref">#fig-penguins-islands</a> shows an Excel spreadsheet with multiple sheets. The data come from the <strong>palmerpenguins</strong> package. Each sheet contains information on penguins from a different island where data were collected.</p>
<div class="cell">
<div class="cell-output-display">
2022-11-19 00:42:43 +08:00
<figure id="fig-penguins-islands"><p><img src="images/import-spreadsheets-penguins-islands.png" alt="A look at the penguins spreadsheet in Excel. The spreadsheet contains has three sheets: Torgersen Island, Biscoe Island, and Dream Island." width="1514"/></p>
<figcaption>Spreadsheet called penguins.xlsx in Excel.</figcaption>
</figure>
</div>
</div>
2022-11-19 00:30:32 +08:00
<p>You can read a single sheet from a spreadsheet with the <code>sheet</code> argument in <code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_excel()</a></code>.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel("data/penguins.xlsx", sheet = "Torgersen Island")
#&gt; # A tibble: 52 × 8
#&gt; species island bill_length_mm bill_dep…¹ flipp…² body_…³ sex year
#&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt;
#&gt; 1 Adelie Torgersen 39.1 18.7 181 3750 male 2007
#&gt; 2 Adelie Torgersen 39.5 17.399999… 186 3800 fema… 2007
#&gt; 3 Adelie Torgersen 40.299999999999997 18 195 3250 fema… 2007
#&gt; 4 Adelie Torgersen NA NA NA NA NA 2007
#&gt; 5 Adelie Torgersen 36.700000000000003 19.3 193 3450 fema… 2007
#&gt; 6 Adelie Torgersen 39.299999999999997 20.6 190 3650 male 2007
#&gt; # … with 46 more rows, and abbreviated variable names ¹bill_depth_mm,
#&gt; # ²flipper_length_mm, ³body_mass_g</pre>
</div>
<p>Some variables that appear to contain numerical data are read in as characters due to the character string <code>"NA"</code> not being recognized as a true <code>NA</code>.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">penguins_torgersen &lt;- read_excel("data/penguins.xlsx", sheet = "Torgersen Island", na = "NA")
penguins_torgersen
#&gt; # A tibble: 52 × 8
#&gt; species island bill_length_mm bill_depth_mm flippe…¹ body_…² sex year
#&gt; &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;chr&gt; &lt;dbl&gt;
#&gt; 1 Adelie Torgersen 39.1 18.7 181 3750 male 2007
#&gt; 2 Adelie Torgersen 39.5 17.4 186 3800 fema… 2007
#&gt; 3 Adelie Torgersen 40.3 18 195 3250 fema… 2007
#&gt; 4 Adelie Torgersen NA NA NA NA &lt;NA&gt; 2007
#&gt; 5 Adelie Torgersen 36.7 19.3 193 3450 fema… 2007
#&gt; 6 Adelie Torgersen 39.3 20.6 190 3650 male 2007
#&gt; # … with 46 more rows, and abbreviated variable names ¹flipper_length_mm,
#&gt; # ²body_mass_g</pre>
</div>
2022-11-19 00:30:32 +08:00
<p>However, we cheated here a bit. We looked inside the Excel spreadsheet, which is not a recommended workflow. Instead, you can use <code><a href="https://readxl.tidyverse.org/reference/excel_sheets.html">excel_sheets()</a></code> to get information on all sheets in an Excel spreadsheet, and then read the one(s) youre interested in.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">excel_sheets("data/penguins.xlsx")
#&gt; [1] "Torgersen Island" "Biscoe Island" "Dream Island"</pre>
</div>
2022-11-19 00:30:32 +08:00
<p>Once you know the names of the sheets, you can read them in individually with <code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_excel()</a></code>.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">penguins_biscoe &lt;- read_excel("data/penguins.xlsx", sheet = "Biscoe Island", na = "NA")
penguins_dream &lt;- read_excel("data/penguins.xlsx", sheet = "Dream Island", na = "NA")</pre>
</div>
<p>In this case the full penguins dataset is spread across three sheets in the spreadsheet. Each sheet has the same number of columns but different numbers of rows.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">dim(penguins_torgersen)
#&gt; [1] 52 8
dim(penguins_biscoe)
#&gt; [1] 168 8
dim(penguins_dream)
#&gt; [1] 124 8</pre>
</div>
2022-11-19 00:30:32 +08:00
<p>We can put them together with <code><a href="https://dplyr.tidyverse.org/reference/bind_rows.html">bind_rows()</a></code>.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">penguins &lt;- bind_rows(penguins_torgersen, penguins_biscoe, penguins_dream)
penguins
#&gt; # A tibble: 344 × 8
#&gt; species island bill_length_mm bill_depth_mm flippe…¹ body_…² sex year
#&gt; &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;chr&gt; &lt;dbl&gt;
#&gt; 1 Adelie Torgersen 39.1 18.7 181 3750 male 2007
#&gt; 2 Adelie Torgersen 39.5 17.4 186 3800 fema… 2007
#&gt; 3 Adelie Torgersen 40.3 18 195 3250 fema… 2007
#&gt; 4 Adelie Torgersen NA NA NA NA &lt;NA&gt; 2007
#&gt; 5 Adelie Torgersen 36.7 19.3 193 3450 fema… 2007
#&gt; 6 Adelie Torgersen 39.3 20.6 190 3650 male 2007
#&gt; # … with 338 more rows, and abbreviated variable names ¹flipper_length_mm,
#&gt; # ²body_mass_g</pre>
</div>
<p>In <a href="#chp-iteration" data-type="xref">#chp-iteration</a> well talk about ways of doing this sort of task without repetitive code.</p>
</section>
<section id="reading-part-of-a-sheet" data-type="sect2">
<h2>
Reading part of a sheet</h2>
<p>Since many use Excel spreadsheets for presentation as well as for data storage, its quite common to find cell entries in a spreadsheet that are not part of the data you want to read into R. <a href="#fig-deaths-excel" data-type="xref">#fig-deaths-excel</a> shows such a spreadsheet: in the middle of the sheet is what looks like a data frame but there is extraneous text in cells above and below the data.</p>
<div class="cell">
<div class="cell-output-display">
2022-11-19 00:42:43 +08:00
<figure id="fig-deaths-excel"><p><img src="images/import-spreadsheets-deaths.png" alt="A look at the deaths spreadsheet in Excel. The spreadsheet has four rows on top that contain non-data information; the text 'For the same of consistency in the data layout, which is really a beautiful thing, I will keep making notes up here.' is spread across cells in these top four rows. Then, there is a data frame that includes information on deaths of 10 famous people, including their names, professions, ages, whether they have kids or not, date of birth and death. At the bottom, there are four more rows of non-data information; the text 'This has been really fun, but we're signing off now!' is spread across cells in these bottom four rows." width="1614"/></p>
<figcaption>Spreadsheet called deaths.xlsx in Excel.</figcaption>
</figure>
</div>
</div>
2022-11-19 00:30:32 +08:00
<p>This spreadsheet is one of the example spreadsheets provided in the readxl package. You can use the <code><a href="https://readxl.tidyverse.org/reference/readxl_example.html">readxl_example()</a></code> function to locate the spreadsheet on your system in the directory where the package is installed. This function returns the path to the spreadsheet, which you can use in <code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_excel()</a></code> as usual.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">deaths_path &lt;- readxl_example("deaths.xlsx")
deaths &lt;- read_excel(deaths_path)
#&gt; New names:
#&gt; • `` -&gt; `...2`
#&gt; • `` -&gt; `...3`
#&gt; • `` -&gt; `...4`
#&gt; • `` -&gt; `...5`
#&gt; • `` -&gt; `...6`
deaths
#&gt; # A tibble: 18 × 6
#&gt; `Lots of people` ...2 ...3 ...4 ...5 ...6
#&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt;
#&gt; 1 simply cannot resist writing &lt;NA&gt; &lt;NA&gt; &lt;NA&gt; &lt;NA&gt; some …
#&gt; 2 at the top &lt;NA&gt; of their…
#&gt; 3 or merging &lt;NA&gt; &lt;NA&gt; &lt;NA&gt; cells
#&gt; 4 Name Profession Age Has kids Date of birth Date …
#&gt; 5 David Bowie musician 69 TRUE 17175 42379
#&gt; 6 Carrie Fisher actor 60 TRUE 20749 42731
#&gt; # … with 12 more rows</pre>
</div>
<p>The top three rows and the bottom four rows are not part of the data frame.</p>
<p>We could skip the top three rows with <code>skip</code>. Note that we set <code>skip = 4</code> since the fourth row contains column names, not the data.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel(deaths_path, skip = 4)
#&gt; # A tibble: 14 × 6
#&gt; Name Profession Age `Has kids` `Date of birth` Date of dea…¹
#&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;dttm&gt; &lt;chr&gt;
#&gt; 1 David Bowie musician 69 TRUE 1947-01-08 00:00:00 42379
#&gt; 2 Carrie Fisher actor 60 TRUE 1956-10-21 00:00:00 42731
#&gt; 3 Chuck Berry musician 90 TRUE 1926-10-18 00:00:00 42812
#&gt; 4 Bill Paxton actor 61 TRUE 1955-05-17 00:00:00 42791
#&gt; 5 Prince musician 57 TRUE 1958-06-07 00:00:00 42481
#&gt; 6 Alan Rickman actor 69 FALSE 1946-02-21 00:00:00 42383
#&gt; # … with 8 more rows, and abbreviated variable name ¹​`Date of death`</pre>
</div>
<p>We could also set <code>n_max</code> to omit the extraneous rows at the bottom.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel(deaths_path, skip = 4, n_max = 10)
#&gt; # A tibble: 10 × 6
#&gt; Name Profe…¹ Age Has k…² `Date of birth` `Date of death`
#&gt; &lt;chr&gt; &lt;chr&gt; &lt;dbl&gt; &lt;lgl&gt; &lt;dttm&gt; &lt;dttm&gt;
#&gt; 1 David Bowie musici… 69 TRUE 1947-01-08 00:00:00 2016-01-10 00:00:00
#&gt; 2 Carrie Fisher actor 60 TRUE 1956-10-21 00:00:00 2016-12-27 00:00:00
#&gt; 3 Chuck Berry musici… 90 TRUE 1926-10-18 00:00:00 2017-03-18 00:00:00
#&gt; 4 Bill Paxton actor 61 TRUE 1955-05-17 00:00:00 2017-02-25 00:00:00
#&gt; 5 Prince musici… 57 TRUE 1958-06-07 00:00:00 2016-04-21 00:00:00
#&gt; 6 Alan Rickman actor 69 FALSE 1946-02-21 00:00:00 2016-01-14 00:00:00
#&gt; # … with 4 more rows, and abbreviated variable names ¹Profession,
#&gt; # ²​`Has kids`</pre>
</div>
<p>Another approach is using cell ranges. In Excel, the top left cell is <code>A1</code>. As you move across columns to the right, the cell label moves down the alphabet, i.e. <code>B1</code>, <code>C1</code>, etc. And as you move down a column, the number in the cell label increases, i.e. <code>A2</code>, <code>A3</code>, etc.</p>
<p>The data we want to read in starts in cell <code>A5</code> and ends in cell <code>F15</code>. In spreadsheet notation, this is <code>A5:F15</code>.</p>
<ul><li>
<p>Supply this information to the <code>range</code> argument:</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel(deaths_path, range = "A5:F15")</pre>
</div>
</li>
<li>
<p>Specify rows:</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel(deaths_path, range = cell_rows(c(5, 15)))</pre>
</div>
</li>
<li>
<p>Specify cells that mark the top-left and bottom-right corners of the data the top-left corner, <code>A5</code>, translates to <code>c(5, 1)</code> (5th row down, 1st column) and the bottom-right corner, <code>F15</code>, translates to <code>c(15, 6)</code>:</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel(deaths_path, range = cell_limits(c(5, 1), c(15, 6)))</pre>
</div>
</li>
</ul><p>If you have control over the sheet, an even better way is to create a “named range”. This is useful within Excel because named ranges help repeat formulas easier to create and they have some useful properties for creating dynamic charts and graphs as well. Even if youre not working in Excel, named ranges can be useful for identifying which cells to read into R. In the example above, the table were reading in is named <code>Table1</code>, so we can read it in with the following.</p>
<p><strong>TO DO:</strong> Add this once reading in named ranges are implemented in readxl.</p>
</section>
<section id="data-types" data-type="sect2">
<h2>
Data types</h2>
<p>In CSV files, all values are strings. This is not particularly true to the data, but it is simple: everything is a string.</p>
<p>The underlying data in Excel spreadsheets is more complex. A cell can be one of five things:</p>
<ul><li><p>A logical, like TRUE / FALSE</p></li>
<li><p>A number, like “10” or “10.5”</p></li>
<li><p>A date, which can also include time like “11/1/21” or “11/1/21 3:00 PM”</p></li>
<li><p>A string, like “ten”</p></li>
<li><p>A currency, which allows numeric values in a limited range and four decimal digits of fixed precision</p></li>
</ul><p>When working with spreadsheet data, its important to keep in mind that how the underlying data is stored can be very different than what you see in the cell. For example, Excel has no notion of an integer. All numbers are stored as floating points, but you can choose to display the data with a customizable number of decimal points. Similarly, dates are actually stored as numbers, specifically the number of seconds since January 1, 1970. You can customize how you display the date by applying formatting in Excel. Confusingly, its also possible to have something that looks like a number but is actually a string (e.g. type <code>'10</code> into a cell in Excel).</p>
<p>These differences between how the underlying data are stored vs. how theyre displayed can cause surprises when the data are loaded into R. By default readxl will guess the data type in a given column. A recommended workflow is to let readxl guess the column types, confirm that youre happy with the guessed column types, and if not, go back and re-import specifying <code>col_types</code> as shown in <a href="#sec-reading-spreadsheets" data-type="xref">#sec-reading-spreadsheets</a>.</p>
<p>Another challenge is when you have a column in your Excel spreadsheet that has a mix of these types, e.g. some cells are numeric, others text, others dates. When importing the data into R readxl has to make some decisions. In these cases you can set the type for this column to <code>"list"</code>, which will load the column as a list of length 1 vectors, where the type of each element of the vector is guessed.</p>
</section>
<section id="data-not-in-cell-values" data-type="sect2">
<h2>
Data not in cell values</h2>
<p><strong>tidyxl</strong> is useful for importing non-tabular data from Excel files into R. For example, tidyxl doesnt coerce a pivot table into a data frame. See <a href="https://nacnudus.github.io/spreadsheet-munging-strategies/" class="uri">https://nacnudus.github.io/spreadsheet-munging-strategies/</a> for more on strategies for working with non-tabular data from Excel.</p>
</section>
<section id="writing-to-excel" data-type="sect2">
<h2>
Writing to Excel</h2>
<p>Lets create a small data frame that we can then write out. Note that <code>item</code> is a factor and <code>quantity</code> is an integer.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">bake_sale &lt;- tibble(
item = factor(c("brownie", "cupcake", "cookie")),
quantity = c(10, 5, 8)
)
bake_sale
#&gt; # A tibble: 3 × 2
#&gt; item quantity
#&gt; &lt;fct&gt; &lt;dbl&gt;
#&gt; 1 brownie 10
#&gt; 2 cupcake 5
#&gt; 3 cookie 8</pre>
</div>
2022-11-19 00:30:32 +08:00
<p>You can write data back to disk as an Excel file using the <code><a href="https://docs.ropensci.org/writexl/reference/write_xlsx.html">write_xlsx()</a></code> from the <strong>writexl</strong> package.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">library(writexl)
write_xlsx(bake_sale, path = "data/bake-sale.xlsx")</pre>
</div>
<p><a href="#fig-bake-sale-excel" data-type="xref">#fig-bake-sale-excel</a> shows what the data looks like in Excel. Note that column names are included and bolded. These can be turned off by setting <code>col_names</code> and <code>format_headers</code> arguments to <code>FALSE</code>.</p>
<div class="cell">
<div class="cell-output-display">
2022-11-19 00:42:43 +08:00
<figure id="fig-bake-sale-excel"><p><img src="images/import-spreadsheets-bake-sale.png" alt="Bake sale data frame created earlier in Excel." width="917"/></p>
<figcaption>Spreadsheet called bake_sale.xlsx in Excel.</figcaption>
</figure>
</div>
</div>
<p>Just like reading from a CSV, information on data type is lost when we read the data back in. This makes Excel files unreliable for caching interim results as well. For alternatives, see <a href="#sec-writing-to-a-file" data-type="xref">#sec-writing-to-a-file</a>.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">read_excel("data/bake-sale.xlsx")
#&gt; # A tibble: 3 × 2
#&gt; item quantity
#&gt; &lt;chr&gt; &lt;dbl&gt;
#&gt; 1 brownie 10
#&gt; 2 cupcake 5
#&gt; 3 cookie 8</pre>
</div>
</section>
<section id="formatted-output" data-type="sect2">
<h2>
Formatted output</h2>
<p>The readxl package is a light-weight solution for writing a simple Excel spreadsheet, but if youre interested in additional features like writing to sheets within a spreadsheet and styling, you will want to use the <strong>openxlsx</strong> package. Note that this package is not part of the tidyverse so the functions and workflows may feel unfamiliar. For example, function names are camelCase, multiple functions cant be composed in pipelines, and arguments are in a different order than they tend to be in the tidyverse. However, this is ok. As your R learning and usage expands outside of this book you will encounter lots of different styles used in various R packages that you might need to use to accomplish specific goals in R. A good way of familiarizing yourself with the coding style used in a new package is to run the examples provided in function documentation to get a feel for the syntax and the output formats as well as reading any vignettes that might come with the package.</p>
<p>Below we show how to write a spreadsheet with three sheets, one for each species of penguins in the <code>penguins</code> data frame.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">library(openxlsx)
library(palmerpenguins)
# Create a workbook (spreadsheet)
penguins_species &lt;- createWorkbook()
# Add three sheets to the spreadsheet
addWorksheet(penguins_species, sheetName = "Adelie")
addWorksheet(penguins_species, sheetName = "Gentoo")
addWorksheet(penguins_species, sheetName = "Chinstrap")
# Write data to each sheet
writeDataTable(
penguins_species,
sheet = "Adelie",
x = penguins |&gt; filter(species == "Adelie")
)
writeDataTable(
penguins_species,
sheet = "Gentoo",
x = penguins |&gt; filter(species == "Gentoo")
)
writeDataTable(
penguins_species,
sheet = "Chinstrap",
x = penguins |&gt; filter(species == "Chinstrap")
)</pre>
</div>
<p>This creates a workbook object:</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">penguins_species
#&gt; A Workbook object.
#&gt;
#&gt; Worksheets:
#&gt; Sheet 1: "Adelie"
#&gt;
#&gt;
#&gt; Sheet 2: "Gentoo"
#&gt;
#&gt;
#&gt; Sheet 3: "Chinstrap"
#&gt;
#&gt;
#&gt;
#&gt; Worksheet write order: 1, 2, 3
#&gt; Active Sheet 1: "Adelie"
#&gt; Position: 1</pre>
</div>
2022-11-19 00:30:32 +08:00
<p>And we can write this to this with <code><a href="https://rdrr.io/pkg/openxlsx/man/saveWorkbook.html">saveWorkbook()</a></code>.</p>
<div class="cell">
2022-11-19 01:26:25 +08:00
<pre data-type="programlisting" data-code-language="r">saveWorkbook(penguins_species, "data/penguins-species.xlsx")</pre>
</div>
<p>The resulting spreadsheet is shown in <a href="#fig-penguins-species" data-type="xref">#fig-penguins-species</a>. By default, openxlsx formats the data as an Excel table.</p>
<div class="cell">
<div class="cell-output-display">
2022-11-19 00:42:43 +08:00
<figure id="fig-penguins-species"><p><img src="images/import-spreadsheets-penguins-species.png" alt="A look at the penguins spreadsheet in Excel. The spreadsheet contains has three sheets: Torgersen Island, Biscoe Island, and Dream Island." width="1106"/></p>
<figcaption>Spreadsheet called penguins.xlsx in Excel.</figcaption>
</figure>
</div>
</div>
<p>See <a href="https://ycphs.github.io/openxlsx/articles/Formatting.html" class="uri">https://ycphs.github.io/openxlsx/articles/Formatting.html</a> for an extensive discussion on further formatting functionality for data written from R to Excel with openxlsx.</p>
</section>
<section id="exercises" data-type="sect2">
<h2>
Exercises</h2>
2022-11-19 00:30:32 +08:00
<ol type="1"><li>Recreate the <code>bake_sale</code> data frame, write it out to an Excel file using the <code><a href="https://rdrr.io/pkg/openxlsx/man/write.xlsx.html">write.xlsx()</a></code> function from the openxlsx package.</li>
<li>What happens if you try to read in a file with <code>.xlsx</code> extension with <code><a href="https://readxl.tidyverse.org/reference/read_excel.html">read_xls()</a></code>?</li>
</ol><!--# Need moar exercises --></section>
</section>
<section id="google-sheets" data-type="sect1">
<h1>
Google Sheets</h1>
<!--# TO DO: Write section. -->
<section id="prerequisites-1" data-type="sect2">
<h2>
Prerequisites</h2>
<p>TO DO:</p>
<ul><li>use googlesheets4</li>
<li>why 4?</li>
</ul></section>
<section id="getting-started-1" data-type="sect2">
<h2>
Getting started</h2>
<p>TO DO:</p>
<ul><li>reading from public sheet with <code>read_sheet()</code> and <code>read_range()</code>
</li>
</ul></section>
<section id="authentication" data-type="sect2">
<h2>
Authentication</h2>
</section>
<section id="read-sheets" data-type="sect2">
<h2>
Read sheets</h2>
</section>
<section id="write-sheets" data-type="sect2">
<h2>
Write sheets</h2>
</section>
<section id="exercises-1" data-type="sect2">
<h2>
Exercises</h2>
</section>
</section>
</section>