Add section on googlesheets (#1182)

* Show all rows

* Can't read named ranges

* Add exercises

* Move to screenshots

* Some structure

* Read in google sheets

* Add more googlesheets content

* Add summary

* A bit more polishing

* Update status

spreadsheets.qmd

@@ -4,7 +4,7 @@
 #| results: "asis"
 #| echo: false
 source("_common.R")
-status("drafting")
+status("polishing")
 ```
 
 ## Introduction
@@ -21,7 +21,7 @@ The best practices presented in this paper will save you much headache down the
 
 ### Prerequisites
 
-In this chapter, you'll learn how to load data from Excel spreadsheets in R with the **readxl** package.
+In this section, you'll learn how to load data from Excel spreadsheets in R with the **readxl** package.
 This package is non-core tidyverse, so you need to load it explicitly but it is installed automatically when you install the tidyverse package.
 
 ```{r}
@@ -31,9 +31,9 @@ library(readxl)
 library(tidyverse)
 ```
 
-**xlsx** and **XLConnect** can be used for reading data from and writing data to Excel spreadsheets.
+**xlsx** and **XLConnect** can also be used for reading data from and writing data to Excel spreadsheets.
 However, these two packages require Java installed on your machine and the rJava package.
-Due to potential challenges with installation, we recommend using alternative packages we've introduced in this chapter.
+Due to potential challenges with installation, we recommend using alternative packages we're introducing in this chapter.
 
 ### Getting started
 
@@ -60,7 +60,7 @@ For the rest of the chapter we will focus on using `read_excel()`.
 #|   information on 6 students, their ID, full name, favourite food, meal plan, 
 #|   and age.
 
-knitr::include_graphics("images/import-spreadsheets-students.png")
+knitr::include_graphics("screenshots/import-spreadsheets-students.png")
 ```
 
 The first argument to `read_excel()` is the path to the file to read.
@@ -81,6 +81,15 @@ However there are a few things we might want to address in this dataset:
 1.  The column names are all over the place.
     You can provide column names that follow a consistent format; we recommend `snake_case` using the `col_names` argument.
 
+    ```{r}
+    #| include: false
+
+    options(
+      dplyr.print_min = 7,
+      dplyr.print_max = 7
+    )
+    ```
+
     ```{r}
     read_excel(
       "data/students.xlsx",
@@ -88,8 +97,17 @@ However there are a few things we might want to address in this dataset:
     )
     ```
 
+    ```{r}
+    #| include: false
+
+    options(
+      dplyr.print_min = 6,
+      dplyr.print_max = 6
+    )
+    ```
+
     Unfortunately, this didn't quite do the trick.
-    You now have the variable names we want, but what was previously the header row now shows up as the first observation in the data.
+    We now have the variable names we want, but what was previously the header row now shows up as the first observation in the data.
     You can explicitly skip that row using the `skip` argument.
 
     ```{r}
@@ -155,8 +173,13 @@ Data science is an iterative process.
 There is no way to know exactly what the data will look like until you load it and take a look at it.
 Well, there is one way, actually.
 You can open the file in Excel and take a peek.
-That might be tempting, but it's strongly not recommended.
-<!--# TO DO: Provide reason why it's not recommended. --> Instead, you should not be afraid of doing what we did here: load the data, take a peek, make adjustments to your code, load it again, and repeat until you're happy with the result.
+That might be tempting, but it's strongly not recommended as Excel can modify certain types of data upon launch with no way to track this change.[^spreadsheets-1]
+Instead, you should not be afraid of doing what we did here: load the data, take a peek, make adjustments to your code, load it again, and repeat until you're happy with the result.
+
+[^spreadsheets-1]: Most notoriously, Excel can convert alphanumeric symbols for genes into dates.
+    For example, SEPT4 (which stands for septin 4) gets converted to 4-Sep in Excel.
+    This has negatively affected researchers' work so much that scientists have decided to rename human genes to stop Excel from misreading them as dates.
+    You can read more about this at <https://www.nature.com/articles/d41586-021-02211-4>.
 
 ### Reading individual sheets
 
@@ -174,7 +197,7 @@ Each sheet contains information on penguins from a different island where data w
 #|   A look at the penguins spreadsheet in Excel. The spreadsheet contains has 
 #|   three sheets: Torgersen Island, Biscoe Island, and Dream Island.
 
-knitr::include_graphics("images/import-spreadsheets-penguins-islands.png")
+knitr::include_graphics("screenshots/import-spreadsheets-penguins-islands.png")
 ```
 
 You can read a single sheet from a spreadsheet with the `sheet` argument in `read_excel()`.
@@ -245,7 +268,7 @@ Since many use Excel spreadsheets for presentation as well as for data storage,
 #|   rows of non-data information; the text 'This has been really fun, but 
 #|   we're signing off now!' is spread across cells in these bottom four rows.
 
-knitr::include_graphics("images/import-spreadsheets-deaths.png")
+knitr::include_graphics("screenshots/import-spreadsheets-deaths.png")
 ```
 
 This spreadsheet is one of the example spreadsheets provided in the readxl package.
@@ -307,13 +330,6 @@ In spreadsheet notation, this is `A5:F15`.
     read_excel(deaths_path, range = cell_limits(c(5, 1), c(15, 6)))
     ```
 
-If you have control over the sheet, an even better way is to create a "named range".
-This is useful within Excel because named ranges help repeat formulas easier to create and they have some useful properties for creating dynamic charts and graphs as well.
-Even if you're not working in Excel, named ranges can be useful for identifying which cells to read into R.
-In the example above, the table we're reading in is named `Table1`, so we can read it in with the following.
-
-**TO DO:** Add this once reading in named ranges are implemented in readxl.
-
 ### Data types
 
 In CSV files, all values are strings.
@@ -388,7 +404,7 @@ These can be turned off by setting `col_names` and `format_headers` arguments to
 #| fig-alt: >
 #|   Bake sale data frame created earlier in Excel.
 
-knitr::include_graphics("images/import-spreadsheets-bake-sale.png")
+knitr::include_graphics("screenshots/import-spreadsheets-bake-sale.png")
 ```
 
 Just like reading from a CSV, information on data type is lost when we read the data back in.
@@ -468,39 +484,264 @@ By default, openxlsx formats the data as an Excel table.
 #|   A look at the penguins spreadsheet in Excel. The spreadsheet contains has 
 #|   three sheets: Torgersen Island, Biscoe Island, and Dream Island.
 
-knitr::include_graphics("images/import-spreadsheets-penguins-species.png")
+knitr::include_graphics("screenshots/import-spreadsheets-penguins-species.png")
 ```
 
 See <https://ycphs.github.io/openxlsx/articles/Formatting.html> for an extensive discussion on further formatting functionality for data written from R to Excel with openxlsx.
 
 ### Exercises
 
-1.  Recreate the `bake_sale` data frame, write it out to an Excel file using the `write.xlsx()` function from the openxlsx package.
-2.  What happens if you try to read in a file with `.xlsx` extension with `read_xls()`?
+1.  In an Excel file, create the following dataset and save it as `survey.xlsx`.
+    Alternatively, you can download it as an Excel file from [here](https://docs.google.com/spreadsheets/d/1yc5gL-a2OOBr8M7B3IsDNX5uR17vBHOyWZq6xSTG2G8/edit?usp=sharing).
 
-<!--# Need moar exercises -->
+    ```{r}
+    #| echo: false
+    #| fig-alt: >
+    #|   A spreadsheet with 3 columns (group, subgroup, and id) and 12 rows. The group column has two values: 1 (spanning 7 merged rows) and 2 (spanning 5 merged rows). The subgroup column has four values: A (spanning 3 merged rows), B (spanning 4 merged rows), A (spanning 2 merged rows), and B (spanning 3 merged rows). The id column has twelve values, number 1 through 12.
+
+    knitr::include_graphics("screenshots/import-spreadsheets-survey.png")
+    ```
+
+    Then, read it into R, with `survey_id` as a character variable and `n_pets` as a numerical variable.
+    Hint: You will need to convert "none" to 0.
+
+    ```{r}
+    #| echo: false
+
+    read_excel("data/survey.xlsx", na = c("", "N/A")) |>
+      mutate(
+        n_pets = case_when(
+          n_pets == "none" ~ "0",
+          n_pets == "two"  ~ "2",
+          TRUE             ~ n_pets
+        ),
+        n_pets = as.numeric(n_pets)
+      )
+    ```
+
+2.  In another Excel file, create the following dataset and save it as `roster.xlsx`.
+    Alternatively, you can download it as an Excel file from [here](https://docs.google.com/spreadsheets/d/1LgZ0Bkg9d_NK8uTdP2uHXm07kAlwx8-Ictf8NocebIE/edit?usp=sharing).
+
+    ```{r}
+    #| echo: false
+    #| fig-alt: >
+    #|   A spreadsheet with 3 columns (group, subgroup, and id) and 12 rows. The group column has two values: 1 (spanning 7 merged rows) and 2 (spanning 5 merged rows). The subgroup column has four values: A (spanning 3 merged rows), B (spanning 4 merged rows), A (spanning 2 merged rows), and B (spanning 3 merged rows). The id column has twelve values, number 1 through 12.
+
+    knitr::include_graphics("screenshots/import-spreadsheets-roster.png")
+    ```
+
+    Then, read it into R.
+    The resulting data frame should be called `roster` and should look like the following.
+
+    ```{r}
+    #| echo: false
+    #| message: false
+
+    read_excel("data/roster.xlsx") |>
+      fill(group, subgroup) |>
+      print(n = 12)
+    ```
+
+3.  In a new Excel file, create the following dataset and save it as `sales.xlsx`.
+    Alternatively, you can download it as an Excel file from [here](https://docs.google.com/spreadsheets/d/1oCqdXUNO8JR3Pca8fHfiz_WXWxMuZAp3YiYFaKze5V0/edit?usp=sharing).
+
+    ```{r}
+    #| echo: false
+    #| fig-alt: >
+    #|   A spreadsheet with 2 columns and 13 rows. The first two rows have text containing information about the sheet. Row 1 says "This file contains information on sales". Row 2 says "Data are organized by brand name, and for each brand, we have the ID number for the item sold, and how many are sold.". Then there are two empty rows, and then 9 rows of data.
+
+    knitr::include_graphics("screenshots/import-spreadsheets-sales.png")
+    ```
+
+    a\.
+    Read `sales.xlsx` in and save as `sales`.
+    The data frame should look like the following, with `id` and `n` as column names and with 9 rows.
+
+    ```{r}
+    #| echo: false
+    #| message: false
+
+    read_excel("data/sales.xlsx", skip = 3, col_names = c("id", "n")) |>
+      print(n = 9)
+    ```
+
+    b\.
+    Modify `sales` further to get it into the following tidy format with three columns (`brand`, `id`, and `n`) and 7 rows of data.
+    Note that `id` and `n` are numeric, `brand` is a character variable.
+
+    ```{r}
+    #| echo: false
+    #| message: false
+
+    read_excel("data/sales.xlsx", skip = 3, col_names = c("id", "n")) |>
+      mutate(brand = if_else(str_detect(id, "Brand"), id, NA)) |>
+      fill(brand) |>
+      filter(n != "n") |>
+      relocate(brand) |>
+      mutate(
+        id = as.numeric(id),
+        n = as.numeric(n)
+      ) |>
+      print(n = 7)
+    ```
+
+4.  Recreate the `bake_sale` data frame, write it out to an Excel file using the `write.xlsx()` function from the openxlsx package.
+
+5.  What happens if you try to read in a file with `.xlsx` extension with `read_xls()`?
 
 ## Google Sheets
 
-<!--# TO DO: Write section. -->
-
 ### Prerequisites
 
-TO DO:
+This section will also focus on spreadsheets, but this time you'll be loading data from a Google Sheet with the **googlesheets4** package.
+This package is non-core tidyverse as well, you need to load it explicitly.
 
--   use googlesheets4
--   why 4?
+```{r}
+library(googlesheets4)
+library(tidyverse)
+```
+
+A quick note about the name of the package: googlesheets4 uses v4 of the [Sheets API v4](https://developers.google.com/sheets/api/) to provide an R interface to Google Sheets, hence the name.
 
 ### Getting started
 
-TO DO:
+The main function of the googlesheets4 package is `read_sheet()`, which reads a Google Sheet from a URL or a file id.
+This function also goes by the name `range_read()`.
 
--   reading from public sheet with `read_sheet()` and `read_range()`
+You can also create a brand new sheet with `gs4_create()` or write to an existing sheet with `sheet_write()` and friends.
 
-### Authentication
+In this section we'll work with the same datasets as the ones in the Excel section to highlight similarities and differences between workflows for reading data from Excel and Google Sheets.
+readxl and googlesheets4 packages are both designed to mimic the functionality of the readr package, which provides the `read_csv()` function you've seen in @sec-data-import. Therefore, many of the tasks can be accomplished with simply swapping out `read_excel()` for `read_sheet()`.
+However you'll also see that Excel and Google Sheets don't behave in exactly the same way, therefore other tasks may require further updates to the function calls.
 
 ### Read sheets
 
+@fig-students-googlesheets shows what the spreadsheet we're going to read into R looks like in Google Sheets.
+This is the same dataset as in @fig-students-excel, except it's stored in a Google Sheet instead of Excel.
+
+```{r}
+#| label: fig-students-googlesheets
+#| echo: false
+#| fig-cap: >
+#|   Google Sheet called students in a browser window.
+#| fig-alt: >
+#|   A look at the students spreadsheet in Google Sheets. The spreadsheet contains 
+#|   information on 6 students, their ID, full name, favourite food, meal plan, 
+#|   and age.
+
+knitr::include_graphics("screenshots/import-googlesheets-students.png")
+```
+
+The first argument to `read_sheet()` is the URL of the file to read.
+You can also access this file via <https://pos.it/r4ds-students>, however note that at the time of writing this book you can't read a sheet directly from a short link.
+
+```{r}
+#| include: false
+
+gs4_deauth()
+```
+
+```{r}
+students <- read_sheet("https://docs.google.com/spreadsheets/d/1V1nPp1tzOuutXFLb3G9Eyxi3qxeEhnOXUzL5_BcCQ0w/edit?usp=sharing")
+```
+
+`read_sheet()` will read the file in as a tibble.
+
+```{r}
+students
+```
+
+Just like we did with `read_excel()`, we can supply column names, NA strings, and column types to `read_sheet()`.
+
+```{r}
+students <- read_sheet(
+  "https://docs.google.com/spreadsheets/d/1V1nPp1tzOuutXFLb3G9Eyxi3qxeEhnOXUzL5_BcCQ0w/edit?usp=sharing",
+  col_names = c("student_id", "full_name", "favourite_food", "meal_plan", "age"),
+  skip = 1,
+  na = c("", "N/A"),
+  col_types = c("dcccc")
+) |>
+  mutate(
+    age = if_else(age == "five", "5", age),
+    age = parse_number(age)
+  )
+
+students
+```
+
+Note that we defined column types a bit differently here, using short codes.
+For example, "dcccc" stands for "double, character, character, character, character".
+
+It's also possible to read individual sheets from Google Sheets as well.
+Let's read the penguins Google Sheet at <https://pos.it/r4ds-penguins>, and specifically the "Torgersen Island" sheet in it.
+
+```{r}
+read_sheet("https://docs.google.com/spreadsheets/d/1aFu8lnD_g0yjF5O-K6SFgSEWiHPpgvFCF0NY9D6LXnY/edit?usp=sharing", sheet = "Torgersen Island")
+```
+
+You can obtain a list of all sheets within a Google Sheet with `sheet_names()`:
+
+```{r}
+sheet_names("https://docs.google.com/spreadsheets/d/1aFu8lnD_g0yjF5O-K6SFgSEWiHPpgvFCF0NY9D6LXnY/edit?usp=sharing")
+```
+
+Finally, just like with `read_excel()`, we can read in a portion of a Google Sheet by defining a `range` in `read_sheet()`.
+Note that we're also using the `gs4_example()` function below to locate an example Google Sheet that comes with the googlesheets4 package.
+
+```{r}
+deaths_url <- gs4_example("deaths")
+deaths <- read_sheet(deaths_url, range = "A5:F15")
+deaths
+```
+
 ### Write sheets
 
+You can write from R to Google Sheets with `write_sheet()`:
+
+```{r}
+#| eval: false
+
+write_sheet(bake_sale, ss = "bake-sale")
+```
+
+If you'd like to write your data to a specific (work)sheet inside a Google Sheet, you can specify that with the `sheet` argument as well.
+
+```{r}
+#| eval: false
+
+write_sheet(bake_sale, ss = "bake-sale", sheet = "Sales")
+```
+
+### Authentication
+
+While you can read from a public Google Sheet without authenticating with your Google account, reading a private sheet or writing to a sheet requires authentication so that googlesheets4 can view and manage *your* Google Sheets.
+
+You can do this with `gs4_auth()`, which will open a browser for authentication and authorization, or by specifying an email address in `gs4_auth()`, e.g. `gs4_auth("mine@example.com")`, which will force the use of a token associated with a specific email.
+For further authentication details, we recommend reading the documentation googlesheets4 auth vignette: <https://googlesheets4.tidyverse.org/articles/auth.html>.
+
 ### Exercises
+
+1.  Read the `students` dataset from earlier in the chapter from Excel and also from Google Sheets, with no additional arguments supplied to the `read_excel()` and `read_sheet()` functions. Are the resulting data frames in R exactly the same? If not, how are they different?
+
+2.  Read the Google Sheet titled survey from <https://pos.it/r4ds-survey>, with `survey_id` as a character variable and `n_pets` as a numerical variable.
+
+3.  Read the Google Sheet titled roster from <https://pos.it/r4ds-roster>. The resulting data frame should be called `roster` and should look like the following.
+
+    ```{r}
+    #| echo: false
+    #| message: false
+
+    read_sheet("https://docs.google.com/spreadsheets/d/1LgZ0Bkg9d_NK8uTdP2uHXm07kAlwx8-Ictf8NocebIE/edit#gid=0") |>
+      fill(group, subgroup) |>
+      print(n = 12)
+    ```
+
+## Summary
+
+In this chapter you learned how to read data into R from spreadsheets: from Microsoft Excel with `read_excel()` from the readxl package and from Google Sheets with `read_sheet()` from the googlesheets4 package.
+These functions work very similarly to each other and have similar arguments for specifying column names, NA strings, rows to skip on top of the file you're reading in, etc.
+Additionally, both functions make it possible to read a single sheet from a spreadsheet as well.
+
+On the other hand, writing to an Excel file requires a different package and function (`writexl::write_xlsx()`) while you can write to a Google Sheet with the googlesheets4 package, with `write_sheet()`.
+
+In the next chapter, you'll learn about a different data source and how to read data from that source into R: databases.