This will build on much of what you've learned in @sec-data-import but we will also discuss additional considerations and complexities when working with data from spreadsheets.
If you or your collaborators are using spreadsheets for organizing data, we strongly recommend reading the paper "Data Organization in Spreadsheets" by Karl Broman and Kara Woo: <https://doi.org/10.1080/00031305.2017.1375989>.
The best practices presented in this paper will save you much headache down the line when you import the data from a spreadsheet into R to analyse and visualize.
Most of readxl's functions allow you to load Excel spreadsheets into R:
- `read_xls()` reads Excel files with `xls` format.
- `read_xlsx()` read Excel files with `xlsx` format.
- `read_excel()` can read files with both `xls` and `xlsx` format. It guesses the file type based on the input.
These functions all have similar syntax just like other functions we have previously introduced for reading other types of files, e.g. `read_csv()`, `read_table()`, etc.
For the rest of the chapter we will focus on using `read_excel()`.
2. In the `favourite_food` column, one of the observations is `N/A`, which stands for "not available" but it's currently not recognized as an `NA` (note the contrast between this `N/A` and the age of the fourth student in the list).
You can specify which character strings should be recognized as `NA`s with the `na` argument.
By default, only `""` (empty string, or, in the case of reading from a spreadsheet, an empty cell) is recognized as an `NA`.
3. One other remaining issue is that `age` is read in as a character variable, but it really should be numeric.
Just like with `read_csv()` and friends for reading data from flat files, you can supply a `col_types` argument to `read_excel()` and specify the column types for the variables you read in.
The syntax is a bit different, though.
Your options are `"skip"`, `"guess"`, `"logical"`, `"numeric"`, `"date"`, `"text"` or `"list"`.
That might be tempting, but it's strongly not recommended as Excel can modify certain types of data upon launch with no way to track this change.[^spreadsheets-1]
Instead, you should not be afraid of doing what we did here: load the data, take a peek, make adjustments to your code, load it again, and repeat until you're happy with the result.
[^spreadsheets-1]: Most notoriously, Excel can convert alphanumeric symbols for genes into dates.
For example, SEPT4 (which stands for septin 4) gets converted to 4-Sep in Excel.
This has negatively affected researchers' work so much that scientists have decided to rename human genes to stop Excel from misreading them as dates.
You can read more about this at <https://www.nature.com/articles/d41586-021-02211-4>.
Since many use Excel spreadsheets for presentation as well as for data storage, it's quite common to find cell entries in a spreadsheet that are not part of the data you want to read into R.
@fig-deaths-excel shows such a spreadsheet: in the middle of the sheet is what looks like a data frame but there is extraneous text in cells above and below the data.
```{r}
#| label: fig-deaths-excel
#| echo: false
#| fig-cap: >
#| Spreadsheet called deaths.xlsx in Excel.
#| fig-alt: >
#| A look at the deaths spreadsheet in Excel. The spreadsheet has four rows
#| on top that contain non-data information; the text 'For the same of
#| consistency in the data layout, which is really a beautiful thing, I will
#| keep making notes up here.' is spread across cells in these top four rows.
#| Then, there is a data frame that includes information on deaths of 10
#| famous people, including their names, professions, ages, whether they have
#| kids or not, date of birth and death. At the bottom, there are four more
#| rows of non-data information; the text 'This has been really fun, but
#| we're signing off now!' is spread across cells in these bottom four rows.
read_excel(deaths_path, range = cell_rows(c(5, 15)))
```
- Specify cells that mark the top-left and bottom-right corners of the data -- the top-left corner, `A5`, translates to `c(5, 1)` (5th row down, 1st column) and the bottom-right corner, `F15`, translates to `c(15, 6)`:
read_excel(deaths_path, range = cell_limits(c(5, 1), c(15, 6)))
```
### Data types
In CSV files, all values are strings.
This is not particularly true to the data, but it is simple: everything is a string.
The underlying data in Excel spreadsheets is more complex.
A cell can be one of five things:
- A logical, like TRUE / FALSE
- A number, like "10" or "10.5"
- A date, which can also include time like "11/1/21" or "11/1/21 3:00 PM"
- A string, like "ten"
- A currency, which allows numeric values in a limited range and four decimal digits of fixed precision
When working with spreadsheet data, it's important to keep in mind that how the underlying data is stored can be very different than what you see in the cell.
For example, Excel has no notion of an integer.
All numbers are stored as floating points, but you can choose to display the data with a customizable number of decimal points.
Similarly, dates are actually stored as numbers, specifically the number of seconds since January 1, 1970.
You can customize how you display the date by applying formatting in Excel.
Confusingly, it's also possible to have something that looks like a number but is actually a string (e.g. type `'10` into a cell in Excel).
These differences between how the underlying data are stored vs. how they're displayed can cause surprises when the data are loaded into R.
By default readxl will guess the data type in a given column.
A recommended workflow is to let readxl guess the column types, confirm that you're happy with the guessed column types, and if not, go back and re-import specifying `col_types` as shown in @sec-reading-spreadsheets.
Another challenge is when you have a column in your Excel spreadsheet that has a mix of these types, e.g. some cells are numeric, others text, others dates.
When importing the data into R readxl has to make some decisions.
In these cases you can set the type for this column to `"list"`, which will load the column as a list of length 1 vectors, where the type of each element of the vector is guessed.
### Data not in cell values
**tidyxl** is useful for importing non-tabular data from Excel files into R.
For example, tidyxl doesn't coerce a pivot table into a data frame.
See <https://nacnudus.github.io/spreadsheet-munging-strategies/> for more on strategies for working with non-tabular data from Excel.
### Writing to Excel
Let's create a small data frame that we can then write out.
Note that `item` is a factor and `quantity` is an integer.
```{r}
bake_sale <- tibble(
item = factor(c("brownie", "cupcake", "cookie")),
quantity = c(10, 5, 8)
)
bake_sale
```
You can write data back to disk as an Excel file using the `write_xlsx()` from the **writexl** package.
The readxl package is a light-weight solution for writing a simple Excel spreadsheet, but if you're interested in additional features like writing to sheets within a spreadsheet and styling, you will want to use the **openxlsx** package.
Note that this package is not part of the tidyverse so the functions and workflows may feel unfamiliar.
For example, function names are camelCase, multiple functions can't be composed in pipelines, and arguments are in a different order than they tend to be in the tidyverse.
However, this is ok.
As your R learning and usage expands outside of this book you will encounter lots of different styles used in various R packages that you might need to use to accomplish specific goals in R.
A good way of familiarizing yourself with the coding style used in a new package is to run the examples provided in function documentation to get a feel for the syntax and the output formats as well as reading any vignettes that might come with the package.
Below we show how to write a spreadsheet with three sheets, one for each species of penguins in the `penguins` data frame.
See <https://ycphs.github.io/openxlsx/articles/Formatting.html> for an extensive discussion on further formatting functionality for data written from R to Excel with openxlsx.
1. In an Excel file, create the following dataset and save it as `survey.xlsx`.
Alternatively, you can download it as an Excel file from [here](https://docs.google.com/spreadsheets/d/1yc5gL-a2OOBr8M7B3IsDNX5uR17vBHOyWZq6xSTG2G8/edit?usp=sharing).
#| A spreadsheet with 3 columns (group, subgroup, and id) and 12 rows. The group column has two values: 1 (spanning 7 merged rows) and 2 (spanning 5 merged rows). The subgroup column has four values: A (spanning 3 merged rows), B (spanning 4 merged rows), A (spanning 2 merged rows), and B (spanning 3 merged rows). The id column has twelve values, number 1 through 12.
Then, read it into R, with `survey_id` as a character variable and `n_pets` as a numerical variable.
Hint: You will need to convert "none" to 0.
```{r}
#| echo: false
read_excel("data/survey.xlsx", na = c("", "N/A")) |>
mutate(
n_pets = case_when(
n_pets == "none" ~ "0",
n_pets == "two" ~ "2",
TRUE ~ n_pets
),
n_pets = as.numeric(n_pets)
)
```
2. In another Excel file, create the following dataset and save it as `roster.xlsx`.
Alternatively, you can download it as an Excel file from [here](https://docs.google.com/spreadsheets/d/1LgZ0Bkg9d_NK8uTdP2uHXm07kAlwx8-Ictf8NocebIE/edit?usp=sharing).
```{r}
#| echo: false
#| fig-alt: >
#| A spreadsheet with 3 columns (group, subgroup, and id) and 12 rows. The group column has two values: 1 (spanning 7 merged rows) and 2 (spanning 5 merged rows). The subgroup column has four values: A (spanning 3 merged rows), B (spanning 4 merged rows), A (spanning 2 merged rows), and B (spanning 3 merged rows). The id column has twelve values, number 1 through 12.
The resulting data frame should be called `roster` and should look like the following.
```{r}
#| echo: false
#| message: false
read_excel("data/roster.xlsx") |>
fill(group, subgroup) |>
print(n = 12)
```
3. In a new Excel file, create the following dataset and save it as `sales.xlsx`.
Alternatively, you can download it as an Excel file from [here](https://docs.google.com/spreadsheets/d/1oCqdXUNO8JR3Pca8fHfiz_WXWxMuZAp3YiYFaKze5V0/edit?usp=sharing).
```{r}
#| echo: false
#| fig-alt: >
#| A spreadsheet with 2 columns and 13 rows. The first two rows have text containing information about the sheet. Row 1 says "This file contains information on sales". Row 2 says "Data are organized by brand name, and for each brand, we have the ID number for the item sold, and how many are sold.". Then there are two empty rows, and then 9 rows of data.
A quick note about the name of the package: googlesheets4 uses v4 of the [Sheets API v4](https://developers.google.com/sheets/api/) to provide an R interface to Google Sheets, hence the name.
In this section we'll work with the same datasets as the ones in the Excel section to highlight similarities and differences between workflows for reading data from Excel and Google Sheets.
readxl and googlesheets4 packages are both designed to mimic the functionality of the readr package, which provides the `read_csv()` function you've seen in @sec-data-import. Therefore, many of the tasks can be accomplished with simply swapping out `read_excel()` for `read_sheet()`.
However you'll also see that Excel and Google Sheets don't behave in exactly the same way, therefore other tasks may require further updates to the function calls.
The first argument to `read_sheet()` is the URL of the file to read.
You can also access this file via <https://pos.it/r4ds-students>, however note that at the time of writing this book you can't read a sheet directly from a short link.
```{r}
#| include: false
gs4_deauth()
```
```{r}
students <- read_sheet("https://docs.google.com/spreadsheets/d/1V1nPp1tzOuutXFLb3G9Eyxi3qxeEhnOXUzL5_BcCQ0w/edit?usp=sharing")
```
`read_sheet()` will read the file in as a tibble.
```{r}
students
```
Just like we did with `read_excel()`, we can supply column names, NA strings, and column types to `read_sheet()`.
You can write from R to Google Sheets with `write_sheet()`:
```{r}
#| eval: false
write_sheet(bake_sale, ss = "bake-sale")
```
If you'd like to write your data to a specific (work)sheet inside a Google Sheet, you can specify that with the `sheet` argument as well.
```{r}
#| eval: false
write_sheet(bake_sale, ss = "bake-sale", sheet = "Sales")
```
### Authentication
While you can read from a public Google Sheet without authenticating with your Google account, reading a private sheet or writing to a sheet requires authentication so that googlesheets4 can view and manage *your* Google Sheets.
You can do this with `gs4_auth()`, which will open a browser for authentication and authorization, or by specifying an email address in `gs4_auth()`, e.g. `gs4_auth("mine@example.com")`, which will force the use of a token associated with a specific email.
For further authentication details, we recommend reading the documentation googlesheets4 auth vignette: <https://googlesheets4.tidyverse.org/articles/auth.html>.
1. Read the `students` dataset from earlier in the chapter from Excel and also from Google Sheets, with no additional arguments supplied to the `read_excel()` and `read_sheet()` functions. Are the resulting data frames in R exactly the same? If not, how are they different?
2. Read the Google Sheet titled survey from <https://pos.it/r4ds-survey>, with `survey_id` as a character variable and `n_pets` as a numerical variable.
3. Read the Google Sheet titled roster from <https://pos.it/r4ds-roster>. The resulting data frame should be called `roster` and should look like the following.
In this chapter you learned how to read data into R from spreadsheets: from Microsoft Excel with `read_excel()` from the readxl package and from Google Sheets with `read_sheet()` from the googlesheets4 package.
These functions work very similarly to each other and have similar arguments for specifying column names, NA strings, rows to skip on top of the file you're reading in, etc.
Additionally, both functions make it possible to read a single sheet from a spreadsheet as well.
On the other hand, writing to an Excel file requires a different package and function (`writexl::write_xlsx()`) while you can write to a Google Sheet with the googlesheets4 package, with `write_sheet()`.
In the next chapter, you'll learn about a different data source and how to read data from that source into R: databases.