course/r4ds
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Typo fixes (#1204)

Browse Source 
* Grammatical edits

* Grammatical edits

* Grammatical edits

* Remove ref since fn not mentioned earlier

* Typo fixes, closes #1191

* Add palmerpenguins, closes #1192

* Grammatical edits

* More grammatical edits

* Omit warning, closes #1193

* Fix link, closes #1197

* Grammatical edits

* Code style + clarify labs() args, closes #1199

* Fix year, closes #1200

* Use penguins instead, closes #1201
This commit is contained in:
Mine Cetinkaya-Rundel
2023-01-03 02:06:27 -05:00
committed by GitHub
parent 26a20c586a
commit e68098f193
7 changed files with 304 additions and 267 deletions
Download Patch File Download Diff File Expand all files Collapse all files

91
data-import.qmd
View File

@@ -9,13 +9,13 @@ status("polishing")
## Introduction
Working with data provided by R packages is a great way to learn the tools of data science, but at some point you want to apply what you've learned to your own data.
Working with data provided by R packages is a great way to learn data science tools, but you want to apply what you've learned to your own data at some point.
In this chapter, you'll learn the basics of reading data files into R.
Specifically, this chapter will focus on reading plain-text rectangular files.
We'll start with some practical advice for handling features like column names and types and missing data.
We'll start with practical advice for handling features like column names, types, and missing data.
You will then learn about reading data from multiple files at once and writing data from R to a file.
Finally, you'll learn how to hand craft data frames in R.
Finally, you'll learn how to handcraft data frames in R.
### Prerequisites
@@ -30,9 +30,9 @@ library(tidyverse)
## Reading data from a file
To begin we'll focus on the most rectangular data file type: the CSV, short for comma-separated values.
To begin, we'll focus on the most rectangular data file type: CSV, which is short for comma-separated values.
Here is what a simple CSV file looks like.
The first row, commonly called the header row, gives the column names, and the following six rows give the data.
The first row, commonly called the header row, gives the column names, and the following six rows provide the data.
```{r}
#| echo: false
@@ -62,16 +62,16 @@ The first argument is the most important: it's the path to the file.
students <- read_csv("data/students.csv")
```
When you run `read_csv()` it prints out a message telling you the number of rows and columns of data, the delimiter that was used, and the column specifications (names of columns organized by the type of data the column contains).
It also prints out some information about how to retrieve the full column specification as well as how to quiet this message.
This message is an important part of readr and we'll come back to in @sec-col-types.
When you run `read_csv()`, it prints out a message telling you the number of rows and columns of data, the delimiter that was used, and the column specifications (names of columns organized by the type of data the column contains).
It also prints out some information about retrieving the full column specification and how to quiet this message.
This message is an integral part of readr, and we'll return to it in @sec-col-types.
### Practical advice
Once you read data in, the first step usually involves transforming it in some way to make it easier to work with in the rest of your analysis.
Let's take another look at the `students` data with that in mind.
In the `favourite.food` column, there are a bunch of food items and then the character string `N/A`, which should have been an real `NA` that R will recognize as "not available".
In the `favourite.food` column, there are a bunch of food items, and then the character string `N/A`, which should have been a real `NA` that R will recognize as "not available".
This is something we can address using the `na` argument.
```{r}
@@ -81,9 +81,9 @@ students <- read_csv("data/students.csv", na = c("N/A", ""))
students
```
You might also notice that the `Student ID` and `Full Name` columns are surrounded by back ticks.
You might also notice that the `Student ID` and `Full Name` columns are surrounded by backticks.
That's because they contain spaces, breaking R's usual rules for variable names.
To refer to them, you need to use those back ticks:
To refer to them, you need to use those backticks:
```{r}
students |>
@@ -104,7 +104,7 @@ students |> janitor::clean_names()
```
Another common task after reading in data is to consider variable types.
For example, `meal_type` is a categorical variable with a known set of possible values, which in R should be represent as factor:
For example, `meal_type` is a categorical variable with a known set of possible values, which in R should be represented as a factor:
```{r}
students |>
@@ -114,10 +114,11 @@ students |>
)
```
Note that the values in the `meal_type` variable has stayed exactly the same, but the type of variable denoted underneath the variable name has changed from character (`<chr>`) to factor (`<fct>`).
Note that the values in the `meal_type` variable have stayed the same, but the type of variable denoted underneath the variable name has changed from character (`<chr>`) to factor (`<fct>`).
You'll learn more about factors in @sec-factors.
Before you move on to analyzing these data, you'll probably want to fix the `age` column as well: currently it's a character variable because of the one observation that is typed out as `five` instead of a numeric `5`.
Before you analyze these data, you'll probably want to fix the `age` column.
Currently, it's a character variable because one of the observations is typed out as `five` instead of a numeric `5`.
We discuss the details of fixing this issue in @sec-import-spreadsheets.
```{r}
@@ -133,7 +134,7 @@ students
### Other arguments
There are a couple of other important arguments that we need to mention, and they'll be easier to demonstrate if we first show you a handy trick: `read_csv()` can read csv files that you've created in a string:
There are a couple of other important arguments that we need to mention, and they'll be easier to demonstrate if we first show you a handy trick: `read_csv()` can read CSV files that you've created in a string:
```{r}
#| message: false
@@ -145,8 +146,8 @@ read_csv(
)
```
Usually `read_csv()` uses the first line of the data for the column names, which is a very common convention.
But sometime there are a few lines of metadata at the top of the file.
Usually, `read_csv()` uses the first line of the data for the column names, which is a very common convention.
But it's not uncommon for a few lines of metadata to be included at the top of the file.
You can use `skip = n` to skip the first `n` lines or use `comment = "#"` to drop all lines that start with (e.g.) `#`:
```{r}
@@ -169,7 +170,7 @@ read_csv(
```
In other cases, the data might not have column names.
You can use `col_names = FALSE` to tell `read_csv()` not to treat the first row as headings, and instead label them sequentially from `X1` to `Xn`:
You can use `col_names = FALSE` to tell `read_csv()` not to treat the first row as headings and instead label them sequentially from `X1` to `Xn`:
```{r}
#| message: false
@@ -181,7 +182,7 @@ read_csv(
)
```
Alternatively you can pass `col_names` a character vector which will be used as the column names:
Alternatively, you can pass `col_names` a character vector which will be used as the column names:
```{r}
#| message: false
@@ -194,25 +195,25 @@ read_csv(
```
These arguments are all you need to know to read the majority of CSV files that you'll encounter in practice.
(For the rest, you'll need to carefully inspect your `.csv` file and carefully read the documentation for `read_csv()`'s many other arguments.)
(For the rest, you'll need to carefully inspect your `.csv` file and read the documentation for `read_csv()`'s many other arguments.)
### Other file types
Once you've mastered `read_csv()`, using readr's other functions is straightforward; it's just a matter of knowing which function to reach for:
- `read_csv2()` reads semicolon separated files.
These use `;` instead of `,` to separate fields, and are common in countries that use `,` as the decimal marker.
- `read_csv2()` reads semicolon-separated files.
These use `;` instead of `,` to separate fields and are common in countries that use `,` as the decimal marker.
- `read_tsv()` reads tab delimited files.
- `read_tsv()` reads tab-delimited files.
- `read_delim()` reads in files with any delimiter, attempting to automatically guess the delimited if you don't specify it.
- `read_delim()` reads in files with any delimiter, attempting to automatically guess the delimiter if you don't specify it.
- `read_fwf()` reads fixed width files.
You can specify fields either by their widths with `fwf_widths()` or their position with `fwf_positions()`.
- `read_fwf()` reads fixed-width files.
You can specify fields by their widths with `fwf_widths()` or by their positions with `fwf_positions()`.
- `read_table()` reads a common variation of fixed width files where columns are separated by white space.
- `read_table()` reads a common variation of fixed-width files where columns are separated by white space.
- `read_log()` reads Apache style log files.
- `read_log()` reads Apache-style log files.
### Exercises
@@ -223,8 +224,8 @@ Once you've mastered `read_csv()`, using readr's other functions is straightforw
3. What are the most important arguments to `read_fwf()`?
4. Sometimes strings in a CSV file contain commas.
To prevent them from causing problems they need to be surrounded by a quoting character, like `"` or `'`. By default, `read_csv()` assumes that the quoting character will be `"`.
What argument to `read_csv()` do you need to specify to read the following text into a data frame?
To prevent them from causing problems, they need to be surrounded by a quoting character, like `"` or `'`. By default, `read_csv()` assumes that the quoting character will be `"`.
To read the following text into a data frame, what argument to `read_csv()` do you need to specify?
```{r}
#| eval: false
@@ -249,8 +250,8 @@ Once you've mastered `read_csv()`, using readr's other functions is straightforw
a. Extracting the variable called `1`.
b. Plotting a scatterplot of `1` vs. `2`.
c. Creating a new column called `3` which is `2` divided by `1`.
d. Renaming the columns to `one`, `two` and `three`.
c. Creating a new column called `3`, which is `2` divided by `1`.
d. Renaming the columns to `one`, `two`, and `three`.
```{r}
annoying <- tibble(
@@ -261,21 +262,21 @@ Once you've mastered `read_csv()`, using readr's other functions is straightforw
## Controlling column types {#sec-col-types}
A CSV file doesn't contain any information about the type of each variable (i.e. whether it's a logical, number, string, etc.), so readr will try to guess the type.
This section describes how the guessing process works, how to resolve some common problems that cause it to fail, and if needed, how to supply the column types yourself.
Finally, we'll mention a couple of general strategies that are a useful if readr is failing catastrophically and you need to get more insight in to the structure of your file.
A CSV file doesn't contain any information about the type of each variable (i.e., whether it's a logical, number, string, etc.), so readr will try to guess the type.
This section describes how the guessing process works, how to resolve some common problems that cause it to fail, and, if needed, how to supply the column types yourself.
Finally, we'll mention a few general strategies that are useful if readr is failing catastrophically and you need to get more insight into the structure of your file.
### Guessing types
readr uses a heuristic to figure out the column types.
For each column, it pulls the values of 1,000[^data-import-2] rows spaced evenly from the first row to the last, ignoring an missing values.
For each column, it pulls the values of 1,000[^data-import-2] rows spaced evenly from the first row to the last, ignoring missing values.
It then works through the following questions:
[^data-import-2]: You can override the default of 1000 with the `guess_max` argument.
- Does it contain only `F`, `T`, `FALSE`, or `TRUE` (ignoring case)? If so, it's a logical.
- Does it contain only numbers (e.g. `1`, `-4.5`, `5e6`, `Inf`)? If so, it's a number.
- Does it match match the ISO8601 standard? If so, it's a date or date-time. (We'll come back to date/times in more detail in @sec-creating-datetimes).
- Does it contain only numbers (e.g., `1`, `-4.5`, `5e6`, `Inf`)? If so, it's a number.
- Does it match the ISO8601 standard? If so, it's a date or date-time. (We'll return to date-times in more detail in @sec-creating-datetimes).
- Otherwise, it must be a string.
You can see that behavior in action in this simple example:
@@ -289,12 +290,12 @@ read_csv("
)
```
This heuristic works well if you have a clean dataset, but in real life you'll encounter a selection of weird and wonderful failures.
This heuristic works well if you have a clean dataset, but in real life, you'll encounter a selection of weird and beautiful failures.
### Missing values, column types, and problems
The most common way column detection fails is that a column contains unexpected values and you get a character column instead of a more specific type.
One of the most common causes for this a missing value, recorded using something other than the `NA` that stringr expects.
The most common way column detection fails is that a column contains unexpected values, and you get a character column instead of a more specific type.
One of the most common causes for this is a missing value, recorded using something other than the `NA` that stringr expects.
Take this simple 1 column CSV file as an example:
@@ -315,7 +316,7 @@ df <- read_csv(csv)
```
In this very small case, you can easily see the missing value `.`.
But what happens if you have thousands of rows with only a few missing values represented by `.`s speckled amongst them?
But what happens if you have thousands of rows with only a few missing values represented by `.`s speckled among them?
One approach is to tell readr that `x` is a numeric column, and then see where it fails.
You can do that with the `col_types` argument, which takes a named list:
@@ -342,9 +343,9 @@ df <- read_csv(csv, na = ".")
readr provides a total of nine column types for you to use:
- `col_logical()` and `col_double()` read logicals and real numbers. They're relatively rarely needed (except as above), since readr will usually guess them for you.
- `col_integer()` reads integers. We distinguish because integers and doubles in this book because they're functionally equivalent, but reading integers explicitly can occasionally be useful because they occupy half the memory of doubles.
- `col_integer()` reads integers. We distinguish integers and doubles in this book because they're functionally equivalent, but reading integers explicitly can occasionally be useful because they occupy half the memory of doubles.
- `col_character()` reads strings. This is sometimes useful to specify explicitly when you have a column that is a numeric identifier, i.e. long series of digits that identifies some object, but it doesn't make sense to (e.g.) divide it in half.
- `col_factor()`, `col_date()` and `col_datetime()` create factors, dates and date-time respectively; you'll learn more about those when we get to those data types in @sec-factors and @sec-dates-and-times.
- `col_factor()`, `col_date()`, and `col_datetime()` create factors, dates, and date-times respectively; you'll learn more about those when we get to those data types in @sec-factors and @sec-dates-and-times.
- `col_number()` is a permissive numeric parser that will ignore non-numeric components, and is particularly useful for currencies. You'll learn more about it in @sec-numbers.
- `col_skip()` skips a column so it's not included in the result.
@@ -429,7 +430,7 @@ There are two main alternative:
```
2. The arrow package allows you to read and write parquet files, a fast binary file format that can be shared across programming languages.
We'll come back to arrow in more depth in @sec-arrow.
We'll return to arrow in more depth in @sec-arrow.
```{r}
#| eval: false