course/r4ds
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fix code language

Browse Source
This commit is contained in:
Hadley Wickham
2022-11-18 11:26:25 -06:00
parent 69b4597f3b
commit 868a35ca71
29 changed files with 912 additions and 907 deletions
Download Patch File Download Diff File Expand all files Collapse all files

62
oreilly/data-import.html
View File

@@ -10,7 +10,7 @@ Introduction</h1>
Prerequisites</h2>
<p>In this chapter, you’ll learn how to load flat files in R with the <strong>readr</strong> package, which is part of the core tidyverse.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">library(tidyverse)</pre>
<pre data-type="programlisting" data-code-language="r">library(tidyverse)</pre>
</div>
</section>
</section>
@@ -73,7 +73,7 @@ Reading data from a file</h1>
</div>
<p>We can read this file into R using <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code>. The first argument is the most important: it’s the path to the file.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">students &lt;- read_csv("data/students.csv")
<pre data-type="programlisting" data-code-language="r">students &lt;- read_csv("data/students.csv")
#&gt; Rows: 6 Columns: 5
#&gt; ── Column specification ─────────────────────────────────────────────────────
#&gt; Delimiter: ","
@@ -91,7 +91,7 @@ Practical advice</h2>
<p>Once you read data in, the first step usually involves transforming it in some way to make it easier to work with in the rest of your analysis. Let’s take another look at the <code>students</code> data with that in mind.</p>
<p>In the <code>favourite.food</code> column, there are a bunch of food items and then the character string <code>N/A</code>, which should have been an real <code>NA</code> that R will recognize as “not available”. This is something we can address using the <code>na</code> argument.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">students &lt;- read_csv("data/students.csv", na = c("N/A", ""))
<pre data-type="programlisting" data-code-language="r">students &lt;- read_csv("data/students.csv", na = c("N/A", ""))
students
#&gt; # A tibble: 6 × 5
@@ -106,7 +106,7 @@ students
</div>
<p>You might also notice that the <code>Student ID</code> and <code>Full Name</code> columns are surrounded by back ticks. That’s because they contain spaces, breaking R’s usual rules for variable names. To refer to them, you need to use those back ticks:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">students |&gt;
<pre data-type="programlisting" data-code-language="r">students |&gt;
rename(
student_id = `Student ID`,
full_name = `Full Name`
@@ -123,7 +123,7 @@ students
</div>
<p>An alternative approach is to use <code><a href="https://rdrr.io/pkg/janitor/man/clean_names.html">janitor::clean_names()</a></code> to use some heuristics to turn them all into snake case at once<span data-type="footnote">The <a href="http://sfirke.github.io/janitor/">janitor</a> package is not part of the tidyverse, but it offers handy functions for data cleaning and works well within data pipelines that uses <code>|&gt;</code>.</span>.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">students |&gt; janitor::clean_names()
<pre data-type="programlisting" data-code-language="r">students |&gt; janitor::clean_names()
#&gt; # A tibble: 6 × 5
#&gt; student_id full_name favourite_food meal_plan age
#&gt; &lt;dbl&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt;
@@ -136,7 +136,7 @@ students
</div>
<p>Another common task after reading in data is to consider variable types. For example, <code>meal_type</code> is a categorical variable with a known set of possible values, which in R should be represent as factor:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">students |&gt;
<pre data-type="programlisting" data-code-language="r">students |&gt;
janitor::clean_names() |&gt;
mutate(
meal_plan = factor(meal_plan)
@@ -154,7 +154,7 @@ students
<p>Note that the values in the <code>meal_type</code> variable has stayed exactly the same, but the type of variable denoted underneath the variable name has changed from character (<code>&lt;chr&gt;</code>) to factor (<code>&lt;fct&gt;</code>). You’ll learn more about factors in <a href="#chp-factors" data-type="xref">#chp-factors</a>.</p>
<p>Before you move on to analyzing these data, you’ll probably want to fix the <code>age</code> column as well: currently it’s a character variable because of the one observation that is typed out as <code>five</code> instead of a numeric <code>5</code>. We discuss the details of fixing this issue in <a href="#chp-spreadsheets" data-type="xref">#chp-spreadsheets</a>.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">students &lt;- students |&gt;
<pre data-type="programlisting" data-code-language="r">students &lt;- students |&gt;
janitor::clean_names() |&gt;
mutate(
meal_plan = factor(meal_plan),
@@ -179,7 +179,7 @@ students
Other arguments</h2>
<p>There are a couple of other important arguments that we need to mention, and they’ll be easier to demonstrate if we first show you a handy trick: <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code> can read csv files that you’ve created in a string:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">read_csv(
<pre data-type="programlisting" data-code-language="r">read_csv(
"a,b,c
1,2,3
4,5,6"
@@ -192,7 +192,7 @@ Other arguments</h2>
</div>
<p>Usually <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code> uses the first line of the data for the column names, which is a very common convention. But sometime there are a few lines of metadata at the top of the file. You can use <code>skip = n</code> to skip the first <code>n</code> lines or use <code>comment = "#"</code> to drop all lines that start with (e.g.) <code>#</code>:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">read_csv(
<pre data-type="programlisting" data-code-language="r">read_csv(
"The first line of metadata
The second line of metadata
x,y,z
@@ -217,7 +217,7 @@ read_csv(
</div>
<p>In other cases, the data might not have column names. You can use <code>col_names = FALSE</code> to tell <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code> not to treat the first row as headings, and instead label them sequentially from <code>X1</code> to <code>Xn</code>:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">read_csv(
<pre data-type="programlisting" data-code-language="r">read_csv(
"1,2,3
4,5,6",
col_names = FALSE
@@ -230,7 +230,7 @@ read_csv(
</div>
<p>Alternatively you can pass <code>col_names</code> a character vector which will be used as the column names:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">read_csv(
<pre data-type="programlisting" data-code-language="r">read_csv(
"1,2,3
4,5,6",
col_names = c("x", "y", "z")
@@ -265,13 +265,13 @@ Exercises</h2>
<li>
<p>Sometimes strings in a CSV file contain commas. To prevent them from causing problems they need to be surrounded by a quoting character, like <code>"</code> or <code>'</code>. By default, <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code> assumes that the quoting character will be <code>"</code>. What argument to <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code> do you need to specify to read the following text into a data frame?</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">"x,y\n1,'a,b'"</pre>
<pre data-type="programlisting" data-code-language="r">"x,y\n1,'a,b'"</pre>
</div>
</li>
<li>
<p>Identify what is wrong with each of the following inline CSV files. What happens when you run the code?</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">read_csv("a,b\n1,2,3\n4,5,6")
<pre data-type="programlisting" data-code-language="r">read_csv("a,b\n1,2,3\n4,5,6")
read_csv("a,b,c\n1,2\n1,2,3,4")
read_csv("a,b\n\"1")
read_csv("a,b\n1,2\na,b")
@@ -285,7 +285,7 @@ read_csv("a;b\n1;3")</pre>
<li>Creating a new column called <code>3</code> which is <code>2</code> divided by <code>1</code>.</li>
<li>Renaming the columns to <code>one</code>, <code>two</code> and <code>three</code>.</li>
</ol><div class="cell">
<pre data-type="programlisting" data-code-language="downlit">annoying &lt;- tibble(
<pre data-type="programlisting" data-code-language="r">annoying &lt;- tibble(
`1` = 1:10,
`2` = `1` * 2 + rnorm(length(`1`))
)</pre>
@@ -309,7 +309,7 @@ Guessing types</h2>
<li>Otherwise, it must be a string.</li>
</ul><p>You can see that behavior in action in this simple example:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">read_csv("
<pre data-type="programlisting" data-code-language="r">read_csv("
logical,numeric,date,string
TRUE,1,2021-01-15,abc
false,4.5,2021-02-15,def
@@ -341,7 +341,7 @@ Missing values, column types, and problems</h2>
<p>The most common way column detection fails is that a column contains unexpected values and you get a character column instead of a more specific type. One of the most common causes for this a missing value, recorded using something other than the <code>NA</code> that stringr expects.</p>
<p>Take this simple 1 column CSV file as an example:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">csv &lt;- "
<pre data-type="programlisting" data-code-language="r">csv &lt;- "
x
10
.
@@ -350,7 +350,7 @@ Missing values, column types, and problems</h2>
</div>
<p>If we read it without any additional arguments, <code>x</code> becomes a character column:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">df &lt;- read_csv(csv)
<pre data-type="programlisting" data-code-language="r">df &lt;- read_csv(csv)
#&gt; Rows: 4 Columns: 1
#&gt; ── Column specification ─────────────────────────────────────────────────────
#&gt; Delimiter: ","
@@ -361,7 +361,7 @@ Missing values, column types, and problems</h2>
</div>
<p>In this very small case, you can easily see the missing value <code>.</code>. But what happens if you have thousands of rows with only a few missing values represented by <code>.</code>s speckled amongst them? One approach is to tell readr that <code>x</code> is a numeric column, and then see where it fails. You can do that with the <code>col_types</code> argument, which takes a named list:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">df &lt;- read_csv(csv, col_types = list(x = col_double()))
<pre data-type="programlisting" data-code-language="r">df &lt;- read_csv(csv, col_types = list(x = col_double()))
#&gt; Warning: One or more parsing issues, call `problems()` on your data frame for
#&gt; details, e.g.:
#&gt; dat &lt;- vroom(...)
@@ -369,7 +369,7 @@ Missing values, column types, and problems</h2>
</div>
<p>Now <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code> reports that there was a problem, and tells us we can find out more with <code><a href="https://readr.tidyverse.org/reference/problems.html">problems()</a></code>:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">problems(df)
<pre data-type="programlisting" data-code-language="r">problems(df)
#&gt; # A tibble: 1 × 5
#&gt; row col expected actual file
#&gt; &lt;int&gt; &lt;int&gt; &lt;chr&gt; &lt;chr&gt; &lt;chr&gt;
@@ -377,7 +377,7 @@ Missing values, column types, and problems</h2>
</div>
<p>This tells us that there was a problem in row 3, col 1 where readr expected a double but got a <code>.</code>. That suggests this dataset uses <code>.</code> for missing values. So then we set <code>na = "."</code>, the automatic guessing succeeds, giving us the numeric column that we want:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">df &lt;- read_csv(csv, na = ".")
<pre data-type="programlisting" data-code-language="r">df &lt;- read_csv(csv, na = ".")
#&gt; Rows: 4 Columns: 1
#&gt; ── Column specification ─────────────────────────────────────────────────────
#&gt; Delimiter: ","
@@ -406,7 +406,7 @@ Column types</h2>
<code><a href="https://readr.tidyverse.org/reference/col_skip.html">col_skip()</a></code> skips a column so it’s not included in the result.</li>
</ul><p>It’s also possible to override the default column by switching from <code><a href="https://rdrr.io/r/base/list.html">list()</a></code> to <code><a href="https://readr.tidyverse.org/reference/cols.html">cols()</a></code>:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">csv &lt;- "
<pre data-type="programlisting" data-code-language="r">csv &lt;- "
x,y,z
1,2,3"
@@ -418,7 +418,7 @@ read_csv(csv, col_types = cols(.default = col_character()))
</div>
<p>Another useful helper is <code><a href="https://readr.tidyverse.org/reference/cols.html">cols_only()</a></code> which will read in only the columns you specify:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">read_csv(
<pre data-type="programlisting" data-code-language="r">read_csv(
"x,y,z
1,2,3",
col_types = cols_only(x = col_character())
@@ -436,7 +436,7 @@ read_csv(csv, col_types = cols(.default = col_character()))
Reading data from multiple files</h1>
<p>Sometimes your data is split across multiple files instead of being contained in a single file. For example, you might have sales data for multiple months, with each month’s data in a separate file: <code>01-sales.csv</code> for January, <code>02-sales.csv</code> for February, and <code>03-sales.csv</code> for March. With <code><a href="https://readr.tidyverse.org/reference/read_delim.html">read_csv()</a></code> you can read these data in at once and stack them on top of each other in a single data frame.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">sales_files &lt;- c("data/01-sales.csv", "data/02-sales.csv", "data/03-sales.csv")
<pre data-type="programlisting" data-code-language="r">sales_files &lt;- c("data/01-sales.csv", "data/02-sales.csv", "data/03-sales.csv")
read_csv(sales_files, id = "file")
#&gt; Rows: 19 Columns: 6
#&gt; ── Column specification ─────────────────────────────────────────────────────
@@ -460,7 +460,7 @@ read_csv(sales_files, id = "file")
<p>With the additional <code>id</code> parameter we have added a new column called <code>file</code> to the resulting data frame that identifies the file the data come from. This is especially helpful in circumstances where the files you’re reading in do not have an identifying column that can help you trace the observations back to their original sources.</p>
<p>If you have many files you want to read in, it can get cumbersome to write out their names as a list. Instead, you can use the base <code><a href="https://rdrr.io/r/base/list.files.html">list.files()</a></code> function to find the files for you by matching a pattern in the file names. You’ll learn more about these patterns in <a href="#chp-regexps" data-type="xref">#chp-regexps</a>.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">sales_files &lt;- list.files("data", pattern = "sales\\.csv$", full.names = TRUE)
<pre data-type="programlisting" data-code-language="r">sales_files &lt;- list.files("data", pattern = "sales\\.csv$", full.names = TRUE)
sales_files
#&gt; [1] "data/01-sales.csv" "data/02-sales.csv" "data/03-sales.csv"</pre>
</div>
@@ -472,11 +472,11 @@ Writing to a file</h1>
<p>readr also comes with two useful functions for writing data back to disk: <code><a href="https://readr.tidyverse.org/reference/write_delim.html">write_csv()</a></code> and <code><a href="https://readr.tidyverse.org/reference/write_delim.html">write_tsv()</a></code>. Both functions increase the chances of the output file being read back in correctly by using the standard UTF-8 encoding for strings and ISO8601 format for date-times.</p>
<p>The most important arguments are <code>x</code> (the data frame to save), and <code>file</code> (the location to save it). You can also specify how missing values are written with <code>na</code>, and if you want to <code>append</code> to an existing file.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">write_csv(students, "students.csv")</pre>
<pre data-type="programlisting" data-code-language="r">write_csv(students, "students.csv")</pre>
</div>
<p>Now let’s read that csv file back in. Note that the type information is lost when you save to csv:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">students
<pre data-type="programlisting" data-code-language="r">students
#&gt; # A tibble: 6 × 5
#&gt; student_id full_name favourite_food meal_plan age
#&gt; &lt;dbl&gt; &lt;chr&gt; &lt;chr&gt; &lt;fct&gt; &lt;dbl&gt;
@@ -502,7 +502,7 @@ read_csv("students-2.csv")
<ol type="1"><li>
<p><code><a href="https://readr.tidyverse.org/reference/read_rds.html">write_rds()</a></code> and <code><a href="https://readr.tidyverse.org/reference/read_rds.html">read_rds()</a></code> are uniform wrappers around the base functions <code><a href="https://rdrr.io/r/base/readRDS.html">readRDS()</a></code> and <code><a href="https://rdrr.io/r/base/readRDS.html">saveRDS()</a></code>. These store data in R’s custom binary format called RDS:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">write_rds(students, "students.rds")
<pre data-type="programlisting" data-code-language="r">write_rds(students, "students.rds")
read_rds("students.rds")
#&gt; # A tibble: 6 × 5
#&gt; student_id full_name favourite_food meal_plan age
@@ -518,7 +518,7 @@ read_rds("students.rds")
<li>
<p>The arrow package allows you to read and write parquet files, a fast binary file format that can be shared across programming languages:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">library(arrow)
<pre data-type="programlisting" data-code-language="r">library(arrow)
write_parquet(students, "students.parquet")
read_parquet("students.parquet")
#&gt; # A tibble: 6 × 5
@@ -540,7 +540,7 @@ read_parquet("students.parquet")
Data entry</h1>
<p>Sometimes you’ll need to assemble a tibble “by hand” doing a little data entry in your R script. There are two useful functions to help you do this which differ in whether you layout the tibble by columns or by rows. <code><a href="https://tibble.tidyverse.org/reference/tibble.html">tibble()</a></code> works by column:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">tibble(
<pre data-type="programlisting" data-code-language="r">tibble(
x = c(1, 2, 5),
y = c("h", "m", "g"),
z = c(0.08, 0.83, 0.60)
@@ -554,7 +554,7 @@ Data entry</h1>
</div>
<p>Note that every column in tibble must be same size, so you’ll get an error if they’re not:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">tibble(
<pre data-type="programlisting" data-code-language="r">tibble(
x = c(1, 2),
y = c("h", "m", "g"),
z = c(0.08, 0.83, 0.6)
@@ -567,7 +567,7 @@ Data entry</h1>
</div>
<p>Laying out the data by column can make it hard to see how the rows are related, so an alternative is <code><a href="https://tibble.tidyverse.org/reference/tribble.html">tribble()</a></code>, short for <strong>tr</strong>ansposed t<strong>ibble</strong>, which lets you lay out your data row by row. <code><a href="https://tibble.tidyverse.org/reference/tribble.html">tribble()</a></code> is customized for data entry in code: column headings start with <code>~</code> and entries are separated by commas. This makes it possible to lay out small amounts of data in an easy to read form:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">tribble(
<pre data-type="programlisting" data-code-language="r">tribble(
~x, ~y, ~z,
"h", 1, 0.08,
"m", 2, 0.83,