Don't transform non-crossref links
This commit is contained in:
		@@ -8,13 +8,13 @@
 | 
			
		||||
Base R
 | 
			
		||||
</h1><p>You are reading the work-in-progress second edition of R for Data Science. This chapter should be readable but is currently undergoing final polishing. You can find the complete first edition at <a href="https://r4ds.had.co.nz" class="uri">https://r4ds.had.co.nz</a>.</p>
 | 
			
		||||
 | 
			
		||||
<p>It’s possible to put a list in a column of a <code>data.frame</code>, but it’s a lot fiddlier because <code><a href="#chp-https://rdrr.io/r/base/data.frame" data-type="xref">#chp-https://rdrr.io/r/base/data.frame</a></code> treats a list as a list of columns:</p><div class="cell">
 | 
			
		||||
<p>It’s possible to put a list in a column of a <code>data.frame</code>, but it’s a lot fiddlier because <code><a href="https://rdrr.io/r/base/data.frame.html">data.frame()</a></code> treats a list as a list of columns:</p><div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">data.frame(x = list(1:3, 3:5))
 | 
			
		||||
#>   x.1.3 x.3.5
 | 
			
		||||
#> 1     1     3
 | 
			
		||||
#> 2     2     4
 | 
			
		||||
#> 3     3     5</pre>
 | 
			
		||||
</div><p>You can force <code><a href="#chp-https://rdrr.io/r/base/data.frame" data-type="xref">#chp-https://rdrr.io/r/base/data.frame</a></code> to treat a list as a list of rows by wrapping it in list <code><a href="#chp-https://rdrr.io/r/base/AsIs" data-type="xref">#chp-https://rdrr.io/r/base/AsIs</a></code>, but the result doesn’t print particularly well:</p><div class="cell">
 | 
			
		||||
</div><p>You can force <code><a href="https://rdrr.io/r/base/data.frame.html">data.frame()</a></code> to treat a list as a list of rows by wrapping it in list <code><a href="https://rdrr.io/r/base/AsIs.html">I()</a></code>, but the result doesn’t print particularly well:</p><div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">data.frame(
 | 
			
		||||
  x = I(list(1:2, 3:5)), 
 | 
			
		||||
  y = c("1, 2", "3, 4, 5")
 | 
			
		||||
@@ -22,13 +22,13 @@ Base R
 | 
			
		||||
#>         x       y
 | 
			
		||||
#> 1    1, 2    1, 2
 | 
			
		||||
#> 2 3, 4, 5 3, 4, 5</pre>
 | 
			
		||||
</div><p>It’s easier to use list-columns with tibbles because <code><a href="#chp-https://tibble.tidyverse.org/reference/tibble" data-type="xref">#chp-https://tibble.tidyverse.org/reference/tibble</a></code> treats lists like either vectors and the print method has been designed with lists in mind.</p></div>
 | 
			
		||||
</div><p>It’s easier to use list-columns with tibbles because <code><a href="https://tibble.tidyverse.org/reference/tibble.html">tibble()</a></code> treats lists like either vectors and the print method has been designed with lists in mind.</p></div>
 | 
			
		||||
 | 
			
		||||
<section id="introduction" data-type="sect1">
 | 
			
		||||
<h1>
 | 
			
		||||
Introduction</h1>
 | 
			
		||||
<p>In this chapter, you’ll learn the art of data <strong>rectangling</strong>, taking data that is fundamentally tree-like and converting it into a rectangular data frames made up of rows and columns. This is important because hierarchical data is surprisingly common, especially when working with data that comes from the web.</p>
 | 
			
		||||
<p>To learn about rectangling, you’ll need to first learn about lists, the data structure that makes hierarchical data possible. Then you’ll learn about two crucial tidyr functions: <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> and <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code>. We’ll then show you a few case studies, applying these simple functions again and again to solve real problems. We’ll finish off by talking about JSON, the most frequent source of hierarchical datasets and a common format for data exchange on the web.</p>
 | 
			
		||||
<p>To learn about rectangling, you’ll need to first learn about lists, the data structure that makes hierarchical data possible. Then you’ll learn about two crucial tidyr functions: <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">tidyr::unnest_longer()</a></code> and <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">tidyr::unnest_wider()</a></code>. We’ll then show you a few case studies, applying these simple functions again and again to solve real problems. We’ll finish off by talking about JSON, the most frequent source of hierarchical datasets and a common format for data exchange on the web.</p>
 | 
			
		||||
 | 
			
		||||
<section id="prerequisites" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
@@ -45,7 +45,7 @@ library(jsonlite)</pre>
 | 
			
		||||
<section id="lists" data-type="sect1">
 | 
			
		||||
<h1>
 | 
			
		||||
Lists</h1>
 | 
			
		||||
<p>So far you’ve worked with data frames that contain simple vectors like integers, numbers, characters, date-times, and factors. These vectors are simple because they’re homogeneous: every element is the same type. If you want to store element of different types in the same vector, you’ll need a <strong>list</strong>, which you create with <code><a href="#chp-https://rdrr.io/r/base/list" data-type="xref">#chp-https://rdrr.io/r/base/list</a></code>:</p>
 | 
			
		||||
<p>So far you’ve worked with data frames that contain simple vectors like integers, numbers, characters, date-times, and factors. These vectors are simple because they’re homogeneous: every element is the same type. If you want to store element of different types in the same vector, you’ll need a <strong>list</strong>, which you create with <code><a href="https://rdrr.io/r/base/list.html">list()</a></code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">x1 <- list(1:4, "a", TRUE)
 | 
			
		||||
x1
 | 
			
		||||
@@ -71,7 +71,7 @@ x2
 | 
			
		||||
#> $c
 | 
			
		||||
#> [1] 1 2 3 4</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>Even for these very simple lists, printing takes up quite a lot of space. A useful alternative is <code><a href="#chp-https://rdrr.io/r/utils/str" data-type="xref">#chp-https://rdrr.io/r/utils/str</a></code>, which generates a compact display of the <strong>str</strong>ucture, de-emphasizing the contents:</p>
 | 
			
		||||
<p>Even for these very simple lists, printing takes up quite a lot of space. A useful alternative is <code><a href="https://rdrr.io/r/utils/str.html">str()</a></code>, which generates a compact display of the <strong>str</strong>ucture, de-emphasizing the contents:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">str(x1)
 | 
			
		||||
#> List of 3
 | 
			
		||||
@@ -84,7 +84,7 @@ str(x2)
 | 
			
		||||
#>  $ b: int [1:3] 1 2 3
 | 
			
		||||
#>  $ c: int [1:4] 1 2 3 4</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>As you can see, <code><a href="#chp-https://rdrr.io/r/utils/str" data-type="xref">#chp-https://rdrr.io/r/utils/str</a></code> displays each child of the list on its own line. It displays the name, if present, then an abbreviation of the type, then the first few values.</p>
 | 
			
		||||
<p>As you can see, <code><a href="https://rdrr.io/r/utils/str.html">str()</a></code> displays each child of the list on its own line. It displays the name, if present, then an abbreviation of the type, then the first few values.</p>
 | 
			
		||||
 | 
			
		||||
<section id="hierarchy" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
@@ -101,7 +101,7 @@ str(x3)
 | 
			
		||||
#>   ..$ : num 3
 | 
			
		||||
#>   ..$ : num 4</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>This is notably different to <code><a href="#chp-https://rdrr.io/r/base/c" data-type="xref">#chp-https://rdrr.io/r/base/c</a></code>, which generates a flat vector:</p>
 | 
			
		||||
<p>This is notably different to <code><a href="https://rdrr.io/r/base/c.html">c()</a></code>, which generates a flat vector:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">c(c(1, 2), c(3, 4))
 | 
			
		||||
#> [1] 1 2 3 4
 | 
			
		||||
@@ -114,7 +114,7 @@ str(x4)
 | 
			
		||||
#>  $ : num 3
 | 
			
		||||
#>  $ : num 4</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>As lists get more complex, <code><a href="#chp-https://rdrr.io/r/utils/str" data-type="xref">#chp-https://rdrr.io/r/utils/str</a></code> gets more useful, as it lets you see the hierarchy at a glance:</p>
 | 
			
		||||
<p>As lists get more complex, <code><a href="https://rdrr.io/r/utils/str.html">str()</a></code> gets more useful, as it lets you see the hierarchy at a glance:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">x5 <- list(1, list(2, list(3, list(4, list(5)))))
 | 
			
		||||
str(x5)
 | 
			
		||||
@@ -129,7 +129,7 @@ str(x5)
 | 
			
		||||
#>   .. .. ..$ :List of 1
 | 
			
		||||
#>   .. .. .. ..$ : num 5</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>As lists get even larger and more complex, <code><a href="#chp-https://rdrr.io/r/utils/str" data-type="xref">#chp-https://rdrr.io/r/utils/str</a></code> eventually starts to fail, and you’ll need to switch to <code><a href="#chp-https://rdrr.io/r/utils/View" data-type="xref">#chp-https://rdrr.io/r/utils/View</a></code><span data-type="footnote">This is an RStudio feature.</span>. <a href="#fig-view-collapsed" data-type="xref">#fig-view-collapsed</a> shows the result of calling <code>View(x4)</code>. The viewer starts by showing just the top level of the list, but you can interactively expand any of the components to see more, as in <a href="#fig-view-expand-1" data-type="xref">#fig-view-expand-1</a>. RStudio will also show you the code you need to access that element, as in <a href="#fig-view-expand-2" data-type="xref">#fig-view-expand-2</a>. We’ll come back to how this code works in <a href="#sec-subset-one" data-type="xref">#sec-subset-one</a>.</p>
 | 
			
		||||
<p>As lists get even larger and more complex, <code><a href="https://rdrr.io/r/utils/str.html">str()</a></code> eventually starts to fail, and you’ll need to switch to <code><a href="https://rdrr.io/r/utils/View.html">View()</a></code><span data-type="footnote">This is an RStudio feature.</span>. <a href="#fig-view-collapsed" data-type="xref">#fig-view-collapsed</a> shows the result of calling <code>View(x4)</code>. The viewer starts by showing just the top level of the list, but you can interactively expand any of the components to see more, as in <a href="#fig-view-expand-1" data-type="xref">#fig-view-expand-1</a>. RStudio will also show you the code you need to access that element, as in <a href="#fig-view-expand-2" data-type="xref">#fig-view-expand-2</a>. We’ll come back to how this code works in <a href="#sec-subset-one" data-type="xref">#sec-subset-one</a>.</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<div class="cell-output-display">
 | 
			
		||||
 | 
			
		||||
@@ -159,7 +159,7 @@ str(x5)
 | 
			
		||||
<section id="list-columns" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
List-columns</h2>
 | 
			
		||||
<p>Lists can also live inside a tibble, where we call them list-columns. List-columns are useful because they allow you to shoehorn in objects that wouldn’t usually belong in a tibble. In particular, list-columns are are used a lot in the <a href="#chp-https://www.tidymodels" data-type="xref">#chp-https://www.tidymodels</a> ecosystem, because they allow you to store things like models or resamples in a data frame.</p>
 | 
			
		||||
<p>Lists can also live inside a tibble, where we call them list-columns. List-columns are useful because they allow you to shoehorn in objects that wouldn’t usually belong in a tibble. In particular, list-columns are are used a lot in the <a href="https://www.tidymodels.org">tidymodels</a> ecosystem, because they allow you to store things like models or resamples in a data frame.</p>
 | 
			
		||||
<p>Here’s a simple example of a list-column:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">df <- tibble(
 | 
			
		||||
@@ -195,18 +195,18 @@ df
 | 
			
		||||
#>   ..$ : num 1
 | 
			
		||||
#>   ..$ : num 2</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>Similarly, if you <code><a href="#chp-https://rdrr.io/r/utils/View" data-type="xref">#chp-https://rdrr.io/r/utils/View</a></code> a data frame in RStudio, you’ll get the standard tabular view, which doesn’t allow you to selectively expand list columns. To explore those fields you’ll need to <code><a href="#chp-https://dplyr.tidyverse.org/reference/pull" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/pull</a></code> and view, e.g. <code>df |> pull(z) |> View()</code>.</p>
 | 
			
		||||
<p>Similarly, if you <code><a href="https://rdrr.io/r/utils/View.html">View()</a></code> a data frame in RStudio, you’ll get the standard tabular view, which doesn’t allow you to selectively expand list columns. To explore those fields you’ll need to <code><a href="https://dplyr.tidyverse.org/reference/pull.html">pull()</a></code> and view, e.g. <code>df |> pull(z) |> View()</code>.</p>
 | 
			
		||||
<div data-type="note"><h1>
 | 
			
		||||
Base R
 | 
			
		||||
</h1><p>You are reading the work-in-progress second edition of R for Data Science. This chapter should be readable but is currently undergoing final polishing. You can find the complete first edition at <a href="https://r4ds.had.co.nz" class="uri">https://r4ds.had.co.nz</a>.</p>
 | 
			
		||||
 | 
			
		||||
<p>It’s possible to put a list in a column of a <code>data.frame</code>, but it’s a lot fiddlier because <code><a href="#chp-https://rdrr.io/r/base/data.frame" data-type="xref">#chp-https://rdrr.io/r/base/data.frame</a></code> treats a list as a list of columns:</p><div class="cell">
 | 
			
		||||
<p>It’s possible to put a list in a column of a <code>data.frame</code>, but it’s a lot fiddlier because <code><a href="https://rdrr.io/r/base/data.frame.html">data.frame()</a></code> treats a list as a list of columns:</p><div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">data.frame(x = list(1:3, 3:5))
 | 
			
		||||
#>   x.1.3 x.3.5
 | 
			
		||||
#> 1     1     3
 | 
			
		||||
#> 2     2     4
 | 
			
		||||
#> 3     3     5</pre>
 | 
			
		||||
</div><p>You can force <code><a href="#chp-https://rdrr.io/r/base/data.frame" data-type="xref">#chp-https://rdrr.io/r/base/data.frame</a></code> to treat a list as a list of rows by wrapping it in list <code><a href="#chp-https://rdrr.io/r/base/AsIs" data-type="xref">#chp-https://rdrr.io/r/base/AsIs</a></code>, but the result doesn’t print particularly well:</p><div class="cell">
 | 
			
		||||
</div><p>You can force <code><a href="https://rdrr.io/r/base/data.frame.html">data.frame()</a></code> to treat a list as a list of rows by wrapping it in list <code><a href="https://rdrr.io/r/base/AsIs.html">I()</a></code>, but the result doesn’t print particularly well:</p><div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">data.frame(
 | 
			
		||||
  x = I(list(1:2, 3:5)), 
 | 
			
		||||
  y = c("1, 2", "3, 4, 5")
 | 
			
		||||
@@ -214,7 +214,7 @@ Base R
 | 
			
		||||
#>         x       y
 | 
			
		||||
#> 1    1, 2    1, 2
 | 
			
		||||
#> 2 3, 4, 5 3, 4, 5</pre>
 | 
			
		||||
</div><p>It’s easier to use list-columns with tibbles because <code><a href="#chp-https://tibble.tidyverse.org/reference/tibble" data-type="xref">#chp-https://tibble.tidyverse.org/reference/tibble</a></code> treats lists like either vectors and the print method has been designed with lists in mind.</p></div>
 | 
			
		||||
</div><p>It’s easier to use list-columns with tibbles because <code><a href="https://tibble.tidyverse.org/reference/tibble.html">tibble()</a></code> treats lists like either vectors and the print method has been designed with lists in mind.</p></div>
 | 
			
		||||
 | 
			
		||||
</section>
 | 
			
		||||
</section>
 | 
			
		||||
@@ -242,13 +242,13 @@ df2 <- tribble(
 | 
			
		||||
  3, list(31, 32),
 | 
			
		||||
)</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>tidyr provides two functions for these two cases: <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> and <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code>. The following sections explain how they work.</p>
 | 
			
		||||
<p>tidyr provides two functions for these two cases: <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> and <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code>. The following sections explain how they work.</p>
 | 
			
		||||
 | 
			
		||||
<section id="unnest_wider" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
<code>unnest_wider()</code>
 | 
			
		||||
</h2>
 | 
			
		||||
<p>When each row has the same number of elements with the same names, like <code>df1</code>, it’s natural to put each component into its own column with <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code>:</p>
 | 
			
		||||
<p>When each row has the same number of elements with the same names, like <code>df1</code>, it’s natural to put each component into its own column with <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">df1 |> 
 | 
			
		||||
  unnest_wider(y)
 | 
			
		||||
@@ -270,7 +270,7 @@ df2 <- tribble(
 | 
			
		||||
#> 2     2    21    22
 | 
			
		||||
#> 3     3    31    32</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>We can also use <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> with unnamed list-columns, as in <code>df2</code>. Since columns require names but the list lacks them, <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> will label them with consecutive integers:</p>
 | 
			
		||||
<p>We can also use <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> with unnamed list-columns, as in <code>df2</code>. Since columns require names but the list lacks them, <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> will label them with consecutive integers:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">df2 |> 
 | 
			
		||||
  unnest_wider(y, names_sep = "_")
 | 
			
		||||
@@ -281,14 +281,14 @@ df2 <- tribble(
 | 
			
		||||
#> 2     2    21    NA    NA
 | 
			
		||||
#> 3     3    31    32    NA</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>You’ll notice that <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code>, much like <code><a href="#chp-https://tidyr.tidyverse.org/reference/pivot_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/pivot_wider</a></code>, turns implicit missing values in to explicit missing values.</p>
 | 
			
		||||
<p>You’ll notice that <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code>, much like <code><a href="https://tidyr.tidyverse.org/reference/pivot_wider.html">pivot_wider()</a></code>, turns implicit missing values in to explicit missing values.</p>
 | 
			
		||||
</section>
 | 
			
		||||
 | 
			
		||||
<section id="unnest_longer" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
<code>unnest_longer()</code>
 | 
			
		||||
</h2>
 | 
			
		||||
<p>When each row contains an unnamed list, it’s most natural to put each element into its own row with <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code>:</p>
 | 
			
		||||
<p>When each row contains an unnamed list, it’s most natural to put each element into its own row with <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">df2 |> 
 | 
			
		||||
  unnest_longer(y)
 | 
			
		||||
@@ -360,7 +360,7 @@ Inconsistent types</h2>
 | 
			
		||||
  "b", list(TRUE, factor("a"), 5)
 | 
			
		||||
)</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p><code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> always keeps the set of columns change, while changing the number of rows. So what happens? How does <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> produce five rows while keeping everything in <code>y</code>?</p>
 | 
			
		||||
<p><code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> always keeps the set of columns change, while changing the number of rows. So what happens? How does <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> produce five rows while keeping everything in <code>y</code>?</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">df4 |> 
 | 
			
		||||
  unnest_longer(y)
 | 
			
		||||
@@ -373,7 +373,7 @@ Inconsistent types</h2>
 | 
			
		||||
#> 4 b     <fct [1]>
 | 
			
		||||
#> 5 b     <dbl [1]></pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>As you can see, the output contains a list-column, but every element of the list-column contains a single element. Because <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> can’t find a common type of vector, it keeps the original types in a list-column. You might wonder if this breaks the commandment that every element of a column must be the same type — not quite: every element is a still a list, even though the contents of each element is a different type.</p>
 | 
			
		||||
<p>As you can see, the output contains a list-column, but every element of the list-column contains a single element. Because <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> can’t find a common type of vector, it keeps the original types in a list-column. You might wonder if this breaks the commandment that every element of a column must be the same type — not quite: every element is a still a list, even though the contents of each element is a different type.</p>
 | 
			
		||||
<p>What happens if you find this problem in a dataset you’re trying to rectangle? There are two basic options. You could use the <code>transform</code> argument to coerce all inputs to a common type. It’s not particularly useful here because there’s only really one class that these five class can be converted to character.</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">df4 |> 
 | 
			
		||||
@@ -398,7 +398,7 @@ Inconsistent types</h2>
 | 
			
		||||
#> 1 a     <dbl [1]>
 | 
			
		||||
#> 2 b     <dbl [1]></pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>Then you can call <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> once more:</p>
 | 
			
		||||
<p>Then you can call <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> once more:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">df4 |> 
 | 
			
		||||
  unnest_longer(y) |> 
 | 
			
		||||
@@ -410,7 +410,7 @@ Inconsistent types</h2>
 | 
			
		||||
#> 1 a         1
 | 
			
		||||
#> 2 b         5</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>You’ll learn more about <code><a href="#chp-https://purrr.tidyverse.org/reference/map" data-type="xref">#chp-https://purrr.tidyverse.org/reference/map</a></code> in <a href="#chp-iteration" data-type="xref">#chp-iteration</a>.</p>
 | 
			
		||||
<p>You’ll learn more about <code><a href="https://purrr.tidyverse.org/reference/map.html">map_lgl()</a></code> in <a href="#chp-iteration" data-type="xref">#chp-iteration</a>.</p>
 | 
			
		||||
</section>
 | 
			
		||||
 | 
			
		||||
<section id="other-functions" data-type="sect2">
 | 
			
		||||
@@ -418,11 +418,11 @@ Inconsistent types</h2>
 | 
			
		||||
Other functions</h2>
 | 
			
		||||
<p>tidyr has a few other useful rectangling functions that we’re not going to cover in this book:</p>
 | 
			
		||||
<ul><li>
 | 
			
		||||
<code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_auto" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_auto</a></code> automatically picks between <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> and <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> based on the structure of the list-column. It’s a great for rapid exploration, but ultimately its a bad idea because it doesn’t force you to understand how your data is structured, and makes your code harder to understand.</li>
 | 
			
		||||
<code><a href="https://tidyr.tidyverse.org/reference/unnest_auto.html">unnest_auto()</a></code> automatically picks between <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> and <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> based on the structure of the list-column. It’s a great for rapid exploration, but ultimately its a bad idea because it doesn’t force you to understand how your data is structured, and makes your code harder to understand.</li>
 | 
			
		||||
<li>
 | 
			
		||||
<code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest</a></code> expands both rows and columns. It’s useful when you have a list-column that contains a 2d structure like a data frame, which you don’t see in this book.</li>
 | 
			
		||||
<code><a href="https://tidyr.tidyverse.org/reference/unnest.html">unnest()</a></code> expands both rows and columns. It’s useful when you have a list-column that contains a 2d structure like a data frame, which you don’t see in this book.</li>
 | 
			
		||||
<li>
 | 
			
		||||
<code><a href="#chp-https://tidyr.tidyverse.org/reference/hoist" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/hoist</a></code> allows you to reach into a deeply nested list and extract just the components that you need. It’s mostly equivalent to repeated invocations of <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> + <code><a href="#chp-https://dplyr.tidyverse.org/reference/select" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/select</a></code> so read up on it if you’re trying to extract just a couple of important variables embedded in a bunch of data that you don’t care about.</li>
 | 
			
		||||
<code><a href="https://tidyr.tidyverse.org/reference/hoist.html">hoist()</a></code> allows you to reach into a deeply nested list and extract just the components that you need. It’s mostly equivalent to repeated invocations of <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> + <code><a href="https://dplyr.tidyverse.org/reference/select.html">select()</a></code> so read up on it if you’re trying to extract just a couple of important variables embedded in a bunch of data that you don’t care about.</li>
 | 
			
		||||
</ul><p>These are good to know about when you’re reading other people’s code or tackling rarer rectangling challenges.</p>
 | 
			
		||||
</section>
 | 
			
		||||
 | 
			
		||||
@@ -430,7 +430,7 @@ Other functions</h2>
 | 
			
		||||
<h2>
 | 
			
		||||
Exercises</h2>
 | 
			
		||||
<ol type="1"><li>
 | 
			
		||||
<p>From time-to-time you encounter data frames with multiple list-columns with aligned values. For example, in the following data frame, the values of <code>y</code> and <code>z</code> are aligned (i.e. <code>y</code> and <code>z</code> will always have the same length within a row, and the first value of <code>y</code> corresponds to the first value of <code>z</code>). What happens if you apply two <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> calls to this data frame? How can you preserve the relationship between <code>x</code> and <code>y</code>? (Hint: carefully read the docs).</p>
 | 
			
		||||
<p>From time-to-time you encounter data frames with multiple list-columns with aligned values. For example, in the following data frame, the values of <code>y</code> and <code>z</code> are aligned (i.e. <code>y</code> and <code>z</code> will always have the same length within a row, and the first value of <code>y</code> corresponds to the first value of <code>z</code>). What happens if you apply two <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> calls to this data frame? How can you preserve the relationship between <code>x</code> and <code>y</code>? (Hint: carefully read the docs).</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">df4 <- tribble(
 | 
			
		||||
  ~x, ~y, ~z,
 | 
			
		||||
@@ -445,7 +445,7 @@ Exercises</h2>
 | 
			
		||||
<section id="case-studies" data-type="sect1">
 | 
			
		||||
<h1>
 | 
			
		||||
Case studies</h1>
 | 
			
		||||
<p>The main difference between the simple examples we used above and real data is that real data typically contains multiple levels of nesting that require multiple calls to <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> and/or <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code>. This section will work through four real rectangling challenges using datasets from the repurrrsive package, inspired by datasets that we’ve encountered in the wild.</p>
 | 
			
		||||
<p>The main difference between the simple examples we used above and real data is that real data typically contains multiple levels of nesting that require multiple calls to <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> and/or <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code>. This section will work through four real rectangling challenges using datasets from the repurrrsive package, inspired by datasets that we’ve encountered in the wild.</p>
 | 
			
		||||
 | 
			
		||||
<section id="very-wide-data" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
@@ -465,7 +465,7 @@ repos
 | 
			
		||||
#> 5 <list [30]>
 | 
			
		||||
#> 6 <list [30]></pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>This tibble contains 6 rows, one row for each child of <code>gh_repos</code>. Each row contains a unnamed list with either 26 or 30 rows. Since these are unnamed, we’ll start with <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> to put each child in its own row:</p>
 | 
			
		||||
<p>This tibble contains 6 rows, one row for each child of <code>gh_repos</code>. Each row contains a unnamed list with either 26 or 30 rows. Since these are unnamed, we’ll start with <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> to put each child in its own row:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">repos |> 
 | 
			
		||||
  unnest_longer(json)
 | 
			
		||||
@@ -480,7 +480,7 @@ repos
 | 
			
		||||
#> 6 <named list [68]>
 | 
			
		||||
#> # … with 170 more rows</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>At first glance, it might seem like we haven’t improved the situation: while we have more rows (176 instead of 6) each element of <code>json</code> is still a list. However, there’s an important difference: now each element is a <strong>named</strong> list so we can use <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> to put each element into its own column:</p>
 | 
			
		||||
<p>At first glance, it might seem like we haven’t improved the situation: while we have more rows (176 instead of 6) each element of <code>json</code> is still a list. However, there’s an important difference: now each element is a <strong>named</strong> list so we can use <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> to put each element into its own column:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">repos |> 
 | 
			
		||||
  unnest_longer(json) |> 
 | 
			
		||||
@@ -502,7 +502,7 @@ repos
 | 
			
		||||
#> #   languages_url <chr>, stargazers_url <chr>, contributors_url <chr>,
 | 
			
		||||
#> #   subscribers_url <chr>, subscription_url <chr>, commits_url <chr>, …</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>This has worked but the result is a little overwhelming: there are so many columns that tibble doesn’t even print all of them! We can see them all with <code><a href="#chp-https://rdrr.io/r/base/names" data-type="xref">#chp-https://rdrr.io/r/base/names</a></code>:</p>
 | 
			
		||||
<p>This has worked but the result is a little overwhelming: there are so many columns that tibble doesn’t even print all of them! We can see them all with <code><a href="https://rdrr.io/r/base/names.html">names()</a></code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">repos |> 
 | 
			
		||||
  unnest_longer(json) |> 
 | 
			
		||||
@@ -550,7 +550,7 @@ repos
 | 
			
		||||
#> # … with 170 more rows</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>You can use this to work back to understand how <code>gh_repos</code> was strucured: each child was a GitHub user containing a list of up to 30 GitHub repositories that they created.</p>
 | 
			
		||||
<p><code>owner</code> is another list-column, and since it contains a named list, we can use <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> to get at the values:</p>
 | 
			
		||||
<p><code>owner</code> is another list-column, and since it contains a named list, we can use <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> to get at the values:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">repos |> 
 | 
			
		||||
  unnest_longer(json) |> 
 | 
			
		||||
@@ -729,7 +729,7 @@ characters |>
 | 
			
		||||
<section id="a-dash-of-text-analysis" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
A dash of text analysis</h2>
 | 
			
		||||
<p>What if we wanted to find the most common words in the title? One simple approach starts by using <code><a href="#chp-https://stringr.tidyverse.org/reference/str_split" data-type="xref">#chp-https://stringr.tidyverse.org/reference/str_split</a></code> to break each element of <code>title</code> up into words by spitting on <code>" "</code>:</p>
 | 
			
		||||
<p>What if we wanted to find the most common words in the title? One simple approach starts by using <code><a href="https://stringr.tidyverse.org/reference/str_split.html">str_split()</a></code> to break each element of <code>title</code> up into words by spitting on <code>" "</code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">titles |> 
 | 
			
		||||
  mutate(word = str_split(title, " "), .keep = "unused")
 | 
			
		||||
@@ -744,7 +744,7 @@ A dash of text analysis</h2>
 | 
			
		||||
#> 6  1074 <chr [6]> 
 | 
			
		||||
#> # … with 47 more rows</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>This creates a unnamed variable length list-column, so we can use <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code>:</p>
 | 
			
		||||
<p>This creates a unnamed variable length list-column, so we can use <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">titles |> 
 | 
			
		||||
  mutate(word = str_split(title, " "), .keep = "unused") |> 
 | 
			
		||||
@@ -798,13 +798,13 @@ titles |>
 | 
			
		||||
#> 6 Queen        5
 | 
			
		||||
#> # … with 70 more rows</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>Breaking up text into individual fragments is a powerful idea that underlies much of text analysis. If this sounds interesting, a good place to learn more is <a href="#chp-https://www.tidytextmining" data-type="xref">#chp-https://www.tidytextmining</a> by Julia Silge and David Robinson.</p>
 | 
			
		||||
<p>Breaking up text into individual fragments is a powerful idea that underlies much of text analysis. If this sounds interesting, a good place to learn more is <a href="https://www.tidytextmining.com">Text Mining with R</a> by Julia Silge and David Robinson.</p>
 | 
			
		||||
</section>
 | 
			
		||||
 | 
			
		||||
<section id="deeply-nested" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
Deeply nested</h2>
 | 
			
		||||
<p>We’ll finish off these case studies with a list-column that’s very deeply nested and requires repeated rounds of <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> and <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> to unravel: <code>gmaps_cities</code>. This is a two column tibble containing five city names and the results of using Google’s <a href="#chp-https://developers.google.com/maps/documentation/geocoding" data-type="xref">#chp-https://developers.google.com/maps/documentation/geocoding</a> to determine their location:</p>
 | 
			
		||||
<p>We’ll finish off these case studies with a list-column that’s very deeply nested and requires repeated rounds of <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> and <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> to unravel: <code>gmaps_cities</code>. This is a two column tibble containing five city names and the results of using Google’s <a href="https://developers.google.com/maps/documentation/geocoding">geocoding API</a> to determine their location:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">gmaps_cities
 | 
			
		||||
#> # A tibble: 5 × 2
 | 
			
		||||
@@ -816,7 +816,7 @@ Deeply nested</h2>
 | 
			
		||||
#> 4 Chicago    <named list [2]>
 | 
			
		||||
#> 5 Arlington  <named list [2]></pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p><code>json</code> is a list-column with internal names, so we start with an <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code>:</p>
 | 
			
		||||
<p><code>json</code> is a list-column with internal names, so we start with an <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">gmaps_cities |> 
 | 
			
		||||
  unnest_wider(json)
 | 
			
		||||
@@ -846,7 +846,7 @@ Deeply nested</h2>
 | 
			
		||||
#> 6 Arlington  <named list [5]>
 | 
			
		||||
#> # … with 1 more row</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>Now <code>results</code> is a named list, so we’ll use <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code>:</p>
 | 
			
		||||
<p>Now <code>results</code> is a named list, so we’ll use <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">locations <- gmaps_cities |> 
 | 
			
		||||
  unnest_wider(json) |> 
 | 
			
		||||
@@ -938,8 +938,8 @@ locations
 | 
			
		||||
#> 6 Arlington  Arlington, TX, USA    32.8  -97.0   32.6  -97.2
 | 
			
		||||
#> # … with 1 more row</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>Note how we unnest two columns simultaneously by supplying a vector of variable names to <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code>.</p>
 | 
			
		||||
<p>This is somewhere that <code><a href="#chp-https://tidyr.tidyverse.org/reference/hoist" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/hoist</a></code>, mentioned briefly above, can be useful. Once you’ve discovered the path to get to the components you’re interested in, you can extract them directly using <code><a href="#chp-https://tidyr.tidyverse.org/reference/hoist" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/hoist</a></code>:</p>
 | 
			
		||||
<p>Note how we unnest two columns simultaneously by supplying a vector of variable names to <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code>.</p>
 | 
			
		||||
<p>This is somewhere that <code><a href="https://tidyr.tidyverse.org/reference/hoist.html">hoist()</a></code>, mentioned briefly above, can be useful. Once you’ve discovered the path to get to the components you’re interested in, you can extract them directly using <code><a href="https://tidyr.tidyverse.org/reference/hoist.html">hoist()</a></code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">locations |> 
 | 
			
		||||
  select(city, formatted_address, geometry) |> 
 | 
			
		||||
@@ -968,7 +968,7 @@ locations
 | 
			
		||||
<h2>
 | 
			
		||||
Exercises</h2>
 | 
			
		||||
<ol type="1"><li><p>Roughly estimate when <code>gh_repos</code> was created. Why can you only roughly estimate the date?</p></li>
 | 
			
		||||
<li><p>The <code>owner</code> column of <code>gh_repo</code> contains a lot of duplicated information because each owner can have many repos. Can you construct a <code>owners</code> data frame that contains one row for each owner? (Hint: does <code><a href="#chp-https://dplyr.tidyverse.org/reference/distinct" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/distinct</a></code> work with <code>list-cols</code>?)</p></li>
 | 
			
		||||
<li><p>The <code>owner</code> column of <code>gh_repo</code> contains a lot of duplicated information because each owner can have many repos. Can you construct a <code>owners</code> data frame that contains one row for each owner? (Hint: does <code><a href="https://dplyr.tidyverse.org/reference/distinct.html">distinct()</a></code> work with <code>list-cols</code>?)</p></li>
 | 
			
		||||
<li>
 | 
			
		||||
<p>Explain the following code line-by-line. Why is it interesting? Why does it work for <code>got_chars</code> but might not work in general?</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
@@ -983,7 +983,7 @@ Exercises</h2>
 | 
			
		||||
  unnest_longer(value)</pre>
 | 
			
		||||
</div>
 | 
			
		||||
</li>
 | 
			
		||||
<li><p>In <code>gmaps_cities</code>, what does <code>address_components</code> contain? Why does the length vary between rows? Unnest it appropriately to figure it out. (Hint: <code>types</code> always appears to contain two elements. Does <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> make it easier to work with than <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code>?) .</p></li>
 | 
			
		||||
<li><p>In <code>gmaps_cities</code>, what does <code>address_components</code> contain? Why does the length vary between rows? Unnest it appropriately to figure it out. (Hint: <code>types</code> always appears to contain two elements. Does <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> make it easier to work with than <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code>?) .</p></li>
 | 
			
		||||
</ol></section>
 | 
			
		||||
</section>
 | 
			
		||||
 | 
			
		||||
@@ -1001,13 +1001,13 @@ Data types</h2>
 | 
			
		||||
<li>A <strong>number</strong> is similar to R’s numbers: they can use integer (e.g. 123), decimal (e.g. 123.45), or scientific (e.g. 1.23e3) notation. JSON doesn’t support Inf, -Inf, or NaN.</li>
 | 
			
		||||
<li>A <strong>boolean</strong> is similar to R’s <code>TRUE</code> and <code>FALSE</code>, but uses lowercase <code>true</code> and <code>false</code>.</li>
 | 
			
		||||
</ul><p>JSON’s strings, numbers, and booleans are pretty similar to R’s character, numeric, and logical vectors. The main difference is that JSON’s scalars can only represent a single value. To represent multiple values you need to use one of the two remaining types: arrays and objects.</p>
 | 
			
		||||
<p>Both arrays and objects are similar to lists in R; the difference is whether or not they’re named. An <strong>array</strong> is like an unnamed list, and is written with <code>[]</code>. For example <code>[1, 2, 3]</code> is an array containing 3 numbers, and <code>[null, 1, "string", false]</code> is an array that contains a null, a number, a string, and a boolean. An <strong>object</strong> is like a named list, and is written with <code><a href="#chp-https://rdrr.io/r/base/Paren" data-type="xref">#chp-https://rdrr.io/r/base/Paren</a></code>. The names (keys in JSON terminology) are strings, so must be surrounded by quotes. For example, <code>{"x": 1, "y": 2}</code> is an object that maps <code>x</code> to 1 and <code>y</code> to 2.</p>
 | 
			
		||||
<p>Both arrays and objects are similar to lists in R; the difference is whether or not they’re named. An <strong>array</strong> is like an unnamed list, and is written with <code>[]</code>. For example <code>[1, 2, 3]</code> is an array containing 3 numbers, and <code>[null, 1, "string", false]</code> is an array that contains a null, a number, a string, and a boolean. An <strong>object</strong> is like a named list, and is written with <code><a href="https://rdrr.io/r/base/Paren.html">{}</a></code>. The names (keys in JSON terminology) are strings, so must be surrounded by quotes. For example, <code>{"x": 1, "y": 2}</code> is an object that maps <code>x</code> to 1 and <code>y</code> to 2.</p>
 | 
			
		||||
</section>
 | 
			
		||||
 | 
			
		||||
<section id="jsonlite" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
jsonlite</h2>
 | 
			
		||||
<p>To convert JSON into R data structures, we recommend the jsonlite package, by Jeroen Ooms. We’ll use only two jsonlite functions: <code><a href="#chp-https://rdrr.io/pkg/jsonlite/man/read_json" data-type="xref">#chp-https://rdrr.io/pkg/jsonlite/man/read_json</a></code> and <code><a href="#chp-https://rdrr.io/pkg/jsonlite/man/read_json" data-type="xref">#chp-https://rdrr.io/pkg/jsonlite/man/read_json</a></code>. In real life, you’ll use <code><a href="#chp-https://rdrr.io/pkg/jsonlite/man/read_json" data-type="xref">#chp-https://rdrr.io/pkg/jsonlite/man/read_json</a></code> to read a JSON file from disk. For example, the repurrsive package also provides the source for <code>gh_user</code> as a JSON file and you can read it with <code><a href="#chp-https://rdrr.io/pkg/jsonlite/man/read_json" data-type="xref">#chp-https://rdrr.io/pkg/jsonlite/man/read_json</a></code>:</p>
 | 
			
		||||
<p>To convert JSON into R data structures, we recommend the jsonlite package, by Jeroen Ooms. We’ll use only two jsonlite functions: <code><a href="https://rdrr.io/pkg/jsonlite/man/read_json.html">read_json()</a></code> and <code><a href="https://rdrr.io/pkg/jsonlite/man/read_json.html">parse_json()</a></code>. In real life, you’ll use <code><a href="https://rdrr.io/pkg/jsonlite/man/read_json.html">read_json()</a></code> to read a JSON file from disk. For example, the repurrsive package also provides the source for <code>gh_user</code> as a JSON file and you can read it with <code><a href="https://rdrr.io/pkg/jsonlite/man/read_json.html">read_json()</a></code>:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit"># A path to a json file inside the package:
 | 
			
		||||
gh_users_json()
 | 
			
		||||
@@ -1020,7 +1020,7 @@ gh_users2 <- read_json(gh_users_json())
 | 
			
		||||
identical(gh_users, gh_users2)
 | 
			
		||||
#> [1] TRUE</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>In this book, I’ll also use <code><a href="#chp-https://rdrr.io/pkg/jsonlite/man/read_json" data-type="xref">#chp-https://rdrr.io/pkg/jsonlite/man/read_json</a></code>, since it takes a string containing JSON, which makes it good for generating simple examples. To get started, here’s three simple JSON datasets, starting with a number, then putting a few number in an array, then putting that array in an object:</p>
 | 
			
		||||
<p>In this book, I’ll also use <code><a href="https://rdrr.io/pkg/jsonlite/man/read_json.html">parse_json()</a></code>, since it takes a string containing JSON, which makes it good for generating simple examples. To get started, here’s three simple JSON datasets, starting with a number, then putting a few number in an array, then putting that array in an object:</p>
 | 
			
		||||
<div class="cell">
 | 
			
		||||
<pre data-type="programlisting" data-code-language="downlit">str(parse_json('1'))
 | 
			
		||||
#>  int 1
 | 
			
		||||
@@ -1036,7 +1036,7 @@ str(parse_json('{"x": [1, 2, 3]}'))
 | 
			
		||||
#>   ..$ : int 2
 | 
			
		||||
#>   ..$ : int 3</pre>
 | 
			
		||||
</div>
 | 
			
		||||
<p>jsonlite has another important function called <code><a href="#chp-https://rdrr.io/pkg/jsonlite/man/fromJSON" data-type="xref">#chp-https://rdrr.io/pkg/jsonlite/man/fromJSON</a></code>. We don’t use it here because it performs automatic simplification (<code>simplifyVector = TRUE</code>). This often works well, particularly in simple cases, but we think you’re better off doing the rectangling yourself so you know exactly what’s happening and can more easily handle the most complicated nested structures.</p>
 | 
			
		||||
<p>jsonlite has another important function called <code><a href="https://rdrr.io/pkg/jsonlite/man/fromJSON.html">fromJSON()</a></code>. We don’t use it here because it performs automatic simplification (<code>simplifyVector = TRUE</code>). This often works well, particularly in simple cases, but we think you’re better off doing the rectangling yourself so you know exactly what’s happening and can more easily handle the most complicated nested structures.</p>
 | 
			
		||||
</section>
 | 
			
		||||
 | 
			
		||||
<section id="starting-the-rectangling-process" data-type="sect2">
 | 
			
		||||
@@ -1107,7 +1107,7 @@ df |>
 | 
			
		||||
<section id="translation-challenges" data-type="sect2">
 | 
			
		||||
<h2>
 | 
			
		||||
Translation challenges</h2>
 | 
			
		||||
<p>Since JSON doesn’t have any way to represent dates or date-times, they’re often stored as ISO8601 date times in strings, and you’ll need to use <code><a href="#chp-https://readr.tidyverse.org/reference/parse_datetime" data-type="xref">#chp-https://readr.tidyverse.org/reference/parse_datetime</a></code> or <code><a href="#chp-https://readr.tidyverse.org/reference/parse_datetime" data-type="xref">#chp-https://readr.tidyverse.org/reference/parse_datetime</a></code> to turn them into the correct data structure. Similarly, JSON’s rules for representing floating point numbers in JSON are a little imprecise, so you’ll also sometimes find numbers stored in strings. Apply <code><a href="#chp-https://readr.tidyverse.org/reference/parse_atomic" data-type="xref">#chp-https://readr.tidyverse.org/reference/parse_atomic</a></code> as needed to the get correct variable type.</p>
 | 
			
		||||
<p>Since JSON doesn’t have any way to represent dates or date-times, they’re often stored as ISO8601 date times in strings, and you’ll need to use <code><a href="https://readr.tidyverse.org/reference/parse_datetime.html">readr::parse_date()</a></code> or <code><a href="https://readr.tidyverse.org/reference/parse_datetime.html">readr::parse_datetime()</a></code> to turn them into the correct data structure. Similarly, JSON’s rules for representing floating point numbers in JSON are a little imprecise, so you’ll also sometimes find numbers stored in strings. Apply <code><a href="https://readr.tidyverse.org/reference/parse_atomic.html">readr::parse_double()</a></code> as needed to the get correct variable type.</p>
 | 
			
		||||
</section>
 | 
			
		||||
 | 
			
		||||
<section id="exercises-2" data-type="sect2">
 | 
			
		||||
@@ -1140,7 +1140,7 @@ df_row <- tibble(json = json_row)</pre>
 | 
			
		||||
<section id="summary" data-type="sect1">
 | 
			
		||||
<h1>
 | 
			
		||||
Summary</h1>
 | 
			
		||||
<p>In this chapter, you learned what lists are, how you can generate the from JSON files, and how turn them into rectangular data frames. Surprisingly we only need two new functions: <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_longer" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_longer</a></code> to put list elements into rows and <code><a href="#chp-https://tidyr.tidyverse.org/reference/unnest_wider" data-type="xref">#chp-https://tidyr.tidyverse.org/reference/unnest_wider</a></code> to put list elements into columns. It doesn’t matter how deeply nested the list-column is, all you need to do is repeatedly call these two functions.</p>
 | 
			
		||||
<p>In this chapter, you learned what lists are, how you can generate the from JSON files, and how turn them into rectangular data frames. Surprisingly we only need two new functions: <code><a href="https://tidyr.tidyverse.org/reference/unnest_longer.html">unnest_longer()</a></code> to put list elements into rows and <code><a href="https://tidyr.tidyverse.org/reference/unnest_wider.html">unnest_wider()</a></code> to put list elements into columns. It doesn’t matter how deeply nested the list-column is, all you need to do is repeatedly call these two functions.</p>
 | 
			
		||||
<p>JSON is the most common data format returned by web APIs. What happens if the website doesn’t have an API, but you can see data you want on the website? That’s the topic of the next chapter: web scraping, extracting data from HTML webpages.</p>
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
 
 | 
			
		||||
		Reference in New Issue
	
	Block a user