course/r4ds
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Don't transform non-crossref links

Browse Source
This commit is contained in:
Hadley Wickham
2022-11-18 10:30:32 -06:00
parent 4caea5281b
commit 78a1c12fe7
32 changed files with 693 additions and 693 deletions
Download Patch File Download Diff File Expand all files Collapse all files

80
oreilly/numbers.html
View File

@@ -12,12 +12,12 @@
<h1>
Introduction</h1>
<p>Numeric vectors are the backbone of data science, and you’ve already used them a bunch of times earlier in the book. Now it’s time to systematically survey what you can do with them in R, ensuring that you’re well situated to tackle any future problem involving numeric vectors.</p>
<p>We’ll start by giving you a couple of tools to make numbers if you have strings, and then going into a little more detail of <code><a href="#chp-https://dplyr.tidyverse.org/reference/count" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/count</a></code>. Then we’ll dive into various numeric transformations that pair well with <code><a href="#chp-https://dplyr.tidyverse.org/reference/mutate" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/mutate</a></code>, including more general transformations that can be applied to other types of vector, but are often used with numeric vectors. We’ll finish off by covering the summary functions that pair well with <code><a href="#chp-https://dplyr.tidyverse.org/reference/summarise" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/summarise</a></code> and show you how they can also be used with <code><a href="#chp-https://dplyr.tidyverse.org/reference/mutate" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/mutate</a></code>.</p>
<p>We’ll start by giving you a couple of tools to make numbers if you have strings, and then going into a little more detail of <code><a href="https://dplyr.tidyverse.org/reference/count.html">count()</a></code>. Then we’ll dive into various numeric transformations that pair well with <code><a href="https://dplyr.tidyverse.org/reference/mutate.html">mutate()</a></code>, including more general transformations that can be applied to other types of vector, but are often used with numeric vectors. We’ll finish off by covering the summary functions that pair well with <code><a href="https://dplyr.tidyverse.org/reference/summarise.html">summarise()</a></code> and show you how they can also be used with <code><a href="https://dplyr.tidyverse.org/reference/mutate.html">mutate()</a></code>.</p>
<section id="prerequisites" data-type="sect2">
<h2>
Prerequisites</h2>
<p>This chapter mostly uses functions from base R, which are available without loading any packages. But we still need the tidyverse because we’ll use these base R functions inside of tidyverse functions like <code><a href="#chp-https://dplyr.tidyverse.org/reference/mutate" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/mutate</a></code> and <code><a href="#chp-https://dplyr.tidyverse.org/reference/filter" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/filter</a></code>. Like in the last chapter, we’ll use real examples from nycflights13, as well as toy examples made with <code><a href="#chp-https://rdrr.io/r/base/c" data-type="xref">#chp-https://rdrr.io/r/base/c</a></code> and <code><a href="#chp-https://tibble.tidyverse.org/reference/tribble" data-type="xref">#chp-https://tibble.tidyverse.org/reference/tribble</a></code>.</p>
<p>This chapter mostly uses functions from base R, which are available without loading any packages. But we still need the tidyverse because we’ll use these base R functions inside of tidyverse functions like <code><a href="https://dplyr.tidyverse.org/reference/mutate.html">mutate()</a></code> and <code><a href="https://dplyr.tidyverse.org/reference/filter.html">filter()</a></code>. Like in the last chapter, we’ll use real examples from nycflights13, as well as toy examples made with <code><a href="https://rdrr.io/r/base/c.html">c()</a></code> and <code><a href="https://tibble.tidyverse.org/reference/tribble.html">tribble()</a></code>.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">library(tidyverse)
library(nycflights13)</pre>
@@ -29,13 +29,13 @@ library(nycflights13)</pre>
<h1>
Making numbers</h1>
<p>In most cases, you’ll get numbers already recorded in one of R’s numeric types: integer or double. In some cases, however, you’ll encounter them as strings, possibly because you’ve created them by pivoting from column headers or something has gone wrong in your data import process.</p>
<p>readr provides two useful functions for parsing strings into numbers: <code><a href="#chp-https://readr.tidyverse.org/reference/parse_atomic" data-type="xref">#chp-https://readr.tidyverse.org/reference/parse_atomic</a></code> and <code><a href="#chp-https://readr.tidyverse.org/reference/parse_number" data-type="xref">#chp-https://readr.tidyverse.org/reference/parse_number</a></code>. Use <code><a href="#chp-https://readr.tidyverse.org/reference/parse_atomic" data-type="xref">#chp-https://readr.tidyverse.org/reference/parse_atomic</a></code> when you have numbers that have been written as strings:</p>
<p>readr provides two useful functions for parsing strings into numbers: <code><a href="https://readr.tidyverse.org/reference/parse_atomic.html">parse_double()</a></code> and <code><a href="https://readr.tidyverse.org/reference/parse_number.html">parse_number()</a></code>. Use <code><a href="https://readr.tidyverse.org/reference/parse_atomic.html">parse_double()</a></code> when you have numbers that have been written as strings:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">x &lt;- c("1.2", "5.6", "1e3")
parse_double(x)
#&gt; [1] 1.2 5.6 1000.0</pre>
</div>
<p>Use <code><a href="#chp-https://readr.tidyverse.org/reference/parse_number" data-type="xref">#chp-https://readr.tidyverse.org/reference/parse_number</a></code> when the string contains non-numeric text that you want to ignore. This is particularly useful for currency data and percentages:</p>
<p>Use <code><a href="https://readr.tidyverse.org/reference/parse_number.html">parse_number()</a></code> when the string contains non-numeric text that you want to ignore. This is particularly useful for currency data and percentages:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">x &lt;- c("$1,234", "USD 3,513", "59%")
parse_number(x)
@@ -46,7 +46,7 @@ parse_number(x)
<section id="counts" data-type="sect1">
<h1>
Counts</h1>
<p>It’s surprising how much data science you can do with just counts and a little basic arithmetic, so dplyr strives to make counting as easy as possible with <code><a href="#chp-https://dplyr.tidyverse.org/reference/count" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/count</a></code>. This function is great for quick exploration and checks during analysis:</p>
<p>It’s surprising how much data science you can do with just counts and a little basic arithmetic, so dplyr strives to make counting as easy as possible with <code><a href="https://dplyr.tidyverse.org/reference/count.html">count()</a></code>. This function is great for quick exploration and checks during analysis:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt; count(dest)
#&gt; # A tibble: 105 × 2
@@ -60,7 +60,7 @@ Counts</h1>
#&gt; 6 AUS 2439
#&gt; # … with 99 more rows</pre>
</div>
<p>(Despite the advice in <a href="#chp-workflow-style" data-type="xref">#chp-workflow-style</a>, we usually put <code><a href="#chp-https://dplyr.tidyverse.org/reference/count" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/count</a></code> on a single line because it’s usually used at the console for a quick check that a calculation is working as expected.)</p>
<p>(Despite the advice in <a href="#chp-workflow-style" data-type="xref">#chp-workflow-style</a>, we usually put <code><a href="https://dplyr.tidyverse.org/reference/count.html">count()</a></code> on a single line because it’s usually used at the console for a quick check that a calculation is working as expected.)</p>
<p>If you want to see the most common values add <code>sort = TRUE</code>:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt; count(dest, sort = TRUE)
@@ -76,7 +76,7 @@ Counts</h1>
#&gt; # … with 99 more rows</pre>
</div>
<p>And remember that if you want to see all the values, you can use <code>|&gt; View()</code> or <code>|&gt; print(n = Inf)</code>.</p>
<p>You can perform the same computation “by hand” with <code><a href="#chp-https://dplyr.tidyverse.org/reference/group_by" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/group_by</a></code>, <code><a href="#chp-https://dplyr.tidyverse.org/reference/summarise" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/summarise</a></code> and <code><a href="#chp-https://dplyr.tidyverse.org/reference/context" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/context</a></code>. This is useful because it allows you to compute other summaries at the same time:</p>
<p>You can perform the same computation “by hand” with <code><a href="https://dplyr.tidyverse.org/reference/group_by.html">group_by()</a></code>, <code><a href="https://dplyr.tidyverse.org/reference/summarise.html">summarise()</a></code> and <code><a href="https://dplyr.tidyverse.org/reference/context.html">n()</a></code>. This is useful because it allows you to compute other summaries at the same time:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt;
group_by(dest) |&gt;
@@ -95,14 +95,14 @@ Counts</h1>
#&gt; 6 AUS 2439 6.02
#&gt; # … with 99 more rows</pre>
</div>
<p><code><a href="#chp-https://dplyr.tidyverse.org/reference/context" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/context</a></code> is a special summary function that doesn’t take any arguments and instead accesses information about the “current” group. This means that it only works inside dplyr verbs:</p>
<p><code><a href="https://dplyr.tidyverse.org/reference/context.html">n()</a></code> is a special summary function that doesn’t take any arguments and instead accesses information about the “current” group. This means that it only works inside dplyr verbs:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">n()
#&gt; Error in `n()`:
#&gt; ! Must only be used inside data-masking verbs like `mutate()`,
#&gt; `filter()`, and `group_by()`.</pre>
</div>
<p>There are a couple of variants of <code><a href="#chp-https://dplyr.tidyverse.org/reference/context" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/context</a></code> that you might find useful:</p>
<p>There are a couple of variants of <code><a href="https://dplyr.tidyverse.org/reference/context.html">n()</a></code> that you might find useful:</p>
<ul><li>
<p><code>n_distinct(x)</code> counts the number of distinct (unique) values of one or more variables. For example, we could figure out which destinations are served by the most carriers:</p>
<div class="cell">
@@ -141,7 +141,7 @@ Counts</h1>
#&gt; 6 N104UW 25157
#&gt; # … with 4,038 more rows</pre>
</div>
<p>Weighted counts are a common problem so <code><a href="#chp-https://dplyr.tidyverse.org/reference/count" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/count</a></code> has a <code>wt</code> argument that does the same thing:</p>
<p>Weighted counts are a common problem so <code><a href="https://dplyr.tidyverse.org/reference/count.html">count()</a></code> has a <code>wt</code> argument that does the same thing:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt; count(tailnum, wt = distance)
#&gt; # A tibble: 4,044 × 2
@@ -157,7 +157,7 @@ Counts</h1>
</div>
</li>
<li>
<p>You can count missing values by combining <code><a href="#chp-https://rdrr.io/r/base/sum" data-type="xref">#chp-https://rdrr.io/r/base/sum</a></code> and <code><a href="#chp-https://rdrr.io/r/base/NA" data-type="xref">#chp-https://rdrr.io/r/base/NA</a></code>. In the <code>flights</code> dataset this represents flights that are cancelled:</p>
<p>You can count missing values by combining <code><a href="https://rdrr.io/r/base/sum.html">sum()</a></code> and <code><a href="https://rdrr.io/r/base/NA.html">is.na()</a></code>. In the <code>flights</code> dataset this represents flights that are cancelled:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt;
group_by(dest) |&gt;
@@ -178,8 +178,8 @@ Counts</h1>
<section id="exercises" data-type="sect2">
<h2>
Exercises</h2>
<ol type="1"><li>How can you use <code><a href="#chp-https://dplyr.tidyverse.org/reference/count" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/count</a></code> to count the number rows with a missing value for a given variable?</li>
<li>Expand the following calls to <code><a href="#chp-https://dplyr.tidyverse.org/reference/count" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/count</a></code> to instead use <code><a href="#chp-https://dplyr.tidyverse.org/reference/group_by" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/group_by</a></code>, <code><a href="#chp-https://dplyr.tidyverse.org/reference/summarise" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/summarise</a></code>, and <code><a href="#chp-https://dplyr.tidyverse.org/reference/arrange" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/arrange</a></code>:
<ol type="1"><li>How can you use <code><a href="https://dplyr.tidyverse.org/reference/count.html">count()</a></code> to count the number rows with a missing value for a given variable?</li>
<li>Expand the following calls to <code><a href="https://dplyr.tidyverse.org/reference/count.html">count()</a></code> to instead use <code><a href="https://dplyr.tidyverse.org/reference/group_by.html">group_by()</a></code>, <code><a href="https://dplyr.tidyverse.org/reference/summarise.html">summarise()</a></code>, and <code><a href="https://dplyr.tidyverse.org/reference/arrange.html">arrange()</a></code>:
<ol type="1"><li><p><code>flights |&gt; count(dest, sort = TRUE)</code></p></li>
<li><p><code>flights |&gt; count(tailnum, wt = distance)</code></p></li>
</ol></li>
@@ -189,7 +189,7 @@ Exercises</h2>
<section id="numeric-transformations" data-type="sect1">
<h1>
Numeric transformations</h1>
<p>Transformation functions work well with <code><a href="#chp-https://dplyr.tidyverse.org/reference/mutate" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/mutate</a></code> because their output is the same length as the input. The vast majority of transformation functions are already built into base R. It’s impractical to list them all so this section will show the most useful ones. As an example, while R provides all the trigonometric functions that you might dream of, we don’t list them here because they’re rarely needed for data science.</p>
<p>Transformation functions work well with <code><a href="https://dplyr.tidyverse.org/reference/mutate.html">mutate()</a></code> because their output is the same length as the input. The vast majority of transformation functions are already built into base R. It’s impractical to list them all so this section will show the most useful ones. As an example, while R provides all the trigonometric functions that you might dream of, we don’t list them here because they’re rarely needed for data science.</p>
<section id="sec-recycling" data-type="sect2">
<h2>
@@ -232,13 +232,13 @@ x * c(1, 2, 3)
#&gt; # ¹​sched_dep_time, ²​dep_delay, ³​arr_time, ⁴​sched_arr_time, ⁵​arr_delay</pre>
</div>
<p>The code runs without error, but it doesn’t return what you want. Because of the recycling rules it finds flights in odd numbered rows that departed in January and flights in even numbered rows that departed in February. And unforuntately there’s no warning because <code>flights</code> has an even number of rows.</p>
<p>To protect you from this type of silent failure, most tidyverse functions use a stricter form of recycling that only recycles single values. Unfortunately that doesn’t help here, or in many other cases, because the key computation is performed by the base R function <code>==</code>, not <code><a href="#chp-https://dplyr.tidyverse.org/reference/filter" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/filter</a></code>.</p>
<p>To protect you from this type of silent failure, most tidyverse functions use a stricter form of recycling that only recycles single values. Unfortunately that doesn’t help here, or in many other cases, because the key computation is performed by the base R function <code>==</code>, not <code><a href="https://dplyr.tidyverse.org/reference/filter.html">filter()</a></code>.</p>
</section>
<section id="minimum-and-maximum" data-type="sect2">
<h2>
Minimum and maximum</h2>
<p>The arithmetic functions work with pairs of variables. Two closely related functions are <code><a href="#chp-https://rdrr.io/r/base/Extremes" data-type="xref">#chp-https://rdrr.io/r/base/Extremes</a></code> and <code><a href="#chp-https://rdrr.io/r/base/Extremes" data-type="xref">#chp-https://rdrr.io/r/base/Extremes</a></code>, which when given two or more variables will return the smallest or largest value in each row:</p>
<p>The arithmetic functions work with pairs of variables. Two closely related functions are <code><a href="https://rdrr.io/r/base/Extremes.html">pmin()</a></code> and <code><a href="https://rdrr.io/r/base/Extremes.html">pmax()</a></code>, which when given two or more variables will return the smallest or largest value in each row:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">df &lt;- tribble(
~x, ~y,
@@ -259,7 +259,7 @@ df |&gt;
#&gt; 2 5 2 2 5
#&gt; 3 7 NA 7 7</pre>
</div>
<p>Note that these are different to the summary functions <code><a href="#chp-https://rdrr.io/r/base/Extremes" data-type="xref">#chp-https://rdrr.io/r/base/Extremes</a></code> and <code><a href="#chp-https://rdrr.io/r/base/Extremes" data-type="xref">#chp-https://rdrr.io/r/base/Extremes</a></code> which take multiple observations and return a single value. You can tell that you’ve used the wrong form when all the minimums and all the maximums have the same value:</p>
<p>Note that these are different to the summary functions <code><a href="https://rdrr.io/r/base/Extremes.html">min()</a></code> and <code><a href="https://rdrr.io/r/base/Extremes.html">max()</a></code> which take multiple observations and return a single value. You can tell that you’ve used the wrong form when all the minimums and all the maximums have the same value:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">df |&gt;
mutate(
@@ -353,8 +353,8 @@ money &lt;- tibble(
</div>
</div>
<p>This a straight line because a little algebra reveals that <code>log(money) = log(starting) + n * log(interest)</code>, which matches the pattern for a line, <code>y = m * x + b</code>. This is a useful pattern: if you see a (roughly) straight line after log-transforming the y-axis, you know that there’s underlying exponential growth.</p>
<p>If you’re log-transforming your data with dplyr you have a choice of three logarithms provided by base R: <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code> (the natural log, base e), <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code> (base 2), and <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code> (base 10). We recommend using <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code> or <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code>. <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code> is easy to interpret because difference of 1 on the log scale corresponds to doubling on the original scale and a difference of -1 corresponds to halving; whereas <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code> is easy to back-transform because (e.g) 3 is 10^3 = 1000.</p>
<p>The inverse of <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code> is <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code>; to compute the inverse of <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code> or <code><a href="#chp-https://rdrr.io/r/base/Log" data-type="xref">#chp-https://rdrr.io/r/base/Log</a></code> you’ll need to use <code>2^</code> or <code>10^</code>.</p>
<p>If you’re log-transforming your data with dplyr you have a choice of three logarithms provided by base R: <code><a href="https://rdrr.io/r/base/Log.html">log()</a></code> (the natural log, base e), <code><a href="https://rdrr.io/r/base/Log.html">log2()</a></code> (base 2), and <code><a href="https://rdrr.io/r/base/Log.html">log10()</a></code> (base 10). We recommend using <code><a href="https://rdrr.io/r/base/Log.html">log2()</a></code> or <code><a href="https://rdrr.io/r/base/Log.html">log10()</a></code>. <code><a href="https://rdrr.io/r/base/Log.html">log2()</a></code> is easy to interpret because difference of 1 on the log scale corresponds to doubling on the original scale and a difference of -1 corresponds to halving; whereas <code><a href="https://rdrr.io/r/base/Log.html">log10()</a></code> is easy to back-transform because (e.g) 3 is 10^3 = 1000.</p>
<p>The inverse of <code><a href="https://rdrr.io/r/base/Log.html">log()</a></code> is <code><a href="https://rdrr.io/r/base/Log.html">exp()</a></code>; to compute the inverse of <code><a href="https://rdrr.io/r/base/Log.html">log2()</a></code> or <code><a href="https://rdrr.io/r/base/Log.html">log10()</a></code> you’ll need to use <code>2^</code> or <code>10^</code>.</p>
</section>
<section id="sec-rounding" data-type="sect2">
@@ -376,13 +376,13 @@ round(123.456, -1) # round to nearest ten
round(123.456, -2) # round to nearest hundred
#&gt; [1] 100</pre>
</div>
<p>There’s one weirdness with <code><a href="#chp-https://rdrr.io/r/base/Round" data-type="xref">#chp-https://rdrr.io/r/base/Round</a></code> that seems surprising at first glance:</p>
<p>There’s one weirdness with <code><a href="https://rdrr.io/r/base/Round.html">round()</a></code> that seems surprising at first glance:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">round(c(1.5, 2.5))
#&gt; [1] 2 2</pre>
</div>
<p><code><a href="#chp-https://rdrr.io/r/base/Round" data-type="xref">#chp-https://rdrr.io/r/base/Round</a></code> uses what’s known as “round half to even” or Banker’s rounding: if a number is half way between two integers, it will be rounded to the <strong>even</strong> integer. This is a good strategy because it keeps the rounding unbiased: half of all 0.5s are rounded up, and half are rounded down.</p>
<p><code><a href="#chp-https://rdrr.io/r/base/Round" data-type="xref">#chp-https://rdrr.io/r/base/Round</a></code> is paired with <code><a href="#chp-https://rdrr.io/r/base/Round" data-type="xref">#chp-https://rdrr.io/r/base/Round</a></code> which always rounds down and <code><a href="#chp-https://rdrr.io/r/base/Round" data-type="xref">#chp-https://rdrr.io/r/base/Round</a></code> which always rounds up:</p>
<p><code><a href="https://rdrr.io/r/base/Round.html">round()</a></code> uses what’s known as “round half to even” or Banker’s rounding: if a number is half way between two integers, it will be rounded to the <strong>even</strong> integer. This is a good strategy because it keeps the rounding unbiased: half of all 0.5s are rounded up, and half are rounded down.</p>
<p><code><a href="https://rdrr.io/r/base/Round.html">round()</a></code> is paired with <code><a href="https://rdrr.io/r/base/Round.html">floor()</a></code> which always rounds down and <code><a href="https://rdrr.io/r/base/Round.html">ceiling()</a></code> which always rounds up:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">x &lt;- 123.456
@@ -400,7 +400,7 @@ floor(x / 0.01) * 0.01
ceiling(x / 0.01) * 0.01
#&gt; [1] 123.46</pre>
</div>
<p>You can use the same technique if you want to <code><a href="#chp-https://rdrr.io/r/base/Round" data-type="xref">#chp-https://rdrr.io/r/base/Round</a></code> to a multiple of some other number:</p>
<p>You can use the same technique if you want to <code><a href="https://rdrr.io/r/base/Round.html">round()</a></code> to a multiple of some other number:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit"># Round to nearest multiple of 4
round(x / 4) * 4
@@ -415,7 +415,7 @@ round(x / 0.25) * 0.25
<section id="cutting-numbers-into-ranges" data-type="sect2">
<h2>
Cutting numbers into ranges</h2>
<p>Use <code><a href="#chp-https://rdrr.io/r/base/cut" data-type="xref">#chp-https://rdrr.io/r/base/cut</a></code><span data-type="footnote">ggplot2 provides some helpers for common cases in <code><a href="#chp-https://ggplot2.tidyverse.org/reference/cut_interval" data-type="xref">#chp-https://ggplot2.tidyverse.org/reference/cut_interval</a></code>, <code><a href="#chp-https://ggplot2.tidyverse.org/reference/cut_interval" data-type="xref">#chp-https://ggplot2.tidyverse.org/reference/cut_interval</a></code>, and <code><a href="#chp-https://ggplot2.tidyverse.org/reference/cut_interval" data-type="xref">#chp-https://ggplot2.tidyverse.org/reference/cut_interval</a></code>. ggplot2 is an admittedly weird place for these functions to live, but they are useful as part of histogram computation and were written before any other parts of the tidyverse existed.</span> to break up a numeric vector into discrete buckets:</p>
<p>Use <code><a href="https://rdrr.io/r/base/cut.html">cut()</a></code><span data-type="footnote">ggplot2 provides some helpers for common cases in <code><a href="https://ggplot2.tidyverse.org/reference/cut_interval.html">cut_interval()</a></code>, <code><a href="https://ggplot2.tidyverse.org/reference/cut_interval.html">cut_number()</a></code>, and <code><a href="https://ggplot2.tidyverse.org/reference/cut_interval.html">cut_width()</a></code>. ggplot2 is an admittedly weird place for these functions to live, but they are useful as part of histogram computation and were written before any other parts of the tidyverse existed.</span> to break up a numeric vector into discrete buckets:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">x &lt;- c(1, 2, 5, 10, 15, 20)
cut(x, breaks = c(0, 5, 10, 15, 20))
@@ -450,13 +450,13 @@ cut(y, breaks = c(0, 5, 10, 15, 20))
<section id="cumulative-and-rolling-aggregates" data-type="sect2">
<h2>
Cumulative and rolling aggregates</h2>
<p>Base R provides <code><a href="#chp-https://rdrr.io/r/base/cumsum" data-type="xref">#chp-https://rdrr.io/r/base/cumsum</a></code>, <code><a href="#chp-https://rdrr.io/r/base/cumsum" data-type="xref">#chp-https://rdrr.io/r/base/cumsum</a></code>, <code><a href="#chp-https://rdrr.io/r/base/cumsum" data-type="xref">#chp-https://rdrr.io/r/base/cumsum</a></code>, <code><a href="#chp-https://rdrr.io/r/base/cumsum" data-type="xref">#chp-https://rdrr.io/r/base/cumsum</a></code> for running, or cumulative, sums, products, mins and maxes. dplyr provides <code><a href="#chp-https://dplyr.tidyverse.org/reference/cumall" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/cumall</a></code> for cumulative means. Cumulative sums tend to come up the most in practice:</p>
<p>Base R provides <code><a href="https://rdrr.io/r/base/cumsum.html">cumsum()</a></code>, <code><a href="https://rdrr.io/r/base/cumsum.html">cumprod()</a></code>, <code><a href="https://rdrr.io/r/base/cumsum.html">cummin()</a></code>, <code><a href="https://rdrr.io/r/base/cumsum.html">cummax()</a></code> for running, or cumulative, sums, products, mins and maxes. dplyr provides <code><a href="https://dplyr.tidyverse.org/reference/cumall.html">cummean()</a></code> for cumulative means. Cumulative sums tend to come up the most in practice:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">x &lt;- 1:10
cumsum(x)
#&gt; [1] 1 3 6 10 15 21 28 36 45 55</pre>
</div>
<p>If you need more complex rolling or sliding aggregates, try the <a href="#chp-https://davisvaughan.github.io/slider/" data-type="xref">#chp-https://davisvaughan.github.io/slider/</a> package by Davis Vaughan. The following example illustrates some of its features.</p>
<p>If you need more complex rolling or sliding aggregates, try the <a href="https://davisvaughan.github.io/slider/">slider</a> package by Davis Vaughan. The following example illustrates some of its features.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">library(slider)
@@ -505,7 +505,7 @@ General transformations</h1>
<section id="ranks" data-type="sect2">
<h2>
Ranks</h2>
<p>dplyr provides a number of ranking functions inspired by SQL, but you should always start with <code><a href="#chp-https://dplyr.tidyverse.org/reference/row_number" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/row_number</a></code>. It uses the typical method for dealing with ties, e.g. 1st, 2nd, 2nd, 4th.</p>
<p>dplyr provides a number of ranking functions inspired by SQL, but you should always start with <code><a href="https://dplyr.tidyverse.org/reference/row_number.html">dplyr::min_rank()</a></code>. It uses the typical method for dealing with ties, e.g. 1st, 2nd, 2nd, 4th.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">x &lt;- c(1, 2, 2, 3, 4, NA)
min_rank(x)
@@ -516,7 +516,7 @@ min_rank(x)
<pre data-type="programlisting" data-code-language="downlit">min_rank(desc(x))
#&gt; [1] 5 3 3 2 1 NA</pre>
</div>
<p>If <code><a href="#chp-https://dplyr.tidyverse.org/reference/row_number" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/row_number</a></code> doesn’t do what you need, look at the variants <code><a href="#chp-https://dplyr.tidyverse.org/reference/row_number" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/row_number</a></code>, <code><a href="#chp-https://dplyr.tidyverse.org/reference/row_number" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/row_number</a></code>, <code><a href="#chp-https://dplyr.tidyverse.org/reference/percent_rank" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/percent_rank</a></code>, and <code><a href="#chp-https://dplyr.tidyverse.org/reference/percent_rank" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/percent_rank</a></code>. See the documentation for details.</p>
<p>If <code><a href="https://dplyr.tidyverse.org/reference/row_number.html">min_rank()</a></code> doesn’t do what you need, look at the variants <code><a href="https://dplyr.tidyverse.org/reference/row_number.html">dplyr::row_number()</a></code>, <code><a href="https://dplyr.tidyverse.org/reference/row_number.html">dplyr::dense_rank()</a></code>, <code><a href="https://dplyr.tidyverse.org/reference/percent_rank.html">dplyr::percent_rank()</a></code>, and <code><a href="https://dplyr.tidyverse.org/reference/percent_rank.html">dplyr::cume_dist()</a></code>. See the documentation for details.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">df &lt;- tibble(x = x)
df |&gt;
@@ -536,8 +536,8 @@ df |&gt;
#&gt; 5 4 5 4 1 1
#&gt; 6 NA NA NA NA NA</pre>
</div>
<p>You can achieve many of the same results by picking the appropriate <code>ties.method</code> argument to base R’s <code><a href="#chp-https://rdrr.io/r/base/rank" data-type="xref">#chp-https://rdrr.io/r/base/rank</a></code>; you’ll probably also want to set <code>na.last = "keep"</code> to keep <code>NA</code>s as <code>NA</code>.</p>
<p><code><a href="#chp-https://dplyr.tidyverse.org/reference/row_number" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/row_number</a></code> can also be used without any arguments when inside a dplyr verb. In this case, it’ll give the number of the “current” row. When combined with <code>%%</code> or <code>%/%</code> this can be a useful tool for dividing data into similarly sized groups:</p>
<p>You can achieve many of the same results by picking the appropriate <code>ties.method</code> argument to base R’s <code><a href="https://rdrr.io/r/base/rank.html">rank()</a></code>; you’ll probably also want to set <code>na.last = "keep"</code> to keep <code>NA</code>s as <code>NA</code>.</p>
<p><code><a href="https://dplyr.tidyverse.org/reference/row_number.html">row_number()</a></code> can also be used without any arguments when inside a dplyr verb. In this case, it’ll give the number of the “current” row. When combined with <code>%%</code> or <code>%/%</code> this can be a useful tool for dividing data into similarly sized groups:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">df &lt;- tibble(x = runif(10))
@@ -563,7 +563,7 @@ df |&gt;
<section id="offsets" data-type="sect2">
<h2>
Offsets</h2>
<p><code><a href="#chp-https://dplyr.tidyverse.org/reference/lead-lag" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/lead-lag</a></code> and <code><a href="#chp-https://dplyr.tidyverse.org/reference/lead-lag" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/lead-lag</a></code> allow you to refer the values just before or just after the “current” value. They return a vector of the same length as the input, padded with <code>NA</code>s at the start or end:</p>
<p><code><a href="https://dplyr.tidyverse.org/reference/lead-lag.html">dplyr::lead()</a></code> and <code><a href="https://dplyr.tidyverse.org/reference/lead-lag.html">dplyr::lag()</a></code> allow you to refer the values just before or just after the “current” value. They return a vector of the same length as the input, padded with <code>NA</code>s at the start or end:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">x &lt;- c(2, 5, 11, 11, 19, 35)
lag(x)
@@ -591,13 +591,13 @@ lead(x)
<section id="exercises-2" data-type="sect2">
<h2>
Exercises</h2>
<ol type="1"><li><p>Find the 10 most delayed flights using a ranking function. How do you want to handle ties? Carefully read the documentation for <code><a href="#chp-https://dplyr.tidyverse.org/reference/row_number" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/row_number</a></code>.</p></li>
<ol type="1"><li><p>Find the 10 most delayed flights using a ranking function. How do you want to handle ties? Carefully read the documentation for <code><a href="https://dplyr.tidyverse.org/reference/row_number.html">min_rank()</a></code>.</p></li>
<li><p>Which plane (<code>tailnum</code>) has the worst on-time record?</p></li>
<li><p>What time of day should you fly if you want to avoid delays as much as possible?</p></li>
<li><p>What does <code>flights |&gt; group_by(dest() |&gt; filter(row_number() &lt; 4)</code> do? What does <code>flights |&gt; group_by(dest() |&gt; filter(row_number(dep_delay) &lt; 4)</code> do?</p></li>
<li><p>For each destination, compute the total minutes of delay. For each flight, compute the proportion of the total delay for its destination.</p></li>
<li>
<p>Delays are typically temporally correlated: even once the problem that caused the initial delay has been resolved, later flights are delayed to allow earlier flights to leave. Using <code><a href="#chp-https://dplyr.tidyverse.org/reference/lead-lag" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/lead-lag</a></code>, explore how the average flight delay for an hour is related to the average delay for the previous hour.</p>
<p>Delays are typically temporally correlated: even once the problem that caused the initial delay has been resolved, later flights are delayed to allow earlier flights to leave. Using <code><a href="https://dplyr.tidyverse.org/reference/lead-lag.html">lag()</a></code>, explore how the average flight delay for an hour is related to the average delay for the previous hour.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt;
mutate(hour = dep_time %/% 100) |&gt;
@@ -623,7 +623,7 @@ Numeric summaries</h1>
<section id="center" data-type="sect2">
<h2>
Center</h2>
<p>So far, we’ve mostly used <code><a href="#chp-https://rdrr.io/r/base/mean" data-type="xref">#chp-https://rdrr.io/r/base/mean</a></code> to summarize the center of a vector of values. Because the mean is the sum divided by the count, it is sensitive to even just a few unusually high or low values. An alternative is to use the <code><a href="#chp-https://rdrr.io/r/stats/median" data-type="xref">#chp-https://rdrr.io/r/stats/median</a></code>, which finds a value that lies in the “middle” of the vector, i.e. 50% of the values is above it and 50% are below it. Depending on the shape of the distribution of the variable you’re interested in, mean or median might be a better measure of center. For example, for symmetric distributions we generally report the mean while for skewed distributions we usually report the median.</p>
<p>So far, we’ve mostly used <code><a href="https://rdrr.io/r/base/mean.html">mean()</a></code> to summarize the center of a vector of values. Because the mean is the sum divided by the count, it is sensitive to even just a few unusually high or low values. An alternative is to use the <code><a href="https://rdrr.io/r/stats/median.html">median()</a></code>, which finds a value that lies in the “middle” of the vector, i.e. 50% of the values is above it and 50% are below it. Depending on the shape of the distribution of the variable you’re interested in, mean or median might be a better measure of center. For example, for symmetric distributions we generally report the mean while for skewed distributions we usually report the median.</p>
<p><a href="#fig-mean-vs-median" data-type="xref">#fig-mean-vs-median</a> compares the mean vs the median when looking at the hourly vs median departure delay. The median delay is always smaller than the mean delay because because flights sometimes leave multiple hours late, but never leave multiple hours early.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt;
@@ -646,13 +646,13 @@ Center</h2>
</figure>
</div>
</div>
<p>You might also wonder about the <strong>mode</strong>, or the most common value. This is a summary that only works well for very simple cases (which is why you might have learned about it in high school), but it doesn’t work well for many real datasets. If the data is discrete, there may be multiple most common values, and if the data is continuous, there might be no most common value because every value is ever so slightly different. For these reasons, the mode tends not to be used by statisticians and there’s no mode function included in base R<span data-type="footnote">The <code><a href="#chp-https://rdrr.io/r/base/mode" data-type="xref">#chp-https://rdrr.io/r/base/mode</a></code> function does something quite different!</span>.</p>
<p>You might also wonder about the <strong>mode</strong>, or the most common value. This is a summary that only works well for very simple cases (which is why you might have learned about it in high school), but it doesn’t work well for many real datasets. If the data is discrete, there may be multiple most common values, and if the data is continuous, there might be no most common value because every value is ever so slightly different. For these reasons, the mode tends not to be used by statisticians and there’s no mode function included in base R<span data-type="footnote">The <code><a href="https://rdrr.io/r/base/mode.html">mode()</a></code> function does something quite different!</span>.</p>
</section>
<section id="sec-min-max-summary" data-type="sect2">
<h2>
Minimum, maximum, and quantiles</h2>
<p>What if you’re interested in locations other than the center? <code><a href="#chp-https://rdrr.io/r/base/Extremes" data-type="xref">#chp-https://rdrr.io/r/base/Extremes</a></code> and <code><a href="#chp-https://rdrr.io/r/base/Extremes" data-type="xref">#chp-https://rdrr.io/r/base/Extremes</a></code> will give you the largest and smallest values. Another powerful tool is <code><a href="#chp-https://rdrr.io/r/stats/quantile" data-type="xref">#chp-https://rdrr.io/r/stats/quantile</a></code> which is a generalization of the median: <code>quantile(x, 0.25)</code> will find the value of <code>x</code> that is greater than 25% of the values, <code>quantile(x, 0.5)</code> is equivalent to the median, and <code>quantile(x, 0.95)</code> will find a value that’s greater than 95% of the values.</p>
<p>What if you’re interested in locations other than the center? <code><a href="https://rdrr.io/r/base/Extremes.html">min()</a></code> and <code><a href="https://rdrr.io/r/base/Extremes.html">max()</a></code> will give you the largest and smallest values. Another powerful tool is <code><a href="https://rdrr.io/r/stats/quantile.html">quantile()</a></code> which is a generalization of the median: <code>quantile(x, 0.25)</code> will find the value of <code>x</code> that is greater than 25% of the values, <code>quantile(x, 0.5)</code> is equivalent to the median, and <code>quantile(x, 0.95)</code> will find a value that’s greater than 95% of the values.</p>
<p>For the <code>flights</code> data, you might want to look at the 95% quantile of delays rather than the maximum, because it will ignore the 5% of most delayed flights which can be quite extreme.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt;
@@ -678,8 +678,8 @@ Minimum, maximum, and quantiles</h2>
<section id="spread" data-type="sect2">
<h2>
Spread</h2>
<p>Sometimes you’re not so interested in where the bulk of the data lies, but in how it is spread out. Two commonly used summaries are the standard deviation, <code>sd(x)</code>, and the inter-quartile range, <code><a href="#chp-https://rdrr.io/r/stats/IQR" data-type="xref">#chp-https://rdrr.io/r/stats/IQR</a></code>. We won’t explain <code><a href="#chp-https://rdrr.io/r/stats/sd" data-type="xref">#chp-https://rdrr.io/r/stats/sd</a></code> here since you’re probably already familiar with it, but <code><a href="#chp-https://rdrr.io/r/stats/IQR" data-type="xref">#chp-https://rdrr.io/r/stats/IQR</a></code> might be new — it’s <code>quantile(x, 0.75) - quantile(x, 0.25)</code> and gives you the range that contains the middle 50% of the data.</p>
<p>We can use this to reveal a small oddity in the <code>flights</code> data. You might expect the spread of the distance between origin and destination to be zero, since airports are always in the same place. But the code below makes it looks like one airport, <a href="#chp-https://en.wikipedia.org/wiki/Eagle_County_Regional_Airport" data-type="xref">#chp-https://en.wikipedia.org/wiki/Eagle_County_Regional_Airport</a>, might have moved.</p>
<p>Sometimes you’re not so interested in where the bulk of the data lies, but in how it is spread out. Two commonly used summaries are the standard deviation, <code>sd(x)</code>, and the inter-quartile range, <code><a href="https://rdrr.io/r/stats/IQR.html">IQR()</a></code>. We won’t explain <code><a href="https://rdrr.io/r/stats/sd.html">sd()</a></code> here since you’re probably already familiar with it, but <code><a href="https://rdrr.io/r/stats/IQR.html">IQR()</a></code> might be new — it’s <code>quantile(x, 0.75) - quantile(x, 0.25)</code> and gives you the range that contains the middle 50% of the data.</p>
<p>We can use this to reveal a small oddity in the <code>flights</code> data. You might expect the spread of the distance between origin and destination to be zero, since airports are always in the same place. But the code below makes it looks like one airport, <a href="https://en.wikipedia.org/wiki/Eagle_County_Regional_Airport">EGE</a>, might have moved.</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt;
group_by(origin, dest) |&gt;
@@ -774,7 +774,7 @@ Positions</h2>
#&gt; # … with 359 more rows</pre>
</div>
<p>(These functions currently lack an <code>na.rm</code> argument but will hopefully be fixed by the time you read this book: <a href="https://github.com/tidyverse/dplyr/issues/6242" class="uri">https://github.com/tidyverse/dplyr/issues/6242</a>).</p>
<p>If you’re familiar with <code>[</code>, you might wonder if you ever need these functions. There are two main reasons: the <code>default</code> argument and the <code>order_by</code> argument. <code>default</code> allows you to set a default value that’s used if the requested position doesn’t exist, e.g. you’re trying to get the 3rd element from a two element group. <code>order_by</code> lets you locally override the existing ordering of the rows, so you can get the element at the position in the ordering by <code><a href="#chp-https://dplyr.tidyverse.org/reference/order_by" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/order_by</a></code>.</p>
<p>If you’re familiar with <code>[</code>, you might wonder if you ever need these functions. There are two main reasons: the <code>default</code> argument and the <code>order_by</code> argument. <code>default</code> allows you to set a default value that’s used if the requested position doesn’t exist, e.g. you’re trying to get the 3rd element from a two element group. <code>order_by</code> lets you locally override the existing ordering of the rows, so you can get the element at the position in the ordering by <code><a href="https://dplyr.tidyverse.org/reference/order_by.html">order_by()</a></code>.</p>
<p>Extracting values at positions is complementary to filtering on ranks. Filtering gives you all variables, with each observation in a separate row:</p>
<div class="cell">
<pre data-type="programlisting" data-code-language="downlit">flights |&gt;
@@ -802,7 +802,7 @@ Positions</h2>
<h2>
With<code>mutate()</code>
</h2>
<p>As the names suggest, the summary functions are typically paired with <code><a href="#chp-https://dplyr.tidyverse.org/reference/summarise" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/summarise</a></code>. However, because of the recycling rules we discussed in <a href="#sec-recycling" data-type="xref">#sec-recycling</a> they can also be usefully paired with <code><a href="#chp-https://dplyr.tidyverse.org/reference/mutate" data-type="xref">#chp-https://dplyr.tidyverse.org/reference/mutate</a></code>, particularly when you want do some sort of group standardization. For example:</p>
<p>As the names suggest, the summary functions are typically paired with <code><a href="https://dplyr.tidyverse.org/reference/summarise.html">summarise()</a></code>. However, because of the recycling rules we discussed in <a href="#sec-recycling" data-type="xref">#sec-recycling</a> they can also be usefully paired with <code><a href="https://dplyr.tidyverse.org/reference/mutate.html">mutate()</a></code>, particularly when you want do some sort of group standardization. For example:</p>
<ul><li>
<code>x / sum(x)</code> calculates the proportion of a total.</li>
<li>