% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/request_retry.R
\name{request_retry}
\alias{request_retry}
\title{Make a Google API request, repeatedly}
\usage{
request_retry(..., max_tries_total = 5, max_total_wait_time_in_seconds = 100)
}
\arguments{
\item{...}{Passed along to \code{\link[=request_make]{request_make()}}.}

\item{max_tries_total}{Maximum number of tries.}

\item{max_total_wait_time_in_seconds}{Total seconds we are willing to
dedicate to waiting, summed across all tries. This is a technical upper
bound and actual cumulative waiting will be less.}
}
\value{
Object of class \code{response} from \link{httr}.
}
\description{
Intended primarily for internal use in client packages that provide
high-level wrappers for users. It is a drop-in substitute for
\code{\link[=request_make]{request_make()}} that also has the ability to retry the request. Codes that
are considered retryable: 408, 429, 500, 502, 503.
}
\details{
Consider an example where we are willing to make a request up to 5 times.

\if{html}{\out{<div class="sourceCode">}}\preformatted{try  1  2    3        4                5
     |--|----|--------|----------------|
wait  1   2      3           4
}\if{html}{\out{</div>}}

There will be up to 5 - 1 = 4 waits and we generally want the waiting period
to get longer, in an exponential way. Such schemes are called exponential
backoff. \code{request_retry()} implements exponential backoff with "full jitter",
where each waiting time is generated from a uniform distribution, where the
interval of support grows exponentially. A common alternative is "equal
jitter", which adds some noise to fixed, exponentially increasing waiting
times.

Either way our waiting times are based on a geometric series, which, by
convention, is usually written in terms of powers of 2:

\if{html}{\out{<div class="sourceCode">}}\preformatted{b , 2b, 4b, 8b, ...
  = b * 2^0, b * 2^1, b * 2^2, b * 2^3, ...
}\if{html}{\out{</div>}}

The terms in this series require knowledge of \code{b}, the so-called exponential
base, and many retry functions and libraries require the user to specify
this. But most users find it easier to declare the total amount of waiting
time they can tolerate for one request. Therefore \code{request_retry()} asks for
that instead and solves for \code{b} internally. This is inspired by the Opnieuw
Python library for retries. Opnieuw's interface is designed to eliminate
uncertainty around:
\itemize{
\item Units: Is this thing given in seconds? minutes? milliseconds?
\item Ambiguity around how things are counted: Are we starting at 0 or 1?
Are we counting tries or just the retries?
\item Non-intuitive required inputs, e.g., the exponential base.
}

Let \emph{n} be the total number of tries we're willing to make (the argument
\code{max_tries_total}) and let \emph{W} be the total amount of seconds we're willing
to dedicate to making and retrying this request (the argument
\code{max_total_wait_time_in_seconds}). Here's how we determine \emph{b}:

\if{html}{\out{<div class="sourceCode">}}\preformatted{sum_\{i=0\}^(n - 1) b * 2^i = W
b * sum_\{i=0\}^(n - 1) 2^i = W
       b * ( (2 ^ n) - 1) = W
                        b = W / ( (2 ^ n) - 1)
}\if{html}{\out{</div>}}
}
\section{Special cases}{

\code{request_retry()} departs from exponential backoff in three special cases:
\itemize{
\item It actually implements \emph{truncated} exponential backoff. There is a floor
and a ceiling on random wait times.
\item \code{Retry-After} header: If the response has a header named \code{Retry-After}
(case-insensitive), it is assumed to provide a non-negative integer
indicating the number of seconds to wait. If present, we wait this many
seconds and do not generate a random waiting time. (In theory, this header
can alternatively provide a datetime after which to retry, but we have no
first-hand experience with this variant for a Google API.)
\item Sheets API quota exhaustion: In the course of googlesheets4 development,
we've grown very familiar with the \verb{429 RESOURCE_EXHAUSTED} error. The
Sheets API v4 has "a limit of 500 requests per 100 seconds per project and
100 requests per 100 seconds per user. Limits for reads and writes are
tracked separately." In our experience, the "100 (read or write) requests
per 100 seconds per user" limit is the one you hit most often. If we detect
this specific failure, the first wait time is a bit more than 100 seconds,
then we revert to exponential backoff.
}
}

\examples{
\dontrun{
req <- gargle::request_build(
  method = "GET",
  path = "path/to/the/resource",
  token = "PRETEND_I_AM_TOKEN"
)
gargle::request_retry(req)
}
}
\seealso{
\itemize{
\item \url{https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/}
\item \url{https://tech.channable.com/posts/2020-02-05-opnieuw.html}
\item \url{https://github.com/channable/opnieuw}
\item \url{https://cloud.google.com/storage/docs/retry-strategy}
\item \url{https://www.rfc-editor.org/rfc/rfc7231#section-7.1.3}
\item \url{https://developers.google.com/sheets/api/limits}
\item \url{https://googleapis.dev/python/google-api-core/latest/retry.html}
}
}
