lib/ureq
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

API changes for 2.0

Browse Source 
* Remove Request::build
* All mutations on Request follow builder pattern

The previous `build()` on request was necessary because mutating
functions did not follow a proper builder pattern (taking `&mut self`
instead of `mut self`). With a proper builder pattern, the need for
`.build()` goes away.

* All Request body and call methods consume self

Anything which "executes" the request will now consume the `Request`
to produce a `Result<Response>`.

* Move all config from request to agent builder

Timeouts, redirect config, proxy settings and TLS config are now on
`AgentBuilder`.

* Rename max_pool_connections -> max_idle_connections
* Rename max_pool_connections_per_host ->  max_idle_connections_per_host

Consistent internal and external naming.

* Introduce new AgentConfig for static config created by builder.

`Agent` can be seen as having two parts. Static config and a mutable
shared state between all states. The static config goes into
`AgentConfig` and the mutable shared state into `AgentState`.

* Replace all use of `Default` for `new`.

Deriving or implementing `Default` makes for a secondary instantiation
API.  It is useful in some cases, but gets very confusing when there
is both `new` _and_ a `Default`. It's especially devious for derived
values where a reasonable default is not `0`, `false` or `None`.

* Remove feature native_tls, we want only native rustls.

This feature made for very clunky handling throughout the code. From a
security point of view, it's better to stick with one single TLS API.
Rustls recently got an official audit (very positive).

https://github.com/ctz/rustls/tree/master/audit

Rustls deliberately omits support for older, insecure TLS such as TLS
1.1 or RC4. This might be a problem for a user of ureq, but on balance
not considered important enough to keep native_tls.

* Remove auth and support for basic auth.

The API just wasn't enough. A future reintroduction should at least
also provide a `Bearer` mechanism and possibly more.

* Rename jar -> cookie_store
* Rename jar -> cookie_tin

Just make some field names sync up with the type.

* Drop "cookies" as default feature

The need for handling cookies is probably rare, let's not enable it by
default.

* Change all feature checks for "cookie" to "cookies"

The outward facing feature is "cookies" and I think it's better form
that the code uses the official feature name instead of the optional
library "cookies".

* Keep `set` on Agent level as well as AgentBuilder.

The idea is that an auth exchange might result in a header that need
to be set _after_ the agent has been built.
This commit is contained in:
Martin Algesten
2020-10-25 11:08:50 +01:00
parent 703ca41960
commit 1369c32351
24 changed files with 398 additions and 647 deletions
Download Patch File Download Diff File Expand all files Collapse all files

332
src/agent.rs
View File

@@ -1,36 +1,45 @@
#[cfg(feature = "cookie")]
use cookie::Cookie;
#[cfg(feature = "cookie")]
use cookie_store::CookieStore;
use std::sync::Arc;
#[cfg(feature = "cookie")]
use url::Url;
#[cfg(feature = "cookie")]
use crate::cookies::CookieTin;
use crate::header::{self, Header};
use crate::pool::ConnectionPool;
use crate::proxy::Proxy;
use crate::request::Request;
use crate::resolve::ArcResolver;
use crate::resolve::{ArcResolver, StdResolver};
use std::time;
#[derive(Debug, Default)]
#[cfg(feature = "cookies")]
use crate::cookies::CookieTin;
#[cfg(feature = "cookies")]
use cookie::Cookie;
#[cfg(feature = "cookies")]
use cookie_store::CookieStore;
#[cfg(feature = "cookies")]
use url::Url;
#[derive(Debug)]
pub struct AgentBuilder {
headers: Vec<Header>,
proxy: Option<Proxy>,
max_idle_connections: usize,
max_idle_connections_per_host: usize,
config: AgentConfig,
/// Cookies saved between requests.
/// Invariant: All cookies must have a nonempty domain and path.
#[cfg(feature = "cookie")]
jar: CookieStore,
#[cfg(feature = "cookies")]
cookie_store: Option<CookieStore>,
resolver: ArcResolver,
}
impl Default for Agent {
fn default() -> Self {
AgentBuilder::new().build()
}
/// Config as built by AgentBuilder and then static for the lifetime of the Agent.
#[derive(Debug, Clone)]
pub(crate) struct AgentConfig {
pub max_idle_connections: usize,
pub max_idle_connections_per_host: usize,
pub proxy: Option<Proxy>,
pub timeout_connect: Option<time::Duration>,
pub timeout_read: Option<time::Duration>,
pub timeout_write: Option<time::Duration>,
pub timeout: Option<time::Duration>,
pub redirects: u32,
#[cfg(feature = "tls")]
pub tls_config: Option<TLSClientConfig>,
}
/// Agents keep state between requests.
@@ -40,12 +49,13 @@ impl Default for Agent {
/// can keep a state.
///
/// ```
/// let agent = ureq::agent();
/// let mut agent = ureq::agent();
///
/// agent.set("x-my-secret-header", "very secret");
///
/// let auth = agent
/// .post("/login")
/// .auth("martin", "rubbermashgum")
/// .call(); // blocks. puts auth cookies in agent.
/// .call(); // blocks.
///
/// if auth.is_err() {
/// println!("Noes!");
@@ -66,6 +76,7 @@ impl Default for Agent {
/// that shares the same underlying connection pool and other state.
#[derive(Debug, Clone)]
pub struct Agent {
pub(crate) config: Arc<AgentConfig>,
/// Copied into each request of this agent.
pub(crate) headers: Vec<Header>,
/// Reused agent state for repeated requests from this agent.
@@ -75,19 +86,47 @@ pub struct Agent {
/// Container of the state
///
/// *Internal API*.
#[derive(Debug, Default)]
#[derive(Debug)]
pub(crate) struct AgentState {
/// Reused connections between requests.
pub(crate) pool: ConnectionPool,
pub(crate) proxy: Option<Proxy>,
/// Cookies saved between requests.
/// Invariant: All cookies must have a nonempty domain and path.
#[cfg(feature = "cookie")]
pub(crate) jar: CookieTin,
#[cfg(feature = "cookies")]
pub(crate) cookie_tin: CookieTin,
pub(crate) resolver: ArcResolver,
}
impl Agent {
/// Creates an Agent with default settings.
///
/// Same as `AgentBuilder::new().build()`.
pub fn new() -> Self {
AgentBuilder::new().build()
}
/// Set a extra header field that will be present in all following requests using the agent.
///
/// This is useful for cases like auth, where we do a number of requests before getting
/// some credential that later must be presented in a header.
///
/// Notice that fixed headers can also be set in the `AgentBuilder`.
///
/// ```
/// let mut agent = ureq::agent();
///
/// agent.set("X-API-Key", "foobar");
/// agent.set("Accept", "text/plain");
///
/// let r = agent
/// .get("/my-page")
/// .call();
/// ```
pub fn set(&mut self, header: &str, value: &str) {
header::add_header(&mut self.headers, Header::new(header, value));
}
/// Request by providing the HTTP verb such as `GET`, `POST`...
///
/// ```
@@ -99,7 +138,7 @@ impl Agent {
/// println!("{:?}", r);
/// ```
pub fn request(&self, method: &str, path: &str) -> Request {
Request::new(&self, method.into(), path.into())
Request::new(self.clone(), method.into(), path.into())
}
/// Store a cookie in this agent.
@@ -112,10 +151,10 @@ impl Agent {
/// .finish();
/// agent.set_cookie(cookie, &"https://example.com/".parse().unwrap());
/// ```
#[cfg(feature = "cookie")]
#[cfg(feature = "cookies")]
pub fn set_cookie(&self, cookie: Cookie<'static>, url: &Url) {
self.state
.jar
.cookie_tin
.store_response_cookies(Some(cookie).into_iter(), url);
}
@@ -161,11 +200,24 @@ impl Agent {
}
impl AgentBuilder {
pub fn new() -> AgentBuilder {
pub fn new() -> Self {
AgentBuilder {
max_idle_connections: 100,
max_idle_connections_per_host: 1,
..Default::default()
headers: vec![],
config: AgentConfig {
max_idle_connections: crate::pool::DEFAULT_MAX_IDLE_CONNECTIONS,
max_idle_connections_per_host: crate::pool::DEFAULT_MAX_IDLE_CONNECTIONS_PER_HOST,
proxy: None,
timeout_connect: Some(time::Duration::from_secs(30)),
timeout_read: None,
timeout_write: None,
timeout: None,
redirects: 5,
#[cfg(feature = "tls")]
tls_config: None,
},
resolver: StdResolver.into(),
#[cfg(feature = "cookies")]
cookie_store: None,
}
}
@@ -175,25 +227,29 @@ impl AgentBuilder {
// not implement clone, so we have to give ownership to the newly
// built Agent.
pub fn build(self) -> Agent {
let config = Arc::new(self.config);
Agent {
headers: self.headers.clone(),
headers: self.headers,
state: Arc::new(AgentState {
pool: ConnectionPool::new(
self.max_idle_connections,
self.max_idle_connections_per_host,
pool: ConnectionPool::new_with_limits(
config.max_idle_connections,
config.max_idle_connections_per_host,
),
proxy: config.proxy.clone(),
#[cfg(feature = "cookies")]
cookie_tin: CookieTin::new(
self.cookie_store.unwrap_or_else(|| CookieStore::default()),
),
proxy: self.proxy.clone(),
#[cfg(feature = "cookie")]
jar: CookieTin::new(self.jar),
resolver: self.resolver,
}),
config,
}
}
/// Set a header field that will be present in all requests using the agent.
///
/// ```
/// let agent = ureq::AgentBuilder::new()
/// let agent = ureq::builder()
/// .set("X-API-Key", "foobar")
/// .set("Accept", "text/plain")
/// .build();
@@ -213,39 +269,18 @@ impl AgentBuilder {
self
}
/// Basic auth that will be present in all requests using the agent.
/// Set the proxy server to use for all connections from this Agent.
///
/// Example:
/// ```
/// let proxy = ureq::Proxy::new("user:password@cool.proxy:9090").unwrap();
/// let agent = ureq::AgentBuilder::new()
/// .auth("martin", "rubbermashgum")
/// .proxy(proxy)
/// .build();
///
/// let r = agent
/// .get("/my_page")
/// .call();
/// println!("{:?}", r);
/// ```
pub fn auth(self, user: &str, pass: &str) -> Self {
let pass = basic_auth(user, pass);
self.auth_kind("Basic", &pass)
}
/// Auth of other kinds such as `Digest`, `Token` etc, that will be present
/// in all requests using the agent.
///
/// ```
/// // sets a header "Authorization: token secret"
/// let agent = ureq::AgentBuilder::new()
/// .auth_kind("token", "secret")
/// .build();
///
/// let r = agent
/// .get("/my_page")
/// .call();
/// ```
pub fn auth_kind(self, kind: &str, pass: &str) -> Self {
let value = format!("{} {}", kind, pass);
self.set("Authorization", &value)
pub fn proxy(mut self, proxy: Proxy) -> Self {
self.config.proxy = Some(proxy);
self
}
/// Sets the maximum number of connections allowed in the connection pool.
@@ -253,10 +288,10 @@ impl AgentBuilder {
/// connection pooling.
///
/// ```
/// let agent = ureq::AgentBuilder::new().max_pool_connections(200).build();
/// let agent = ureq::AgentBuilder::new().max_idle_connections(200).build();
/// ```
pub fn max_pool_connections(mut self, max: usize) -> Self {
self.max_idle_connections = max;
pub fn max_idle_connections(mut self, max: usize) -> Self {
self.config.max_idle_connections = max;
self
}
@@ -265,10 +300,10 @@ impl AgentBuilder {
/// would disable connection pooling.
///
/// ```
/// let agent = ureq::AgentBuilder::new().max_pool_connections_per_host(200).build();
/// let agent = ureq::AgentBuilder::new().max_idle_connections_per_host(200).build();
/// ```
pub fn max_pool_connections_per_host(mut self, max: usize) -> Self {
self.max_idle_connections_per_host = max;
pub fn max_idle_connections_per_host(mut self, max: usize) -> Self {
self.config.max_idle_connections_per_host = max;
self
}
@@ -296,27 +331,125 @@ impl AgentBuilder {
self
}
/// Set the proxy server to use for all connections from this Agent.
/// Timeout for the socket connection to be successful.
/// If both this and `.timeout()` are both set, `.timeout_connect()`
/// takes precedence.
///
/// The default is 30 seconds.
///
/// ```
/// let agent = ureq::builder()
/// .timeout_connect(std::time::Duration::from_secs(1))
/// .build();
/// let r = agent.get("/my_page").call();
/// ```
pub fn timeout_connect(mut self, timeout: time::Duration) -> Self {
self.config.timeout_connect = Some(timeout);
self
}
/// Timeout for the individual reads of the socket.
/// If both this and `.timeout()` are both set, `.timeout()`
/// takes precedence.
///
/// The default is `0`, which means it can block forever.
///
/// ```
/// let agent = ureq::builder()
/// .timeout_read(std::time::Duration::from_secs(1))
/// .build();
/// let r = agent.get("/my_page").call();
/// ```
pub fn timeout_read(mut self, timeout: time::Duration) -> Self {
self.config.timeout_read = Some(timeout);
self
}
/// Timeout for the individual writes to the socket.
/// If both this and `.timeout()` are both set, `.timeout()`
/// takes precedence.
///
/// The default is `0`, which means it can block forever.
///
/// ```
/// let agent = ureq::builder()
/// .timeout_write(std::time::Duration::from_secs(1))
/// .build();
/// let r = agent.get("/my_page").call();
/// ```
pub fn timeout_write(mut self, timeout: time::Duration) -> Self {
self.config.timeout_write = Some(timeout);
self
}
/// Timeout for the overall request, including DNS resolution, connection
/// time, redirects, and reading the response body. Slow DNS resolution
/// may cause a request to exceed the timeout, because the DNS request
/// cannot be interrupted with the available APIs.
///
/// This takes precedence over `.timeout_read()` and `.timeout_write()`, but
/// not `.timeout_connect()`.
///
/// ```
/// // wait max 1 second for whole request to complete.
/// let agent = ureq::builder()
/// .timeout(std::time::Duration::from_secs(1))
/// .build();
/// let r = agent.get("/my_page").call();
/// ```
pub fn timeout(mut self, timeout: time::Duration) -> Self {
self.config.timeout = Some(timeout);
self
}
/// How many redirects to follow.
///
/// Defaults to `5`. Set to `0` to avoid redirects and instead
/// get a response object with the 3xx status code.
///
/// If the redirect count hits this limit (and it's > 0), TooManyRedirects is returned.
///
/// ```
/// let r = ureq::builder()
/// .redirects(10)
/// .build()
/// .get("/my_page")
/// .call();
/// println!("{:?}", r);
/// ```
pub fn redirects(mut self, n: u32) -> Self {
self.config.redirects = n;
self
}
/// Set the TLS client config to use for the connection. See [`ClientConfig`](https://docs.rs/rustls/latest/rustls/struct.ClientConfig.html).
///
/// See [`ClientConfig`](https://docs.rs/rustls/latest/rustls/struct.ClientConfig.html).
///
/// Example:
/// ```
/// let proxy = ureq::Proxy::new("user:password@cool.proxy:9090").unwrap();
/// let agent = ureq::AgentBuilder::new()
/// .proxy(proxy)
/// let tls_config = std::sync::Arc::new(rustls::ClientConfig::new());
/// let agent = ureq::builder()
/// .set_tls_config(tls_config.clone())
/// .build();
/// let req = agent.post("https://cool.server");
/// ```
pub fn proxy(mut self, proxy: Proxy) -> Self {
self.proxy = Some(proxy);
#[cfg(feature = "tls")]
pub fn set_tls_config(mut self, tls_config: Arc<rustls::ClientConfig>) -> Self {
self.config.tls_config = Some(TLSClientConfig(tls_config));
self
}
}
pub(crate) fn basic_auth(user: &str, pass: &str) -> String {
let safe = match user.find(':') {
Some(idx) => &user[..idx],
None => user,
};
base64::encode(&format!("{}:{}", safe, pass))
#[cfg(feature = "tls")]
#[derive(Clone)]
pub(crate) struct TLSClientConfig(pub(crate) Arc<rustls::ClientConfig>);
#[cfg(feature = "tls")]
impl std::fmt::Debug for TLSClientConfig {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("TLSClientConfig").finish()
}
}
#[cfg(test)]
@@ -330,31 +463,4 @@ mod tests {
let _agent: Box<dyn Send> = Box::new(AgentBuilder::new().build());
let _agent: Box<dyn Sync> = Box::new(AgentBuilder::new().build());
}
#[test]
#[cfg(any(feature = "tls", feature = "native-tls"))]
fn agent_pool() {
use std::io::Read;
let agent = crate::agent();
let url = "http://example.com";
// req 1
let resp = agent.get(url).call().unwrap();
let mut reader = resp.into_reader();
let mut buf = vec![];
// reading the entire content will return the connection to the pool
reader.read_to_end(&mut buf).unwrap();
fn poolsize(agent: &Agent) -> usize {
agent.state.pool.len()
}
assert_eq!(poolsize(&agent), 1);
// req 2 should be done with a reused connection
let resp = agent.get(url).call().unwrap();
assert_eq!(poolsize(&agent), 0);
let mut reader = resp.into_reader();
let mut buf = vec![];
reader.read_to_end(&mut buf).unwrap();
}
}