Commit Graph

620 Commits

Author SHA1 Message Date
Jacob Hoffman-Andrews
b01be808c5 Fix set_max_idle_connections(0).
This broke during some recent refactorings. The special case branch for
max_connections == 0 wasn't actually setting the max_idle_connections
field.

This change simply deletes that branch, since the existing code that
whittles down the pool size one element at a time suffices. In the
common case this will be set on an empty pool anyhow.
2020-09-27 10:20:30 +02:00
Ulrik Mikaelsson
11413726cd Implement Pluggable Name-resolution (#148)
This defines a new trait `Resolver`, which turns an address into a
Vec<SocketAddr>. It also provides an implementation of Resolver for
`Fn(&str)` so it's easy to define simple resolvers with a closure.


Fixes #82

Co-authored-by: Ulrik <ulrikm@spotify.com>
2020-09-26 16:35:13 -07:00
Daniel Rivas
8bba07a9af Add req field to Unit and remove cloned parts from request (#158)
Instead of cloning most of `Request`'s fields individually when
creating a `Unit`, this PR switches to just cloning `Request` and
stuffing it in `Unit`, and changes references to `unit.[field]` to
`unit.req.[field]` where appropriate.

Fixes #155
2020-09-26 10:22:10 -07:00
Jacob Hoffman-Andrews
be9e3ca936 Update comment about retries.
This comment previously assumed we would have at most one pooled
connection per host. That's no longer true, so update the comment.
2020-09-21 08:14:28 +02:00
Martin Algesten
155edeef19 cargo fmt (#151) 2020-09-19 00:28:39 -07:00
Jacob Hoffman-Andrews
b87299e988 Document and tidy up new Pool invariants. (#142)
Now that we allow multiple connections for the same PoolKey, update our
invariants accordingly. Also provide a couple of helper functions for
removing the first or last match of an entry in a VecDeque.

This also changes which entry from `lru` gets removed when a stream is
removed from the pool. Previously it was the oldest matching one. Now it's
the newest matching one, which matches the semantics we are applying to
`recycle[K]`.

Followup to #133 

/cc @cfal
2020-09-18 19:35:22 -07:00
Martin Algesten
05ffe53e4c CI: error on dead/unused code for features (#146)
By using `RUSTFLAGS="-D dead_code -D unused-variables -D unused"`, we tell
the compiler to upgrade warnings for unused code to errors. This combined
with a more elaborate test matrix means we can have the CI catch feature
flag combos that cause warnings.
2020-09-18 13:55:23 -07:00
Ulrik Mikaelsson
f599828c6d Turn Option<AgentState> into AgentState (#149)
`Some(AgentState)` seem to be assumed pretty much everywhere. I could not
find any test or piece of code hinting at how `None` should be interpreted,
or even see how a state of `None` could even be constructed.

Co-authored-by: Ulrik <ulrikm@spotify.com>
2020-09-18 13:47:11 -07:00
Ulrik
90d0b35ae5 ConnectionPool::default: Explicit implementation
The auto-derived implementation would have unexpectedly initialized the
`usize` fields to 0. Having a `::default()` leading to an unuseable value
is counter-intuitive.
2020-09-18 07:40:23 +02:00
Jacob Hoffman-Andrews
c343587afb Use #[cfg(feature = "cookie")] for some functions.
Some helper functions introduced as part of the cookies_on_redirect
test are only used by that test, which means they are only used when the
cookie feature is enabled. This causes clippy warnings for some of the
other feature combinations in CI.
2020-09-14 08:58:02 +02:00
Martin Algesten
b8b67a41a9 1.4.1 2020-09-13 21:46:49 +02:00
Martin Algesten
64ab65ba35 CHANGELOG 1.4.1 2020-09-13 21:46:49 +02:00
Jacob Hoffman-Andrews
f38c203215 Fix cookies_on_redirect unittest.
Previously this test relied on a specific ordering of cookies, which was
not guaranteed. This makes the test robust to random ordering.

Also, fix the check for the `second` cookie to actually check the right
cookie.
2020-09-13 21:14:40 +02:00
Jacob Hoffman-Andrews
5cb212f7b7 Put the crate version in the User-Agent header. (#138)
* Put the crate version in the User-Agent header.

https://tools.ietf.org/html/rfc7231#section-5.5.3
Each product identifier consists of a name and optional version.

     product         = token ["/" product-version]
     product-version = token

* User-Agent of rust-ureq

* Update src/unit.rs

Co-authored-by: Martin Algesten <martin@algesten.se>
2020-09-13 14:13:54 +02:00
Jacob Hoffman-Andrews
99cdff7519 Add unittest for cookie handling on redirects. 2020-09-13 14:12:17 +02:00
Alex L
d52fa78ebc Allow saving multiple idle streams per host. (#133)
Adds set_max_idle_connections and set_max_idle_connections_per_host.

This turns the values of Pool.recycle into a VecDeque of Streams for the same PoolKey.
The freshest stream (most recently used) is at the back; the stalest stream is at the front.

This also removes the invariant "Each PoolKey exists in recycle at most once and lru at
most once," replacing it with "each PoolKey has the same number of entries in lru as in
recycle."

Fixes #110
2020-09-12 23:16:27 -07:00
Jacob Hoffman-Andrews
6a88c2c8bf Clean up unused code and long imports. (#137)
This removes some commented out methods, and also changes instances of
::std::foo to use a more idiomatic import path.
2020-09-12 18:42:15 -07:00
Martin Algesten
50c19c5484 Read buffer to avoid byte-by-byte syscalls (#141)
Fixes #140
2020-09-12 18:27:15 -07:00
Martin Algesten
960c0ff43b 1.4.0 2020-08-14 23:08:08 +02:00
Martin Algesten
29782bc97d Add CHANGELOG.md 2020-08-14 23:08:08 +02:00
Martin Algesten
a198ba0a65 Test case for preserving io::Error timed out 2020-08-07 21:51:35 +02:00
Martin Algesten
4bc6f79847 cargo fmt 2020-08-07 21:51:35 +02:00
Martin Algesten
6614856163 Make Response::to_json preserve io::Error of ErrorKind::TimedOut
Close #119
2020-08-07 21:51:35 +02:00
Martin Algesten
db4cc90956 Normalize timeout ErrorKind::WouldBlock to be TimedOut (#131)
`set_read_timeout` and `set_write_timeout` can cause `ErrorKind::WouldBlock` on unix-y platforms.

This PR normalizes those cases to `ErrorKind::TimedOut`. This will make it simpler higher up in the
stack to deal with timeouts.
2020-08-07 10:06:56 -07:00
cazgp
22d815e5e7 Make Cookie header RFC6265 compliant (#130) 2020-08-07 10:04:15 -07:00
André Cruz
75d5e52a45 Added method to set the TLS connection builder (#116)
This allows configuring the accepted certs, client authentication,
etc, when using the native TLS crate.
2020-07-28 21:44:35 +02:00
André Cruz
74afb82de9 Remove brackets from IPv6 literals before passing to TLS connector (#115) 2020-07-13 11:46:28 +02:00
dependabot-preview[bot]
81aab6e79a Update rustls requirement from 0.17 to 0.18 (#112)
Updates the requirements on [rustls](https://github.com/ctz/rustls) to permit the latest version.
- [Release notes](https://github.com/ctz/rustls/releases)
- [Changelog](https://github.com/ctz/rustls/blob/main/OLDCHANGES.md)
- [Commits](https://github.com/ctz/rustls/compare/v/0.17.0...v/0.18.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-08 08:43:17 -07:00
dependabot-preview[bot]
06aa1eb63a Update rustls-native-certs requirement from 0.3 to 0.4 (#111)
Updates the requirements on [rustls-native-certs](https://github.com/ctz/rustls-native-certs) to permit the latest version.
- [Release notes](https://github.com/ctz/rustls-native-certs/releases)
- [Commits](https://github.com/ctz/rustls-native-certs/compare/v/0.3.0...v/0.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-07-08 08:42:28 -07:00
Alex L
ea3b93dcab Add Proxy field to PoolKey. (#114)
Fixes #109
2020-07-08 08:41:43 -07:00
Jacob Hoffman-Andrews
87b8a381bc Remove default base URL of http://localhost/ (#108)
This doesn't change the API at all, since it delays any errors until do_call.
Since there was already an `Into<Response> for Error`, and do_call already
had a `.unwrap_or_else(|e| e.into())`, nothing in do_call needed to change.
The only visible effect of this is that code that was previously relying on
`get("/path")` to fetch something from localhost will now get an error. I think
most of those cases would probably actually be accidents, possibly caused
by typos, e.g. `get("example.com/path")` (forgetting the https: prefix).

Fixes #105
2020-07-05 14:25:59 -07:00
Jacob Hoffman-Andrews
f16f1a5e47 Error when LimitedRead gets too few bytes. (#106) 2020-07-05 14:25:11 -07:00
Jacob Hoffman-Andrews
7fce93d1c8 Fix tests. 2020-07-02 23:07:53 -07:00
Jacob Hoffman-Andrews
b53de7a7ed Remove default URL_BASE of localhost. 2020-07-02 23:01:51 -07:00
dependabot-preview[bot]
6e69f6cf69 Update cookie requirement from 0.13 to 0.14
Updates the requirements on [cookie](https://github.com/SergioBenitez/cookie-rs) to permit the latest version.
- [Release notes](https://github.com/SergioBenitez/cookie-rs/releases)
- [Changelog](https://github.com/SergioBenitez/cookie-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/SergioBenitez/cookie-rs/compare/0.13.0...0.14.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-01 10:55:27 +02:00
Jacob Hoffman-Andrews
ea7ed88399 Request: rename path field to URL. (#104)
* Request: rename path field to URL.

This more accurately reflects its use.
2020-07-01 09:53:10 +02:00
Jacob Hoffman-Andrews
e4ad3a5e5f Remove unnecessary lifetime annotations. (#103) 2020-07-01 09:40:47 +02:00
Jacob Hoffman-Andrews
bbde617719 Add --no-deps to doc build. (#102) 2020-07-01 09:36:57 +02:00
Jacob Hoffman-Andrews
4d7bf5dcda Validate header field names. (#99)
This rejects spaces after field names as well as various other invalid
characters.

Fixes #96
2020-06-30 19:32:17 -07:00
dependabot-preview[bot]
c9ccbd5f09 Update webpki-roots requirement from 0.19 to 0.20 (#100)
Updates the requirements on [webpki-roots](https://github.com/ctz/webpki-roots) to permit the latest version.
- [Release notes](https://github.com/ctz/webpki-roots/releases)
- [Commits](https://github.com/ctz/webpki-roots/compare/v/0.19.0...v/0.20.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-06-30 19:17:19 -07:00
Jacob Hoffman-Andrews
b47f90e773 Factor out TestServer. (#98)
This creates a struct that encapsulates the test server setup, and adds
a shutdown method to clean up the thread when done.
2020-06-30 19:16:34 -07:00
Jacob Hoffman-Andrews
dafdf6a718 Tweak README. (#92)
The layout of crates.io means that code blocks overflow after 60
characters, so wrap before 60 to avoid needing a scrollbar.

Remove the warning about the Agent code. I think after our recent
testing, I believe it's good enough that it doesn't need a separate
warning.

Remove the old-style macro-use directive and the extern crate directive.
2020-06-26 19:09:24 +02:00
Jacob Hoffman-Andrews
3f78953874 Add a GitHub action for building docs. (#89)
This helps ensure we don't break that doc build.
2020-06-24 21:20:07 +02:00
Martin Algesten
4e1e3d1ad4 Make features to use for docs.rs build explicit (#90)
Since tls and native-tls are mutually exclusive, we can't use
all-features anymore. Instead we enumerate the features needed to
build the docs for docs.rs.
2020-06-24 09:37:15 -07:00
Martin Algesten
6bb838f205 Fix building with socks-proxy feature (#91)
This PR corrects a variable name and type, and ensures we
test the socks-proxy feature as part of the test matrix.
2020-06-24 09:36:08 -07:00
Jacob Hoffman-Andrews
3cb178ed7c 1.3.0 (#83)
1.3.0
2020-06-24 09:14:00 +02:00
Jacob Hoffman-Andrews
1a40bed7dd Fix reference to Instant in socks feature. (#88)
The reference to time::Instant under feature = socks-proxy was
incorrectly scoped, and should have been just Instant. This breaks the
doc build and any builds that use feature = socks-proxy.
2020-06-24 09:13:12 +02:00
Jacob Hoffman-Andrews
a85f4c037e Remove some panics in deadline checking. (#87)
time_until_deadline had a time of check to time of use problem - the
deadline could pass between a call to checked_duration_since and the
evaluation of `deadline - now` (which panics if the result would be
negative). Resolve that by flipping the order of
checked_duration_since's arguments and using the result rather than
ignoring it.

Also there were three places that called deadline - now(), which could
panic. Replace those with time_until_deadline().
2020-06-23 23:45:18 -07:00
Jacob Hoffman-Andrews
2d6747717d Limit max idle connections. (#81)
Adds limit of 100 connections to ConnectionPool. This is implemented
with a VecDeque acting as an LRU. When the pool becomes full, the oldest
stream is popped off the back from the VecDeque and also removed from
the map of PoolKeys to Streams.

Fixes #77
2020-06-23 23:44:47 -07:00
Jacob Hoffman-Andrews
a6e99c8b36 Add smoke test in examples. (#85)
This loads a list of top domain names (e.g. from
https://tranco-list.eu/) and tries to fetch them all, in parallel. This
can be used to exercise ureq and find panics.
2020-06-23 21:09:09 -07:00