Commit Graph

73 Commits

Author SHA1 Message Date
Jacob Hoffman-Andrews
203573d27c Implement more realistic doctests. (#222)
Add is_test and fn main headers to various doctests, and use real
URLs along with the ? operator.
2020-11-14 09:53:15 -08:00
Jacob Hoffman-Andrews
ec8dace1af Turn Unit into a built Request (#223)
This involved removing the Request reference from Unit, and adding an
Agent, a method, and headers.

Also, move is_retryable to Unit.
2020-11-14 01:12:01 -08:00
Jacob Hoffman-Andrews
a52c6021cf Merge branch 'master' into release-2.0 2020-10-25 14:07:32 -07:00
Martin Algesten
1369c32351 API changes for 2.0
* Remove Request::build
* All mutations on Request follow builder pattern

The previous `build()` on request was necessary because mutating
functions did not follow a proper builder pattern (taking `&mut self`
instead of `mut self`). With a proper builder pattern, the need for
`.build()` goes away.

* All Request body and call methods consume self

Anything which "executes" the request will now consume the `Request`
to produce a `Result<Response>`.

* Move all config from request to agent builder

Timeouts, redirect config, proxy settings and TLS config are now on
`AgentBuilder`.

* Rename max_pool_connections -> max_idle_connections
* Rename max_pool_connections_per_host ->  max_idle_connections_per_host

Consistent internal and external naming.

* Introduce new AgentConfig for static config created by builder.

`Agent` can be seen as having two parts. Static config and a mutable
shared state between all states. The static config goes into
`AgentConfig` and the mutable shared state into `AgentState`.

* Replace all use of `Default` for `new`.

Deriving or implementing `Default` makes for a secondary instantiation
API.  It is useful in some cases, but gets very confusing when there
is both `new` _and_ a `Default`. It's especially devious for derived
values where a reasonable default is not `0`, `false` or `None`.

* Remove feature native_tls, we want only native rustls.

This feature made for very clunky handling throughout the code. From a
security point of view, it's better to stick with one single TLS API.
Rustls recently got an official audit (very positive).

https://github.com/ctz/rustls/tree/master/audit

Rustls deliberately omits support for older, insecure TLS such as TLS
1.1 or RC4. This might be a problem for a user of ureq, but on balance
not considered important enough to keep native_tls.

* Remove auth and support for basic auth.

The API just wasn't enough. A future reintroduction should at least
also provide a `Bearer` mechanism and possibly more.

* Rename jar -> cookie_store
* Rename jar -> cookie_tin

Just make some field names sync up with the type.

* Drop "cookies" as default feature

The need for handling cookies is probably rare, let's not enable it by
default.

* Change all feature checks for "cookie" to "cookies"

The outward facing feature is "cookies" and I think it's better form
that the code uses the official feature name instead of the optional
library "cookies".

* Keep `set` on Agent level as well as AgentBuilder.

The idea is that an auth exchange might result in a header that need
to be set _after_ the agent has been built.
2020-10-25 11:47:38 +01:00
Jacob Hoffman-Andrews
2bf9362eff Reinstate read timeouts on body.
This feature was broken in #67, which reset timeouts on the
stream before passing it to set_stream.

As part of this change, refactor the internal storage of
timeouts on the Request object to use Option<Duration>.

Remove the deadline field on Response. It wasn't used. The
deadline field on unit was used instead.

Add a unittest.
2020-10-24 12:12:07 +02:00
Jacob Hoffman-Andrews
67c28d28a3 Check for synthetic_error early in unit::connect 2020-10-24 12:08:27 +02:00
Jacob Hoffman-Andrews
e36c1c2aa1 Switch to Result-based API. (#132)
Gets rid of synthetic_error, and makes the various send_* methods return `Result<Response, Error>`.
Introduces a new error type "HTTP", which represents an error due to status codes 4xx or 5xx.
The HTTP error type contains a boxed Response, so users can read the actual response if they want.
Adds an `error_for_status` setting to disable the functionality of treating 4xx and 5xx as errors.
Adds .unwrap() to a lot of tests.

Fixes #128.
2020-10-17 00:40:48 -07:00
Jacob Hoffman-Andrews
b58a3a53b0 Add test.sh. (#173)
This emulates the test matrix that gets run in CI, making it easier to
find failures locally.

There was one conflict in the matrix: when JSON is enabled and TLS is
disabled, two of the doctests would fail. This was previous worked
around as an exclude in the github workflow. I changed the JSON doctest
to use HTTP instead.
2020-10-03 20:47:35 -07:00
Jacob Hoffman-Andrews
17d7e147eb Handle ConnectionReset+ConnectionAbort at any time (#168)
Previously we had a special case for BadStatusRead that would happen
only when we got a ConnectionAborted error reading the status line.
However, sometimes we get ConnectionReset instead. Also the HTTP
spec says that idempotent requests may be retried anytime a connection
is closed prematurely.

The change treats as retryable any ConnectionAborted OR ConnectionReset
error while reading the status line and headers. It removes the special
case BadStatusRead error.

Fixes #165 (I think).
2020-09-29 01:55:34 -07:00
Jacob Hoffman-Andrews
995f6e44a9 Pass through io::Errors when reading headers. (#166)
Previously any io::Error on reading a status line or response headers
would be unconditionally mapping into BadStatus or BadHeader. I think
it's better to pass through the actual io::Error.

BadStatusRead is still kept, since it has special status when dealing
with timed out connections, and BadStatus is still used when the status
line is malformed.
2020-09-28 09:14:26 -07:00
Jacob Hoffman-Andrews
5801f87c56 Remove Sync 2020-09-27 20:24:24 +02:00
Jacob Hoffman-Andrews
5e00b5c5e3 Add Send + Sync marker traits to into_reader.
This allows the resulting Read to be shared among threads.
2020-09-27 20:24:24 +02:00
Jacob Hoffman-Andrews
7046b07518 Replace IoResult and IoError with io:: versions. (#161) 2020-09-27 10:20:24 -07:00
Jacob Hoffman-Andrews
6a88c2c8bf Clean up unused code and long imports. (#137)
This removes some commented out methods, and also changes instances of
::std::foo to use a more idiomatic import path.
2020-09-12 18:42:15 -07:00
Martin Algesten
50c19c5484 Read buffer to avoid byte-by-byte syscalls (#141)
Fixes #140
2020-09-12 18:27:15 -07:00
Martin Algesten
6614856163 Make Response::to_json preserve io::Error of ErrorKind::TimedOut
Close #119
2020-08-07 21:51:35 +02:00
Jacob Hoffman-Andrews
f16f1a5e47 Error when LimitedRead gets too few bytes. (#106) 2020-07-05 14:25:11 -07:00
Jacob Hoffman-Andrews
e4ad3a5e5f Remove unnecessary lifetime annotations. (#103) 2020-07-01 09:40:47 +02:00
Jacob Hoffman-Andrews
fdc1f37662 Update documentation. (#73)
* Update documentation.

Synchronize goals section between README and rustdoc, and add two goals
(blocking API; no unsafe) that were mentioned elsewhere in the README.

Add error handling to examples for the module and for the Response
object.

And a section on synthetic errors to the top-level module documentation.

* Add back missing close brace.

* Add main function that returns Result.

* Add links to send_bytes and send_form.

* Document chunked encoding for send.

* Use a larger vec of bytes for send.
2020-06-21 09:55:50 +02:00
Jacob Hoffman-Andrews
7adbd57308 Fix up cargo test --no-default-features. (#75)
Adds some feature guards, and removes an unnecessary feature guard
around a call to connect_https (there's an implementation available for
non-TLS that returns UnknownScheme).

Also, remove unnecessary agent.state() method that was only available in
TLS builds. The state field is directly accessible within the crate, and
can be used in both TLS and non-TLS builds.

Co-authored-by: Martin Algesten <martin@algesten.se>
2020-06-21 09:54:03 +02:00
Jacob Hoffman-Andrews
57be414d97 Add overall timeout for requests. (#67)
This deprecates timeout_read() and timeout_write() in favor of
timeout(). The new timeout method on Request takes a Duration instead
of a number of milliseconds, and is measured against overall request
time, not per-read time.

Once a request is started, the timeout is turned into a deadline
specific to that call. The deadline is used in conjunction with the
new DeadlineStream class, which sets a timeout on each read according
to the remaining time for the request. Once the request is done,
the DeadlineStream is unwrapped via .into::<Stream>() to become
an undecorated Stream again for return to the pool. Timeouts on the
stream are unset at this point.

Still to be done:

Add a setting on Agent for default timeout.
Change header-writing code to apply overall deadline rather than
per-write timeout.
Fixes #28.
2020-06-21 09:47:35 +02:00
Drake Tetreault
069775d3e0 Use From instead of custom ReclaimStream. 2020-06-16 10:04:52 +02:00
Drake Tetreault
af6491cd59 Remove unsafe usage by taking advantage of new Decoder::unwrap function. 2020-06-16 10:02:57 +02:00
Paolo Barbolini
4e744f87c1 Keep the old Response::into_json around and call this Response::into_json_deserialize 2020-05-06 13:47:37 +02:00
Paolo Barbolini
0b69c595b6 Make Response::into_json deserialize into a serde DeserializeOwned
This removes the necessity to take the result of Response::into_json and
having to convert it into a struct by using serde_json::from_value

This adds no new dependencies since serde_json already depends on serde.
Users of ureq will have to include `serde_derive` either by importing it
directly or by using serde with the `derive` feature, unless they want to
manually implement `Deserialize` on their structs.
2020-05-06 13:47:37 +02:00
Koga Kazuo
661853b95d Check status code that has no response body 2020-04-12 09:10:39 +02:00
Martin Algesten
2956683870 bump deps, fix clippy warnings 2020-03-14 09:54:54 +01:00
Martin Algesten
753d61b454 Retry some pooled connections failing when server closes. Close #10
This is not a perfect solution. It works as long as we are not sending
any body bytes. We discover the error first when attempting to read
the response status line. That means we discover the error after
sending body bytes. To be able to re-send the body, we would need to
introduce a buffer to be able to replay the body on the next
request. We don't currently do that.
2019-10-20 21:36:38 +02:00
Martin Algesten
d9b8fef9a0 Code comment on *mut Stream effect on thread safety 2019-10-20 14:08:21 +02:00
Martin Algesten
acffc5fe13 Rename YoloRead -> ReclaimingRead
To better reflect why this is necessary. Also provide a doc comment why this is needed.
2019-10-20 12:31:55 +02:00
Martin Algesten
27be8117b8 Avoid tuple to describe response status line 2019-10-20 12:25:54 +02:00
Martin Algesten
ed999b579d Removing AsciiString dep and one unsafe
This library is not about enforcing standards, so the internal use of
AsciiString for headers and status lines is not necessary.
2019-10-20 10:40:30 +02:00
Chris West (Faux)
a97dfcb408 fix warnings 2019-09-09 19:54:45 +01:00
Martin Algesten
9d069b60b4 fix tests 2019-06-21 15:53:23 +02:00
Chris West (Faux)
ceb7c3ac14 use pub(crate) instead of include!() 2019-05-27 17:44:14 +01:00
Martin Algesten
d4d68ae585 fix parse error when no status text. close #4 2019-03-30 11:29:53 +01:00
Martin Algesten
141668acba list all header names in request and response 2019-02-03 11:52:05 +05:30
Martin Algesten
cb3d6e8124 resolved url in response 2018-12-20 11:08:39 +01:00
Martin Algesten
07fd4d2cd5 pub(crate) where we can 2018-12-20 11:08:20 +01:00
Martin Algesten
1519dcb0b1 fix redirect logic 2018-12-20 10:54:47 +01:00
Martin Algesten
5ba6b3cd4d edition 2018, clippy, fmt 2018-12-18 13:17:19 +01:00
Martin Algesten
2c9e62ad8c fix dealloc issues 2018-10-23 20:18:24 +01:00
Martin Algesten
7ba59abbef format 2018-09-03 11:22:20 +02:00
Martin Algesten
8decd8e489 auth url test 2018-07-01 18:40:49 +02:00
Martin Algesten
2d2daf58ba doc 2018-07-01 11:13:21 +02:00
Martin Algesten
03928a05c5 tidy up 2018-07-01 10:19:40 +02:00
Martin Algesten
567a19a656 resp.status() return u16 2018-07-01 09:04:50 +02:00
Martin Algesten
e2686e3bd5 deal with http/1.0 and connection closed 2018-06-30 22:43:47 +02:00
Martin Algesten
552728d1d1 connection pool done 2018-06-30 17:55:11 +02:00
Martin Algesten
4a5944443f connection pooling 2018-06-30 16:52:54 +02:00