Previously, we treated HTTP/1.0 responses as always being
close-delimited. However, that's not quite right. HTTP/1.0 responses
_default_ to Connection: close, but the server may send Connection:
keep-alive, along with a Content-Length header. Update body_type to
understand this.
Also, slightly reorganize body_type. has_no_body was being checked
redundantly when we could just early-return when has_no_body is true. And
regardless of whether we see Connection: close (on any HTTP version),
we must honor Transfer-Encoding or Content-Length if they are present.
Change the handling of `Connection: close` to more directly affect
whether a connection is pooled, regardless of what method was used
to delimit the body.
Fixes#600
These test were failing because they made live requests to httpbin.org,
which is serving intermittent 500s.
In general we strive to make our test and doctests interact with a local
test server rather than a live website. For doctests we do this with
some sleight of hand in a hidden `ureq::is_test(true)`, which swaps out
the default agent returned by ureq::agent. However, since the middleware
doctests rely on constructing a custom agent with the middleware, they
can't rely on this.
I suspect we can get around this problem but it may take some thinking.
Similarly disable the AgentBuilder::user_agent and redirects doctests
and the agent_set_header unittest.
This removes the cookie header on redirect, otherwise if there are
multiple redirects, a new cookie header gets appended each time, while
the existing one remains. This could result in the cookies being leaked
to a third party.
Previously, `ureq` used `HTTP CONNECT` for *all* requests, including
plain-text HTTP requests. However, some proxy servers like Squid
only allow tunneling via the `HTTP CONNECT` method on port 443 - since
it is usually only used to proxy HTTPS requests. As a result,
it was not possible to use `ureq` with a Squid server in its default configuration.
With the changes from this commit, ureq will interact with HTTP proxies
in a more standard-conforming way, where `CONNECT` is only used for
HTTPS requests. HTTP request paths are transformed in such a way that they
comply with RFC 7230 [1].
Tested against Squid 4.13 in standard configuration, with and without basic authentication,
for HTTP and HTTPS requests.
[1] https://www.rfc-editor.org/rfc/rfc7230#section-5.3.2
This breaks a reference cycle between PoolReturner and Agent that was
causing Agents (and their contained ConnectionPool) to never be dropped
so long as there was any stream in the ConnectionPool. This cause
sockets to leak over time, particularly when the convenience functions
ureq::get(), ureq::post(), etc were used, since those functions create
a new Agent each time.
