SwiftNIO 2.42.0 has deprecated Lock and replaced it with a new NIOLock. This commit removes all uses of Lock and replaces them with NIOLock. Further, now, we must require SwiftNIO 2.42.0
* Fix thread leak in `FileDownloadDelegate`
* `SwiftFormat`
* Add a shared file IO thread pool per HTTPClient
* User bigger thread pool and initlize lazily during first file write
* thread pool is actually not used in tests
* Update documentation
* fix review comments
* make `fileIOThreadPool` internal
* Add test to verify that we actually share the same thread pool across all delegates for a given HTTPClient
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation
We should apply the connect timeout to the complete set of connection
attempts, rather than the request deadline. This allows users
fine-grained control over how long we attempt to connect for. This is
also the behaviour of our old-school interface.
Modifications
- Changed the connect deadline calculation for async/await to match that
of the future-based code.
- Added a connect timeout test.
Result
Connect timeouts are properly handled
Motivation
Documentation is nice, and we can help support users by providing useful
clear docs.
Modifications
Add Docc to 5.6 and later builds
Make sure symbol references work
Add overview docs
Result
Nice rendering docs
Motivation
Warnings aren't great, and NIOAtomic is deprecated.
Modifications
Replace the last use of NIOAtomic with ManagedAtomic.
Result
Fewer warnings
Fixes#606
Motivation
Currently error reporting with NIO Transport Services is often sub-par.
This occurs because the Network.framework connections may enter the
waiting state until the network connectivity state changes. We were not
watching for the user event that contains the error in that state, so if
we timed out in that state we'd just give a generic timeout error,
instead of telling the user anything more detailed.
Additionally, several of our tests assume that failure will be fast, but
in NIO Transport Services we will enter that .waiting state. This is
reasonable, as changed network connections may make a connection that
was not succeeding suddenly viable. However, it's inconvenient for
testing, where we're mostly interested in confirming that the error path
works as expected.
Modifications
- Add an observer of the WaitingForConnectivity event that records it
into our state machine for later reporting.
- Add support for disabling waiting for connectivity for testing
purposes.
- Add annotations to several tests to stop them waiting for
connectivity.
Results
Faster tests, better coverage, better errors for our users.
Co-authored-by: David Nadoba <dnadoba@gmail.com>
* refactor RedirectHandler
- `redirectState` is no longer a property of `HTTPClient.Request`. RedirectHandler now stores this state directly and therefore no longer optional.
- we no longer count the number of allowed redirects down. Instead the number of redirects is dervied from `self.visited.count` and we compare it to the maxRedirect to check if we git the limit.
* `HTTPClient.Configuration.RedirectConfiguration.Configuration` is now called `HTTPClient.Configuration.RedirectConfiguration.Mode`
only two `Configuration`s left in the type name
* add redirect logger test
### Motivation
Generally we want to inform users that they need to shutdown their HTTPClient. Until `1.6.0` we did this with an assert in HTTPClient's deinit. With `1.6.0` this behavior was raised to a precondition. Because of this adopters might suddenly crash in production where they didn't before.
### Changes
- This pr reverts the current behavior back to something pre `1.6.0`
### Result
- HTTPClient doesn't crash in production anymore.
### Motivation
Fixes#238 and #231.
### Changes
- Extracted the unclean shutdown test from `HTTPClientTests` into their own file `HTTPClientUncleanSSLConnectionShutdownTests`
- Copy and pasted @weissi great explanation from #238 into the test file
- Removed property `ignoreUncleanSSLShutdown` everywhere
### Result
`ignoreUncleanSSLShutdown` on `HTTPClient.Configuration` is deprecated and ignored.
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
- Adding four new protocols:
- `HTTPExecutingRequest` a protocol representing an HTTP task that is executed on a ChannelHandler.
- `HTTPScheduledRequest` a protocol representing an HTTP task that is scheduled for execution, but in a waiting state (example: Waiting for an idle connection in the connection pool).
- `HTTPRequestExecutor` a protocol that can be used from the `HTTPExecutingRequest` abstracting away functionality that will normally be implemented by a `ChannelHandler`
- `HTTPRequestScheduler` a protocol that can be used from the `HTTPScheduledRequest` abstracting away functionality that will normally be implemented by a `ConnectionPool`
- An implementation of the `HTTPExecutingTask` and `HTTPScheduledRequest` called `RequestBag`. It implements our current API using the new protocols
Co-authored-by: George Barnett <gbarnett@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
- The connection creation logic has been refactored into a number of smaller methods that can be combined
- Connection creation now has a logical home. It is moved from `Utils.swift` into a `ConnectionFactory`
- There are explicit `ChannelHandlers` that are used for connection creation:
- `TLSEventsHandler` got its own file and unit tests
- `HTTP1ProxyConnectHandler` got its own file and unit tests
- `SOCKSEventsHandler` got its own file and unit tests
- Some small things are already part of this pr that will get their context later. For example:
- `HTTPConnectionPool` is added as a namespace to not cause major renames in follow up PRs
- `HTTPConnectionPool.Connection.ID` and its generator were added now. (This will be used later to identify a connection during its lifetime)
- the file `HTTPConnectionPool+Manager` was added to give `HTTPConnectionPool.Connection.ID.Generator` already its final destination.
Updated:
NIO
NIOSSL
NIO Extras
NIOTS
Also fix TLSConfiguration.forClient() warnings by converting to TLSConfiguration.makeClientConfiguration(). Also the same for forServer().
Motivation:
At the moment, AHC assumes that creating a `NIOSSLContext` is both cheap
and doesn't block.
Neither of these two assumptions are true.
To create a `NIOSSLContext`, BoringSSL will have to read a lot of
certificates in the trust store (on disk) which require a lot of ASN1
parsing and much much more.
On my Ubuntu test machine, creating one `NIOSSLContext` is about 27,000
allocations!!! To make it worse, AHC allocates a fresh `NIOSSLContext`
for _every single connection_, whether HTTP or HTTPS. Yes, correct.
Modification:
- Cache NIOSSLContexts per TLSConfiguration in a LRU cache
- Don't get an NIOSSLContext for HTTP (plain text) connections
Result:
New connections should be _much_ faster in general assuming that you're
not using a different TLSConfiguration for every connection.
Motivation:
AsyncHTTPClient attempts to avoid the problem of Happy Eyeballs making
it hard to know which Channel will be returned by only inserting the
TLSEventsHandler upon completion of the connect promise. Unfortunately,
as this may involve event loop hops, there are some awkward timing
windows in play where the connect may complete before this handler gets
added.
We should remove that timing window by ensuring that all channels always
have this handler in place, and instead of trying to wait until we know
which Channel will win, we can find the TLSEventsHandler that belongs to
the winning channel after the fact.
Modifications:
- TLSEventsHandler no longer removes itself from the pipeline or throws
away its promise.
- makeHTTP1Channel now searches for the TLSEventsHandler from the
pipeline that was created and is also responsible for removing it.
- Better sanity checking that the proxy TLS case does not overlap with
the connection-level TLS case.
Results:
Further shrinking windows for pipeline management issues.
Motivation:
There is an awkward timing window in the TLSEventsHandler flow where it
is possible for the NIOSSLClientHandler to fail the handshake on
handlerAdded. If this happens, the TLSEventsHandler will not be in the
pipeline, and so the handshake failure error will be lost and we'll get
a generic one instead.
This window can be resolved without performance penalty if we use the
new synchronous pipeline operations view to add the two handlers
backwards. If this is done then we can ensure that the TLSEventsHandler
is always in the pipeline before the NIOSSLClientHandler, and so there
is no risk of event loss.
While I'm here, AHC does a lot of pipeline modification. This has led to
lengthy future chains with lots of event loop hops for no particularly
good reason. I've therefore replaced all pipeline operations with their
synchronous counterparts. All but one sequence was happening on the
correct event loop, and for the one that may not I've added a fast-path
dispatch that should tolerate being on the wrong one. The result is
cleaner, more linear code that also reduces the allocations and event
loop hops.
Modifications:
- Use synchronous pipeline operations everywhere
- Change the order of adding TLSEventsHandler and NIOSSLClientHandler
Result:
Faster, safer, fewer timing windows.
Motivation:
Right now documentation states that timrout defaults to no timeout, this
is no actually true, if timeout is not set NIO bootstrap defaults to 10
seconds connect timeout.
Modifications:
Updates documentation comment.
Result:
Closes#118
Motivation:
Right now we only handle one type of SSL error: `.handshakeFailed`,
but in reality a multitude of errors can happen, for example, remote
party might just close connection that will in turn raise
`.uncleanShutdown` error, that will be dropped on the floor and
users will only get non-descriptive `NoResult` error.
Modifications:
Handle all types of SSL errors during handshake instead of just one.
Adds a test.
Result:
Closes#313
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
Streams length parameter is optional to allow cases were stream length is not known in advance, but we do not support this in request validation. This PR aims to address that.
Modifications:
Modifies request validation to default to chunked encoding if body length is zero or to passed in content-length header
Adds a test
Result:
Closes#218
`redirectConfiguration` can't default to `false` as it's not a boolean value, and the default value is `RedirectConfiguration()`.
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Co-authored-by: Artem Redkin <artem@redkin.me>
David Nadoba
Fabian Fett
Johannes Weiss
Karl
Cory Benfield
Christian Priebe
David Evans
Artem Redkin
Adam Fowler
Max Desiatov