Motivation
This patch adds support for HTTP/1 connection pre-warming. This allows
the user to request that the HTTP/1 connection pool create and maintain
extra connections, above-and-beyond those strictly needed to run the
pool. This pool can be used to absorb small spikes in request traffic
without increasing latency to account for connection creation.
Modifications
- Added new configuration properties for pre-warmed connections.
- Amended the HTTP/1 state machine to create new connections where
necessary.
- Added state machine tests.
Results
Pre-warmed connections are available.
Motivation:
Now that strict concurrency has been adopted the AHC should avoid
regressing by treating all warnings as errors in CI.
Modifications:
- Treat warnings as errors in CI
Result:
Stricter CI
Motivation
We have some Task error handling functions that are generic for no
apparent reason. They're also typically called from contexts where they
also report the error to the delegate, but one of the call sites doesn't
do that. So add a test for that as well.
Modifications
- Rewrite Task.fail(with:delegate:) to be non-generic.
- Add a call to the delegate error handler on the path that is missing
it.
- Add a test for that call
Results
Cleaner, easier to follow code
Motivation:
We're about to go on a sendability journey. Let's pick some low hanging
fruit to get started.
Modifications:
- Add a few assume-isolated calls
- Stop using static var
- Use a dispatch group instead of a work item to wait for work to be
done.
Result:
Fewer warnings
Motivation:
As requested in #596, it can be handy to have a lower-level access to
channels (HTTP/1 connection, HTTP/2 connection, or HTTP/2 stream) to
enable a more fine-grained interaction for, say, observability, testing,
etc.
Modifications:
- Add 3 new properties (`http1_1ConnectionDebugInitializer`,
`http2ConnectionDebugInitializer` and
`http2StreamChannelDebugInitializer`) to `HTTPClient.Configuration` with
access to the respective channels. These properties are of `Optional`
type `@Sendable (Channel) -> EventLoopFuture<Void>` and are called when
creating a connection/stream.
Result:
Provides APIs for a lower-level access to channels.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: David Nadoba <d_nadoba@apple.com>
Co-authored-by: George Barnett <gbarnett@apple.com>
fixes#784
`writeChunks` had 3 bugs:
1. An actually wrong `UnsafeMutableTransferBox` -> removed that type
which should never be created
2. A loooong future chain (instead of one final promise) -> implemented
3. Potentially infinite recursion which lead to the crash in #784) ->
fixed too
Migrate CI to use GitHub Actions.
### Motivation:
To migrate to GitHub actions and centralised infrastructure.
### Modifications:
Changes of note:
* Adopt swift-format using rules from SwiftNIO.
* Remove scripts and docker files which are no longer needed.
* Disabled warnings-as-errors on Swift 6.0 CI pipelines for now.
### Result:
Feature parity with old CI.
Multipath TCP (MPTCP) is a TCP extension allowing to enhance the
reliability of the network by using multiple interfaces. This extension
provides a seamless handover between interfaces in case of deterioration
of the connection on the original one. In the context of iOS and Mac OS
X, it could be really interesting to leverage the capabilities of MPTCP
as they could benefit from their multiple interfaces (ethernet + Wi-fi
for Mac OS X, Wi-fi + cellular for iOS).
This contribution introduces patches to HTTPClient.Configuration and
establishment of the Bootstraps. A supplementary field "enableMultipath"
was added to the configuration, allowing to request the use of MPTCP.
This flag is then used when creating the channels to configure the
client.
Note that in the future, it might also be potentially interesting to
offer more precise configuration options for MPTCP on MacOS, as the
Network framework allows also to select a type of service, instead of
just offering the option to create MPTCP connections. Currently, when
enabling MPTCP, only the Handover mode is used.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
* Make ConnectionPool's `retryConnectionEstablishment` public
* Unified tests to consistently use `enableFastFailureModeForTesting()`
* Add `retryConnectionEstablishment` as optional parameter to the initializer of `HTTPClient.Configuration.ConnectionPool`
* Reverted change to initializer to prevent API stability breakage
* Add parameterless initializer for `HTTPClient.Configuration.ConnectionPool`
* Moved default values for `HTTPClient.Configuration.ConnectionPool` to the property declarations, so they only have to be specified at one point
* Removed superfluous spaces
Co-authored-by: Cory Benfield <lukasa@apple.com>
* Re-added missing line break
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
Now that Swift 5.9 is GM we should update the supported versions and
remove 5.6
Modifications:
* Update `Package.swift`
* Remove `#if swift(>=5.7)` guards
* Delete the 5.6 docker compose file and make a 5.10 one
* Update docs
Result:
Remove support for Swift 5.6, add 5.10
Motivation:
Sometimes it can be helpful to limit the number of times a connection
can be used before discarding it. AHC has no such support for this at
the moment.
Modifications:
- Add a `maximumUsesPerConnection` configuration option which defaults
to `nil` (i.e. no limit).
- For HTTP1 we count down uses in the state machine and close the
connection if it hits zero.
- For HTTP2, each use maps to a stream so we count down remaining uses
in the state machine which we combine with max concurrent streams to
limit how many streams are available per connection. We also count
remaining uses in the HTTP2 idle handler: we treat no remaining uses
as receiving a GOAWAY frame and notify the pool which then drains the
streams and replaces the connection.
Result:
Users can control how many times each connection can be used.
Motivation:
not accumulate too many bytes
Modifications:
Implementing collect function to use NIOCore version to prevent overflowing
Co-authored-by: Cory Benfield <lukasa@apple.com>
Sometimes it can be useful to connect to one host e.g. `x.example.com` but request and validate the certificate chain as if we would connect to `y.example.com`. This is what this PR adds support for by adding a `dnsOverride` configuration to `HTTPClient.Configuration`. This is similar to curls `—resolve-to` option but only allows overriding host and not ports for now.
* Make syncShutdown unavailable from async
Motivation
syncShutdown can cause unbounded thread blocking, we shouldn't allow it
in concurrent code.
Modification
Mark syncShutdown unavailable from async.
Result
Users are warned if they try to syncShutdown in an async context
* Only noasync on 5.7
Motivation
Allowing arbitrary data in outbound header field values allows for the
possibility that users of AHC will accidentally pass untrusted data into
those values. That untrusted data can substantially alter the parsing
and content of the HTTP requests, which is extremely dangerous. The
result of this is vulnerability to CRLF injection.
Modifications
Add validation of outbound header field values.
Result
No longer vulnerable to CRLF injection
SwiftNIO 2.42.0 has deprecated Lock and replaced it with a new NIOLock. This commit removes all uses of Lock and replaces them with NIOLock. Further, now, we must require SwiftNIO 2.42.0
* Fix thread leak in `FileDownloadDelegate`
* `SwiftFormat`
* Add a shared file IO thread pool per HTTPClient
* User bigger thread pool and initlize lazily during first file write
* thread pool is actually not used in tests
* Update documentation
* fix review comments
* make `fileIOThreadPool` internal
* Add test to verify that we actually share the same thread pool across all delegates for a given HTTPClient
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation
We should apply the connect timeout to the complete set of connection
attempts, rather than the request deadline. This allows users
fine-grained control over how long we attempt to connect for. This is
also the behaviour of our old-school interface.
Modifications
- Changed the connect deadline calculation for async/await to match that
of the future-based code.
- Added a connect timeout test.
Result
Connect timeouts are properly handled
Motivation
Documentation is nice, and we can help support users by providing useful
clear docs.
Modifications
Add Docc to 5.6 and later builds
Make sure symbol references work
Add overview docs
Result
Nice rendering docs
Motivation
Warnings aren't great, and NIOAtomic is deprecated.
Modifications
Replace the last use of NIOAtomic with ManagedAtomic.
Result
Fewer warnings
Fixes#606
Motivation
Currently error reporting with NIO Transport Services is often sub-par.
This occurs because the Network.framework connections may enter the
waiting state until the network connectivity state changes. We were not
watching for the user event that contains the error in that state, so if
we timed out in that state we'd just give a generic timeout error,
instead of telling the user anything more detailed.
Additionally, several of our tests assume that failure will be fast, but
in NIO Transport Services we will enter that .waiting state. This is
reasonable, as changed network connections may make a connection that
was not succeeding suddenly viable. However, it's inconvenient for
testing, where we're mostly interested in confirming that the error path
works as expected.
Modifications
- Add an observer of the WaitingForConnectivity event that records it
into our state machine for later reporting.
- Add support for disabling waiting for connectivity for testing
purposes.
- Add annotations to several tests to stop them waiting for
connectivity.
Results
Faster tests, better coverage, better errors for our users.
Co-authored-by: David Nadoba <dnadoba@gmail.com>
* refactor RedirectHandler
- `redirectState` is no longer a property of `HTTPClient.Request`. RedirectHandler now stores this state directly and therefore no longer optional.
- we no longer count the number of allowed redirects down. Instead the number of redirects is dervied from `self.visited.count` and we compare it to the maxRedirect to check if we git the limit.
* `HTTPClient.Configuration.RedirectConfiguration.Configuration` is now called `HTTPClient.Configuration.RedirectConfiguration.Mode`
only two `Configuration`s left in the type name
* add redirect logger test
### Motivation
Generally we want to inform users that they need to shutdown their HTTPClient. Until `1.6.0` we did this with an assert in HTTPClient's deinit. With `1.6.0` this behavior was raised to a precondition. Because of this adopters might suddenly crash in production where they didn't before.
### Changes
- This pr reverts the current behavior back to something pre `1.6.0`
### Result
- HTTPClient doesn't crash in production anymore.
### Motivation
Fixes#238 and #231.
### Changes
- Extracted the unclean shutdown test from `HTTPClientTests` into their own file `HTTPClientUncleanSSLConnectionShutdownTests`
- Copy and pasted @weissi great explanation from #238 into the test file
- Removed property `ignoreUncleanSSLShutdown` everywhere
### Result
`ignoreUncleanSSLShutdown` on `HTTPClient.Configuration` is deprecated and ignored.
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Konrad `ktoso` MalawskiandGitHub
Cory BenfieldandGitHub
Johannes WeissandGitHub
Rick Newton-RogersandGitHub
Gustavo CairoandGitHub
Mahdi BahramiandGitHub
Rick Newton-RogersandGitHub
Fabian FettandGitHub
KarlandGitHub
Christian PriebeandGitHub