Motivation
In machines with more complex network topologies it is possible for us
to have multiple possible NICs we might want to use for a request. Users
may wish to vary this on a per-request or even a per-client basis.
This control can typically be expressed by offering a local address to
bind to before making the connection attempt.
Modifications
Allow users to express a preferred local address at request or client
scope.
Make this part of the connection pool key.
Bind the local address when specified.
Test all of this.
Results
More capable clients.
Motivation
Right now, we insert HTTP/2 handlers in a callback on a future that is
done very late. The result of this is that an entire ALPN negotiaton
_can_ complete before this callback is attached. That can in rare cases
cause the HTTP/2 handler to miss the server preamble, because it gets
added too late.
Modifications
This patch refactors the existing code to close that window. It does so
by passing a promise into the connection path and completing that
promise _on_ the event loop where we add the ALPN handlers, which should
ensure this will execute immediately when the ALPN negotiation
completes. Immportantly, we attach our promise callbacks to that promise
_before_ we hand it off, making sure the timing windows go away.
Results
Timing window is closed
Motivation:
The connection pool holds much of the low level logic in AHC. We should
fix its sendability issues before moving to higher levels.
Modifications:
- Make HTTP1ConnectionDelegate and HTTP2Delegate sendable, this requires
passing IDs rather than connections to their methods
- Make HTTPConnectionRequester sendable and have its methods take
Sendable views of the HTTP1Connection and HTTP2Connection types
- Add sendable views to HTTP1Connection and HTTP2Connection
- Mark HTTP1Connection and HTTP2Connection as not sendable
- Make HTTPRequestExecutor and HTTPExecutableRequest sendable
- Update tests
Result:
Connection pool has stricter sendability requirements
Motivation:
As requested in #596, it can be handy to have a lower-level access to
channels (HTTP/1 connection, HTTP/2 connection, or HTTP/2 stream) to
enable a more fine-grained interaction for, say, observability, testing,
etc.
Modifications:
- Add 3 new properties (`http1_1ConnectionDebugInitializer`,
`http2ConnectionDebugInitializer` and
`http2StreamChannelDebugInitializer`) to `HTTPClient.Configuration` with
access to the respective channels. These properties are of `Optional`
type `@Sendable (Channel) -> EventLoopFuture<Void>` and are called when
creating a connection/stream.
Result:
Provides APIs for a lower-level access to channels.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: David Nadoba <d_nadoba@apple.com>
Co-authored-by: George Barnett <gbarnett@apple.com>
# Motivation
The NIO 2.78 release introduced a bunch of new warnings. These warnings
cause us a bunch of trouble, so we should fix them.
# Modifications
Mostly use a bunch of assumeIsolated() and syncOperations.
# Result
CI passes again.
Note that Swift 6 has _many_ more warnings than this, but we expect more
to come and we aren't using warnings-as-errors on that mode at the
moment. We'll be cleaning that up soon.
Migrate CI to use GitHub Actions.
### Motivation:
To migrate to GitHub actions and centralised infrastructure.
### Modifications:
Changes of note:
* Adopt swift-format using rules from SwiftNIO.
* Remove scripts and docker files which are no longer needed.
* Disabled warnings-as-errors on Swift 6.0 CI pipelines for now.
### Result:
Feature parity with old CI.
Multipath TCP (MPTCP) is a TCP extension allowing to enhance the
reliability of the network by using multiple interfaces. This extension
provides a seamless handover between interfaces in case of deterioration
of the connection on the original one. In the context of iOS and Mac OS
X, it could be really interesting to leverage the capabilities of MPTCP
as they could benefit from their multiple interfaces (ethernet + Wi-fi
for Mac OS X, Wi-fi + cellular for iOS).
This contribution introduces patches to HTTPClient.Configuration and
establishment of the Bootstraps. A supplementary field "enableMultipath"
was added to the configuration, allowing to request the use of MPTCP.
This flag is then used when creating the channels to configure the
client.
Note that in the future, it might also be potentially interesting to
offer more precise configuration options for MPTCP on MacOS, as the
Network framework allows also to select a type of service, instead of
just offering the option to create MPTCP connections. Currently, when
enabling MPTCP, only the Handover mode is used.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
Sometimes it can be helpful to limit the number of times a connection
can be used before discarding it. AHC has no such support for this at
the moment.
Modifications:
- Add a `maximumUsesPerConnection` configuration option which defaults
to `nil` (i.e. no limit).
- For HTTP1 we count down uses in the state machine and close the
connection if it hits zero.
- For HTTP2, each use maps to a stream so we count down remaining uses
in the state machine which we combine with max concurrent streams to
limit how many streams are available per connection. We also count
remaining uses in the HTTP2 idle handler: we treat no remaining uses
as receiving a GOAWAY frame and notify the pool which then drains the
streams and replaces the connection.
Result:
Users can control how many times each connection can be used.
Sometimes it can be useful to connect to one host e.g. `x.example.com` but request and validate the certificate chain as if we would connect to `y.example.com`. This is what this PR adds support for by adding a `dnsOverride` configuration to `HTTPClient.Configuration`. This is similar to curls `—resolve-to` option but only allows overriding host and not ports for now.
* Fix crash if connection is closed very early
If the channel is closed before flatMap is executed, all ChannelHandler are removed and `TLSEventsHandler` is therefore not present either. We need to tolerate this even though it is very rare.
Testing ideas welcome.
Fixes#670
* drop precondition to assert
Motivation
Currently error reporting with NIO Transport Services is often sub-par.
This occurs because the Network.framework connections may enter the
waiting state until the network connectivity state changes. We were not
watching for the user event that contains the error in that state, so if
we timed out in that state we'd just give a generic timeout error,
instead of telling the user anything more detailed.
Additionally, several of our tests assume that failure will be fast, but
in NIO Transport Services we will enter that .waiting state. This is
reasonable, as changed network connections may make a connection that
was not succeeding suddenly viable. However, it's inconvenient for
testing, where we're mostly interested in confirming that the error path
works as expected.
Modifications
- Add an observer of the WaitingForConnectivity event that records it
into our state machine for later reporting.
- Add support for disabling waiting for connectivity for testing
purposes.
- Add annotations to several tests to stop them waiting for
connectivity.
Results
Faster tests, better coverage, better errors for our users.
Co-authored-by: David Nadoba <dnadoba@gmail.com>
* make `Scheme` a type
* introduce new Endpoint type
* use endpoint as storage in `HTTPClient.Request`
* fix merge conflicts
* rename Endpoint to DeconstructedURL
* swift-format
* make `DeconstructedURL` properties `var`'s
* move scheme into global namespace
- rename `useTLS` to `usesTLS` where posible without breaking public API
- only import Foundation.URL
* fix review comments
### Motivation
Our current swiftformat version does not support async/await. Since we want to add support for async/await we must update swiftformat or disable it. I tried my very best to keep the number of changes as small as possible. I assume we want to stick with the new 0.48.8 for some time.
### Changes
- Update swiftformat to 0.48.8
### Result
We can land async/await code.
- The connection creation logic has been refactored into a number of smaller methods that can be combined
- Connection creation now has a logical home. It is moved from `Utils.swift` into a `ConnectionFactory`
- There are explicit `ChannelHandlers` that are used for connection creation:
- `TLSEventsHandler` got its own file and unit tests
- `HTTP1ProxyConnectHandler` got its own file and unit tests
- `SOCKSEventsHandler` got its own file and unit tests
- Some small things are already part of this pr that will get their context later. For example:
- `HTTPConnectionPool` is added as a namespace to not cause major renames in follow up PRs
- `HTTPConnectionPool.Connection.ID` and its generator were added now. (This will be used later to identify a connection during its lifetime)
- the file `HTTPConnectionPool+Manager` was added to give `HTTPConnectionPool.Connection.ID.Generator` already its final destination.
Cory Benfield
George Barnett
Clinton Nkwocha
Allan Shortlidge
Rick Newton-Rogers
Anthony Doeraene
David Nadoba
Fabian Fett
Karl