## Summary
- Adds `HTTPClient.Configuration.DNSResolver` with `.system` (default)
and `.randomized`
cases. When `.randomized` is selected, `NIORandomizedDNSResolver` is
installed on the
underlying `ClientBootstrap`, shuffling addresses returned by
`getaddrinfo`.
- Motivation: enables DNS-based load balancing for services that publish
multiple A/AAAA
records (e.g. Kubernetes headless services). The system resolver returns
addresses in a
deterministic order (typically RFC 6724 destination-address selection),
which defeats
round-robin DNS — clients hammer the first address.
---------
Co-authored-by: Pavan sai kumar alladi <p_alladi@apple.com>
Co-authored-by: George Barnett <gbarnett@apple.com>
Motivation:
Swift 6.0 has dropped out of the support policy
Modifications:
- Remove Package@swift-6.0.swift
- Ensure min tools version is 6.1
Result:
AHC only supports Swift 6.1+
Motivation
In machines with more complex network topologies it is possible for us
to have multiple possible NICs we might want to use for a request. Users
may wish to vary this on a per-request or even a per-client basis.
This control can typically be expressed by offering a local address to
bind to before making the connection attempt.
Modifications
Allow users to express a preferred local address at request or client
scope.
Make this part of the connection pool key.
Bind the local address when specified.
Test all of this.
Results
More capable clients.
Replaces all the foundation imports.
One issue is that `HTTPClient.init?(httpsURLWithSocketPath socketPath:
String, uri: String = "/")` uses `addingPercentEncoding()` from
Foundation. So instead, we use a pure Swift impl. that does the same.
We also need to disable default traits from `swift-configuration` to
prevent linking Foundation, because the `JSON` trait does that.
This also adds a linkage test to prevent regressions to CI.
- Fix a `fatalError("Invalid state: idle")` crash in
`HTTP1ConnectionStateMachine.demandMoreResponseBodyParts()` that occurs
when a response completes before the request body finishes uploading
- The root cause is that `self.request` was only nilled out inside the
write-completion callback for `.sendRequestEnd`, creating a window where
`demandResponseBodyStream` could still see the old request and call into
the state machine after it had already transitioned to .idle
- The fix nils out `self.request` synchronously when handling
`.sendRequestEnd` (before the write completes), and moves
`requestBodyStreamSent()` to fire after the final action rather than
before it
---------
Co-authored-by: George Barnett <gbarnett@apple.com>
Add a configuration option to retain the HTTP method and body receiving
301 or 302 responses.
Currently we automatically change the method to GET, and remove the
body, before following a 301 or 302. This is compliant with the fetch
specification: https://fetch.spec.whatwg.org/#http-redirect-fetch
However, it is useful to be able to override this behaviour and retain
the method and body.
Changes
- Add a new struct to encapsulate the (now 4) arguments of the follow
case of the redirect mode
- Add new options `retainHTTPMethodAndBodyOn301` and
`retainHTTPMethodAndBodyOn302`. Defaults to false because thats the
existing behaviour today
- When it is true, do not convert requests to GET after following a
redirect
- Note: this does not affect 307/308 (or any other) redirects. They
always preserve their method
---------
Co-authored-by: Fabian Fett <fabianfett@apple.com>
Users should be able to use swift-configuration to create the http
client configuration object
Changes
- duplicate package.swift to create a separate version for 6.0 and 6.1,
since Configuration is 6.2+
- add helper to create http client configuration using ConfigReader
> ## Note:
> This is a long LLM generated PR description. However it captures very
well, what has been changed and has already been reduced for brevity.
The PR is sadly quite complex but I think the description captures the
changes quite well.
This is foundational work needed to properly support HTTP trailers and
scenarios where the server sends a complete response before the client
finishes uploading (e.g., early rejection, 100-continue flows, or
bidirectional streaming protocols).
## Changes
### State Machine Improvements
- **Added `endForwarded` state** to
`Transaction.StateMachine.RequestStreamState`
- This new state distinguishes between "request data forwarded to the
channel" and "request data written to the network"
- Properly handles the race condition where response completes before
the request write completes
- **Renamed `succeedRequest` → `forwardResponseEnd`** in both
`HTTPRequestStateMachine.Action` and
`HTTP1ConnectionStateMachine.Action`
- Better reflects the semantic meaning: we're forwarding the end of the
response stream, not necessarily succeeding the entire request yet
- More accurate naming for bidirectional streaming scenarios
### Protocol Changes
- **Added `requestBodyStreamSent()` to `HTTPExecutableRequest`
protocol**
- Called by the channel handler when the request body stream has been
fully written to the network
- Allows proper coordination between request and response stream
completion
- Implemented in both `Transaction` and `RequestBag`
### Request State Machine Updates
- **Updated `FinalSuccessfulRequestAction`**
- Changed `.sendRequestEnd(EventLoopPromise<Void>?)` to simpler
`.requestDone`
- Added `.none` case for when response completes but request is still
in-flight
- Removed the need to pass promises around, simplifying the state
machine
- **`sendRequestEnd` action now includes
`FinalSuccessfulRequestAction`**
- Allows the state machine to signal what should happen after the
request completes
- Enables proper cleanup coordination (idle connection, close, or
continue)
### Channel Handler Updates
- **HTTP1ClientChannelHandler**
- `sendRequestEnd` now properly handles scenarios where response has
already completed
- Added future callback to coordinate request completion with final
actions
- Properly manages connection state (idle vs close) based on both
streams completing
- **HTTP2ClientRequestHandler**
- Updated to handle new `sendRequestEnd` signature
- Properly ignores HTTP/1-specific final actions (like `.requestDone`)
### RequestBag State Machine
- **Added `endReceived` state to `ResponseStreamState`**
- Tracks when the response has completed while request is still ongoing
- Enables proper sequencing: response end → request end → task
completion
- **Updated `FinishAction`**
- Added `.forwardStreamFinishedAndSucceedTask` for the case where both
streams complete simultaneously
- Ensures delegate methods are called in the correct order
### Error Handling
- **Improved failure handling in `Transaction.StateMachine`**
- Now properly handles errors that occur after response completes but
before request finishes
- Added `cancelExecutor` action to the fail path
- Executor is now passed to `failRequestStreamContinuation` for proper
cleanup
## Technical Details
### The Problem
Previously, when a server sent a complete response before the client
finished uploading the request body, AHC would:
1. Receive the full response (head, body, end)
2. But NOT inform the user that the response was complete if the request
was still streaming
3. Only succeed the request after both streams completed
This made it impossible to implement proper bidirectional streaming or
handle scenarios like:
- Server rejecting a large upload early (e.g., 413 Payload Too Large)
- 100-continue flows where the server responds before request completes
- HTTP trailers sent by the server
### The Solution
The new state machine properly tracks four completion states:
1. **Neither complete**: Normal request/response in flight
2. **Response complete, request ongoing**: New
`endForwarded`/`endReceived` states
3. **Request complete, response ongoing**: Existing logic
4. **Both complete**: Request succeeds
The key insight is the `endForwarded` state, which represents "we've
given all request data to the channel, but it hasn't been written to the
network yet". This allows us to:
- Immediately forward response completion to the user
- Wait for the write to complete before cleaning up resources
- Properly sequence connection state transitions
## Future Work
This PR lays the groundwork for:
- Proper internal HTTP trailer support (both sending and receiving)
---------
Co-authored-by: George Barnett <gbarnett@apple.com>
Swift log now has an InMemoryLogHandler. Lets depend on that instead of
having our own `CollectEverythingLogHandler`.
I've added an extension on top, to make it easier to create the logger
too
Result: less code
Motivation:
The various 'withMumbleContinuation' APIs are supposed to be invoked
synchronously with the caller. This assumption allows a lock to be
acquired before the call and released from the body of the
'withMumbleContinuation' after e.g. storing the continuation. However
this isn't the case and the job may be re-enqueued on the executor
meaning that this is pattern is vulnerable to deadlocks.
Modifications:
- Rework the test helpers to avoid holding a lock when a continuation is
created.
- Switch to using NIOLockedValue box
Result:
Lower chance of deadlock
### Motivation
When creating a connection, we wrongfully assumed that
`failedToCreateNewConnection` will always be called before
`http*ConnectionClosed` in the `HTTPConnectionPoolStateMachine`. However
this is far from correct. In NIO Futures are fulfilled before
`ChannelHandler` callbacks. Ordering in futures should not be assumed in
such a complex project.
### Change
We change the `http*ConnectionClosed` methods to be noops, if the
connection is in the starting state. We instead wait for the
`failedToCreateNewConnection` to create backoff timers and friends.
rdar://164674912
---------
Co-authored-by: George Barnett <gbarnett@apple.com>
Motivation
Right now, we insert HTTP/2 handlers in a callback on a future that is
done very late. The result of this is that an entire ALPN negotiaton
_can_ complete before this callback is attached. That can in rare cases
cause the HTTP/2 handler to miss the server preamble, because it gets
added too late.
Modifications
This patch refactors the existing code to close that window. It does so
by passing a promise into the connection path and completing that
promise _on_ the event loop where we add the ALPN handlers, which should
ensure this will execute immediately when the ALPN negotiation
completes. Immportantly, we attach our promise callbacks to that promise
_before_ we hand it off, making sure the timing windows go away.
Results
Timing window is closed
Motivation
This patch adds support for HTTP/1 connection pre-warming. This allows
the user to request that the HTTP/1 connection pool create and maintain
extra connections, above-and-beyond those strictly needed to run the
pool. This pool can be used to absorb small spikes in request traffic
without increasing latency to account for connection creation.
Modifications
- Added new configuration properties for pre-warmed connections.
- Amended the HTTP/1 state machine to create new connections where
necessary.
- Added state machine tests.
Results
Pre-warmed connections are available.
Motivation:
testSimpleGetRequest asserts right after submitting the request for
execution that no streams have been closed. While unlikely there's
nothing to prevent this assertion from failing (and indeed it has
failed): the request may have executed immediately and the stream closed
before checking the delegate.
Modifications:
- Remove the assertion
Result:
Tests are less flaky
Motivation:
Now that strict concurrency has been adopted the AHC should avoid
regressing by treating all warnings as errors in CI.
Modifications:
- Treat warnings as errors in CI
Result:
Stricter CI
Motivation:
The tests shouldn't be making sendability violations.
Modifications:
Fix the warnings
Result:
- No warnings
- Strict concurrency is adopted!
Motivation:
There are a couple of sendability warnings leftover in Sources.
Transaction moves a closure into a task. The closure isn't Sendable (and
shouldn't be). However, higher up the stack there's a closure which
generates the non-sendable closure which can be sendable.
Modifications:
- Pass the sendable closure generating closure down rather
- Add a few more explicit sendable annotations
Result:
Fewer warnings
Motivation
We have some Task error handling functions that are generic for no
apparent reason. They're also typically called from contexts where they
also report the error to the delegate, but one of the call sites doesn't
do that. So add a test for that as well.
Modifications
- Rewrite Task.fail(with:delegate:) to be non-generic.
- Add a call to the delegate error handler on the path that is missing
it.
- Add a test for that call
Results
Cleaner, easier to follow code
Motivation:
Delegates can be passed from any thread and are executed on an arbitrary
event loop. That means they need to be Sendable. Rather than making them
all Sendable in one go, we'll do the larger ones separately.
Modifications:
- Make FileDownloadDelegate sendable
Result:
Safe to pass FileDownloadDelegate across isolation domains
Motivation:
The connection pool holds much of the low level logic in AHC. We should
fix its sendability issues before moving to higher levels.
Modifications:
- Make HTTP1ConnectionDelegate and HTTP2Delegate sendable, this requires
passing IDs rather than connections to their methods
- Make HTTPConnectionRequester sendable and have its methods take
Sendable views of the HTTP1Connection and HTTP2Connection types
- Add sendable views to HTTP1Connection and HTTP2Connection
- Mark HTTP1Connection and HTTP2Connection as not sendable
- Make HTTPRequestExecutor and HTTPExecutableRequest sendable
- Update tests
Result:
Connection pool has stricter sendability requirements
Motivation:
As requested in #596, it can be handy to have a lower-level access to
channels (HTTP/1 connection, HTTP/2 connection, or HTTP/2 stream) to
enable a more fine-grained interaction for, say, observability, testing,
etc.
Modifications:
- Add 3 new properties (`http1_1ConnectionDebugInitializer`,
`http2ConnectionDebugInitializer` and
`http2StreamChannelDebugInitializer`) to `HTTPClient.Configuration` with
access to the respective channels. These properties are of `Optional`
type `@Sendable (Channel) -> EventLoopFuture<Void>` and are called when
creating a connection/stream.
Result:
Provides APIs for a lower-level access to channels.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: David Nadoba <d_nadoba@apple.com>
Co-authored-by: George Barnett <gbarnett@apple.com>
Trying to pave the way for closing
https://github.com/swift-server/async-http-client/issues/790 with some
direction from @Lukasa.
I have no idea where is best to insert this new delegate method. I'm
currently doing it first thing in `receiveResponseHead0`, and not using
an EventLoopFuture for back pressure management. The state machine seems
pretty fragile and I don't want to leave too much of an imprint. Trying
to be a part of an EventLoopFuture chain seems really complicated and
would really leave a mark on the codebase, so I'm wondering if it's
possible to just warn the user "do not block"?
Anyway, just a jumping-off point and happy to take direction!
I needed a way to use a `FileDownloadDelegate` task to fish out the
recommended file name.
```swift
let response = try await downloadTask.get()
// access content-disposition
response.head.headers.first(name: "Content-Disposition")
```
The `head` property is an explicitly unwrapped optional because there is
no "default value" to set it to, and it won't be accessed by the user
until it's already been set anyway. This is a little inelegant, so I
could change it to something like below where I fill in bogus init data,
but that seems worse for some reason.
```swift
public struct Progress: Sendable {
public var totalBytes: Int?
public var receivedBytes: Int
public var head: HTTPResponseHead
}
private var progress = Progress(
totalBytes: nil,
receivedBytes: 0,
head: .init(
version: .init(major: 0, minor: 0),
status: .badRequest
)
)
```
Motivation
Foundation has reverted several of the changes of behaviour in the URL
type, leaving 6.0 and 6.1 with a different behaviour on non-Apple
platforms than all other versions.
We should tolerate that.
Modifications
Update the tests to understand the difference.
Result
Tests pass
Motivation
EmbeddedEventLoop is not thread-safe, which means that outside of very
rare use-cases it's not safe to use it in Swift Concurrency.
Modifications
Replace invalid uses of EmbeddedEventLoop with NIOAsyncTestingEventLoop
Result
Better safety
At the moment, `HTTPClient`'s entire API surface violates Structured
Concurrency. Both the creation & shutdown of a HTTP client as well as
making requests (#807) doesn't follow Structured Concurrency. Some of
the problems are:
1. Upon return of methods, resources are still in active use in other
threads/tasks
2. Cancellation doesn't always work
This PR is baby steps towards a Structured Concurrency API, starting
with start/shutdown of the HTTP client.
Co-authored-by: Johannes Weiss <johannes@jweiss.io>
### Motivation:
In some cases we can crash because of a precondition failure when the
write timeout fires and we aren't in the running state. This can happen
for example if the connection is closed whilst the write timer is
active.
### Modifications:
* Remove the precondition and instead take no action if the timeout
fires outside of the running state. Instead we take a new `Action`,
`.noAction` when the timer fires.
* Clear write timeouts upon request completion. When a request completes
we have no use for the idle write timer, we clear the read timer and we
should clear the write one too.
### Result:
Fewer crashes.
The supplied tests fails without these changes and passes with either of them.
# Motivation
The NIO 2.78 release introduced a bunch of new warnings. These warnings
cause us a bunch of trouble, so we should fix them.
# Modifications
Mostly use a bunch of assumeIsolated() and syncOperations.
# Result
CI passes again.
Note that Swift 6 has _many_ more warnings than this, but we expect more
to come and we aren't using warnings-as-errors on that mode at the
moment. We'll be cleaning that up soon.
Enable MemberImportVisibility check on all targets. Use a standard
string header and footer to bracket the new block for ease of updating
in the future with scripts.
fixes#784
`writeChunks` had 3 bugs:
1. An actually wrong `UnsafeMutableTransferBox` -> removed that type
which should never be created
2. A loooong future chain (instead of one final promise) -> implemented
3. Potentially infinite recursion which lead to the crash in #784) ->
fixed too
Migrate CI to use GitHub Actions.
### Motivation:
To migrate to GitHub actions and centralised infrastructure.
### Modifications:
Changes of note:
* Adopt swift-format using rules from SwiftNIO.
* Remove scripts and docker files which are no longer needed.
* Disabled warnings-as-errors on Swift 6.0 CI pipelines for now.
### Result:
Feature parity with old CI.
Motivation:
As an HTTP library, async-http-client should have authentication
support.
Modifications:
This adds a `setBasicAuth()` method to both HTTPClientRequest and
`HTTPClient.Request` and their related unit tests.
Result:
Library users will be able to leverage this method to use basic
authentication on their requests without implementing this in their own
projects.
Note: I also ran the tests (`swift test`) with the
`docker.io/library/swift:6.0-focal` and
`docker.io/library/swift:5.10.1-focal` to ensure linux compatibility.
Signed-off-by: Agam Dua <agam_dua@apple.com>
Multipath TCP (MPTCP) is a TCP extension allowing to enhance the
reliability of the network by using multiple interfaces. This extension
provides a seamless handover between interfaces in case of deterioration
of the connection on the original one. In the context of iOS and Mac OS
X, it could be really interesting to leverage the capabilities of MPTCP
as they could benefit from their multiple interfaces (ethernet + Wi-fi
for Mac OS X, Wi-fi + cellular for iOS).
This contribution introduces patches to HTTPClient.Configuration and
establishment of the Bootstraps. A supplementary field "enableMultipath"
was added to the configuration, allowing to request the use of MPTCP.
This flag is then used when creating the channels to configure the
client.
Note that in the future, it might also be potentially interesting to
offer more precise configuration options for MPTCP on MacOS, as the
Network framework allows also to select a type of service, instead of
just offering the option to create MPTCP connections. Currently, when
enabling MPTCP, only the Handover mode is used.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
The Darwin module is slowly being split up, and as it gets further
along, it will stop importing some of the split-out modules like the one
for locale.h that provides newlocale() and other locale API. However,
there's a wrinkle that on platforms with xlocale, it's xlocale.h that
provides most of the POSIX locale.h functions and not locale.h, so
prefer the xlocale module when available.
PAVANSAI KUMAR ALLADI
hamzahrmalik
George Barnett
Cory Benfield
Mads Odgaard
Fabian Fett
Konrad `ktoso` Malawski
Konrad `ktoso` Malawski
Clinton Nkwocha
Gus Cairo
Greg Cotten
Johannes Weiss
Rick Newton-Rogers
Marc Prud'hommeaux
Agam Dua
Anthony Doeraene
Ian Anderson