Motivation:
Swift 6.0 has dropped out of the support policy
Modifications:
- Remove Package@swift-6.0.swift
- Ensure min tools version is 6.1
Result:
AHC only supports Swift 6.1+
Motivation
In machines with more complex network topologies it is possible for us
to have multiple possible NICs we might want to use for a request. Users
may wish to vary this on a per-request or even a per-client basis.
This control can typically be expressed by offering a local address to
bind to before making the connection attempt.
Modifications
Allow users to express a preferred local address at request or client
scope.
Make this part of the connection pool key.
Bind the local address when specified.
Test all of this.
Results
More capable clients.
Replaces all the foundation imports.
One issue is that `HTTPClient.init?(httpsURLWithSocketPath socketPath:
String, uri: String = "/")` uses `addingPercentEncoding()` from
Foundation. So instead, we use a pure Swift impl. that does the same.
We also need to disable default traits from `swift-configuration` to
prevent linking Foundation, because the `JSON` trait does that.
This also adds a linkage test to prevent regressions to CI.
Add a configuration option to retain the HTTP method and body receiving
301 or 302 responses.
Currently we automatically change the method to GET, and remove the
body, before following a 301 or 302. This is compliant with the fetch
specification: https://fetch.spec.whatwg.org/#http-redirect-fetch
However, it is useful to be able to override this behaviour and retain
the method and body.
Changes
- Add a new struct to encapsulate the (now 4) arguments of the follow
case of the redirect mode
- Add new options `retainHTTPMethodAndBodyOn301` and
`retainHTTPMethodAndBodyOn302`. Defaults to false because thats the
existing behaviour today
- When it is true, do not convert requests to GET after following a
redirect
- Note: this does not affect 307/308 (or any other) redirects. They
always preserve their method
---------
Co-authored-by: Fabian Fett <fabianfett@apple.com>
Swift log now has an InMemoryLogHandler. Lets depend on that instead of
having our own `CollectEverythingLogHandler`.
I've added an extension on top, to make it easier to create the logger
too
Result: less code
Motivation:
The tests shouldn't be making sendability violations.
Modifications:
Fix the warnings
Result:
- No warnings
- Strict concurrency is adopted!
Motivation
We have some Task error handling functions that are generic for no
apparent reason. They're also typically called from contexts where they
also report the error to the delegate, but one of the call sites doesn't
do that. So add a test for that as well.
Modifications
- Rewrite Task.fail(with:delegate:) to be non-generic.
- Add a call to the delegate error handler on the path that is missing
it.
- Add a test for that call
Results
Cleaner, easier to follow code
Motivation:
As requested in #596, it can be handy to have a lower-level access to
channels (HTTP/1 connection, HTTP/2 connection, or HTTP/2 stream) to
enable a more fine-grained interaction for, say, observability, testing,
etc.
Modifications:
- Add 3 new properties (`http1_1ConnectionDebugInitializer`,
`http2ConnectionDebugInitializer` and
`http2StreamChannelDebugInitializer`) to `HTTPClient.Configuration` with
access to the respective channels. These properties are of `Optional`
type `@Sendable (Channel) -> EventLoopFuture<Void>` and are called when
creating a connection/stream.
Result:
Provides APIs for a lower-level access to channels.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: David Nadoba <d_nadoba@apple.com>
Co-authored-by: George Barnett <gbarnett@apple.com>
I needed a way to use a `FileDownloadDelegate` task to fish out the
recommended file name.
```swift
let response = try await downloadTask.get()
// access content-disposition
response.head.headers.first(name: "Content-Disposition")
```
The `head` property is an explicitly unwrapped optional because there is
no "default value" to set it to, and it won't be accessed by the user
until it's already been set anyway. This is a little inelegant, so I
could change it to something like below where I fill in bogus init data,
but that seems worse for some reason.
```swift
public struct Progress: Sendable {
public var totalBytes: Int?
public var receivedBytes: Int
public var head: HTTPResponseHead
}
private var progress = Progress(
totalBytes: nil,
receivedBytes: 0,
head: .init(
version: .init(major: 0, minor: 0),
status: .badRequest
)
)
```
Motivation
Foundation has reverted several of the changes of behaviour in the URL
type, leaving 6.0 and 6.1 with a different behaviour on non-Apple
platforms than all other versions.
We should tolerate that.
Modifications
Update the tests to understand the difference.
Result
Tests pass
# Motivation
The NIO 2.78 release introduced a bunch of new warnings. These warnings
cause us a bunch of trouble, so we should fix them.
# Modifications
Mostly use a bunch of assumeIsolated() and syncOperations.
# Result
CI passes again.
Note that Swift 6 has _many_ more warnings than this, but we expect more
to come and we aren't using warnings-as-errors on that mode at the
moment. We'll be cleaning that up soon.
Enable MemberImportVisibility check on all targets. Use a standard
string header and footer to bracket the new block for ease of updating
in the future with scripts.
Migrate CI to use GitHub Actions.
### Motivation:
To migrate to GitHub actions and centralised infrastructure.
### Modifications:
Changes of note:
* Adopt swift-format using rules from SwiftNIO.
* Remove scripts and docker files which are no longer needed.
* Disabled warnings-as-errors on Swift 6.0 CI pipelines for now.
### Result:
Feature parity with old CI.
Motivation:
As an HTTP library, async-http-client should have authentication
support.
Modifications:
This adds a `setBasicAuth()` method to both HTTPClientRequest and
`HTTPClient.Request` and their related unit tests.
Result:
Library users will be able to leverage this method to use basic
authentication on their requests without implementing this in their own
projects.
Note: I also ran the tests (`swift test`) with the
`docker.io/library/swift:6.0-focal` and
`docker.io/library/swift:5.10.1-focal` to ensure linux compatibility.
Signed-off-by: Agam Dua <agam_dua@apple.com>
Multipath TCP (MPTCP) is a TCP extension allowing to enhance the
reliability of the network by using multiple interfaces. This extension
provides a seamless handover between interfaces in case of deterioration
of the connection on the original one. In the context of iOS and Mac OS
X, it could be really interesting to leverage the capabilities of MPTCP
as they could benefit from their multiple interfaces (ethernet + Wi-fi
for Mac OS X, Wi-fi + cellular for iOS).
This contribution introduces patches to HTTPClient.Configuration and
establishment of the Bootstraps. A supplementary field "enableMultipath"
was added to the configuration, allowing to request the use of MPTCP.
This flag is then used when creating the channels to configure the
client.
Note that in the future, it might also be potentially interesting to
offer more precise configuration options for MPTCP on MacOS, as the
Network framework allows also to select a type of service, instead of
just offering the option to create MPTCP connections. Currently, when
enabling MPTCP, only the Handover mode is used.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
### Motivation:
- The properties that store the request body length and the cumulative number of bytes sent as part of a request are of type `Int`.
- On 32-bit devices, when sending requests larger than `Int32.max`, these properties overflow and cause a crash.
- To solve this problem, the properties should use the explicit `Int64` type.
### Modifications:
- Changed the type of the `known` field of the `RequestBodyLength` enum to `Int64`.
- Changed the type of `expectedBodyLength` and `sentBodyBytes` in `HTTPRequestStateMachine` to `Int64?` and `Int64` respectively.
- Deprecated the `public var length: Int?` property of `HTTPClient.Body` and backed it with a new property: `contentLength: Int64?`
- Added a new initializer and "overloaded" the `stream` function in `HTTPClient.Body` to work with the new `contentLength` property.
- **Note:** The newly added `stream` function has different parameter names (`length` -> `contentLength` and `stream` -> `bodyStream`) to avoid ambiguity problems.
- Added a test case that streams a 3GB request -- verified this fails with the types of the properties set explicitly to `Int32`.
### Result:
- 32-bit devices can send requests larger than 2GB without integer overflow issues.
It was previously accidentally moved to a different test, which does not need it
Async tests need the guard for macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0
## Motivation
Some of the test code was missing availability guards for Apple platforms, resulting in build failures for these platforms, e.g.
```
error: 'AsyncSequence' is only available in iOS 13.0 or newer
```
## Modifications
Add missing availability guards. I've tried to keep them as scoped as possible.
## Result
Tests can now build for run on iOS and other Apple platforms.
Motivation
URL parsing has changed in macOS Sonoma and associated releases. Our
tests were reliant on the old behaviour. The behaviour is controlled
by the OS on which the program was linked, which makes it impossible
for us to programmatically work out which result we should see.
The affected tests are not actually useful, we don't really care how
the URLs parse, so we can safely just remove them.
Modifications
Remove the affected tests.
Result
Tests pass!
Motivation:
Sometimes it can be helpful to limit the number of times a connection
can be used before discarding it. AHC has no such support for this at
the moment.
Modifications:
- Add a `maximumUsesPerConnection` configuration option which defaults
to `nil` (i.e. no limit).
- For HTTP1 we count down uses in the state machine and close the
connection if it hits zero.
- For HTTP2, each use maps to a stream so we count down remaining uses
in the state machine which we combine with max concurrent streams to
limit how many streams are available per connection. We also count
remaining uses in the HTTP2 idle handler: we treat no remaining uses
as receiving a GOAWAY frame and notify the pool which then drains the
streams and replaces the connection.
Result:
Users can control how many times each connection can be used.
* Reproducer
* Refactor test case
* Refactor tests
* Remove debugging artefacts
* Fix typo
* Fix formatting
* Remove `promise?.succeed(())`
* Add test for HTTP2 request with large header
Motivation
We currently don't handle large headers well which trigger a channel writability change event.
Modification
Add failing (but currently skipped) tests which reproduces the issue
Result
We can reliably reproduce the large request header issue in an integration and unit test.
Note that the actual fix is not included to make reviewing easier and will come in a follow up PR.
* Remove logging
* Fix crash for large HTTP request headers
Fix crash for when sending HTTP request headers result in a channel writability change event
* Formatting and linux tests
* Formatting and linux tests
* Generate linux tests
* Use previous default max concurrent streams value of 10
* Fix crash if request is canceled after request header is send
* generate linux tests and run swift format
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation
We currently don't handle large headers well which trigger a channel writability change event.
Modification
Add failing (but currently skipped) tests which reproduces the issue
Result
We can reliably reproduce the large request header issue in an integration and unit test.
Note that the actual fix is not included to make reviewing easier and will come in a follow up PR.
* Handle ResponseAccumulator not being able to buffer large response in memory
Check content length header for early exit
* Add test which currently hangs indefinitely
* Run `generate_linux_tests.rb` and `SwiftFormat`
* Print type and value if assert fails
* Run `generate_linux_tests.rb` and `SwiftFormat`
* Remove duplicate test due too merge conflict
* Validate that maxBodySize is positive
* Address review comments
Motivation
When receiving certain patterns of response body parts, we can end up
recursing almost indefinitely to deliver them to the application. This
can lead to crashes, so we might politely describe it as "sub-optimal".
Modifications
Keep track of our stack depth and avoid creating too many stack frames.
Added some unit tests.
Result
We no longer explode when handling bodies with lots of tiny parts.
Co-authored-by: David Nadoba <d_nadoba@apple.com>
Motivation
When users stream their bodies they may still want to send Connection:
close headers and terminate the connection early. This should work
properly.
Unfortunately it became clear that we didn't correctly pass the
information that the connection needed to be closed. As a result, we'd
inappropriately re-use the connection, potentially causing unnecessary
HTTP errors.
Modifications
Signal whether the connection needs to be closed when the final
connection action is to send .end.
Results
We behave better with streaming uploads.
Motivation
In some cases, the last thing that happens in a request-response pair is
that we send our HTTP/1.1 .end. This can happen when the peer has sent
an early response, before we have finished uploading our body. When it
does, we need to be diligent about cleaning up our connection state.
Unfortunately, there was an edge in the HTTP1ConnectionStateMachine that
processed .succeedRequest but that did not transition the state into
either .idle or .closing. That was an error, and needed to be fixed.
Modifications
Transition to .idle when we're returning
.succeedRequest(.sendRequestEnd).
Result
Fewer crashes
George Barnett
Cory Benfield
Mads Odgaard
hamzahrmalik
Clinton Nkwocha
Greg Cotten
Rick Newton-Rogers
Agam Dua
Anthony Doeraene
aryan-25
Johannes Weiss
Gustavo Cairo
Si Beaumont
David Nadoba
carolinacass