Motivation:
As requested in #596, it can be handy to have a lower-level access to
channels (HTTP/1 connection, HTTP/2 connection, or HTTP/2 stream) to
enable a more fine-grained interaction for, say, observability, testing,
etc.
Modifications:
- Add 3 new properties (`http1_1ConnectionDebugInitializer`,
`http2ConnectionDebugInitializer` and
`http2StreamChannelDebugInitializer`) to `HTTPClient.Configuration` with
access to the respective channels. These properties are of `Optional`
type `@Sendable (Channel) -> EventLoopFuture<Void>` and are called when
creating a connection/stream.
Result:
Provides APIs for a lower-level access to channels.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: David Nadoba <d_nadoba@apple.com>
Co-authored-by: George Barnett <gbarnett@apple.com>
Trying to pave the way for closing
https://github.com/swift-server/async-http-client/issues/790 with some
direction from @Lukasa.
I have no idea where is best to insert this new delegate method. I'm
currently doing it first thing in `receiveResponseHead0`, and not using
an EventLoopFuture for back pressure management. The state machine seems
pretty fragile and I don't want to leave too much of an imprint. Trying
to be a part of an EventLoopFuture chain seems really complicated and
would really leave a mark on the codebase, so I'm wondering if it's
possible to just warn the user "do not block"?
Anyway, just a jumping-off point and happy to take direction!
I needed a way to use a `FileDownloadDelegate` task to fish out the
recommended file name.
```swift
let response = try await downloadTask.get()
// access content-disposition
response.head.headers.first(name: "Content-Disposition")
```
The `head` property is an explicitly unwrapped optional because there is
no "default value" to set it to, and it won't be accessed by the user
until it's already been set anyway. This is a little inelegant, so I
could change it to something like below where I fill in bogus init data,
but that seems worse for some reason.
```swift
public struct Progress: Sendable {
public var totalBytes: Int?
public var receivedBytes: Int
public var head: HTTPResponseHead
}
private var progress = Progress(
totalBytes: nil,
receivedBytes: 0,
head: .init(
version: .init(major: 0, minor: 0),
status: .badRequest
)
)
```
Specifically Swift 5.10 _on Intel on Ubuntu Noble (24.04)_ has a crazy
bug which leads to compilation failures in a `#if compiler(>=6.0)`
block: https://github.com/swiftlang/swift/issues/79285 .
This workaround fixes the compilation by _changing the whitespace_.
Thanks @gwynne for finding this workaround!
---------
Co-authored-by: Johannes Weiss <johannes@jweiss.io>
At the moment, `HTTPClient`'s entire API surface violates Structured
Concurrency. Both the creation & shutdown of a HTTP client as well as
making requests (#807) doesn't follow Structured Concurrency. Some of
the problems are:
1. Upon return of methods, resources are still in active use in other
threads/tasks
2. Cancellation doesn't always work
This PR is baby steps towards a Structured Concurrency API, starting
with start/shutdown of the HTTP client.
Co-authored-by: Johannes Weiss <johannes@jweiss.io>
### Motivation:
In some cases we can crash because of a precondition failure when the
write timeout fires and we aren't in the running state. This can happen
for example if the connection is closed whilst the write timer is
active.
### Modifications:
* Remove the precondition and instead take no action if the timeout
fires outside of the running state. Instead we take a new `Action`,
`.noAction` when the timer fires.
* Clear write timeouts upon request completion. When a request completes
we have no use for the idle write timer, we clear the read timer and we
should clear the write one too.
### Result:
Fewer crashes.
The supplied tests fails without these changes and passes with either of them.
# Motivation
The NIO 2.78 release introduced a bunch of new warnings. These warnings
cause us a bunch of trouble, so we should fix them.
# Modifications
Mostly use a bunch of assumeIsolated() and syncOperations.
# Result
CI passes again.
Note that Swift 6 has _many_ more warnings than this, but we expect more
to come and we aren't using warnings-as-errors on that mode at the
moment. We'll be cleaning that up soon.
Enable MemberImportVisibility check on all targets. Use a standard
string header and footer to bracket the new block for ease of updating
in the future with scripts.
fixes#784
`writeChunks` had 3 bugs:
1. An actually wrong `UnsafeMutableTransferBox` -> removed that type
which should never be created
2. A loooong future chain (instead of one final promise) -> implemented
3. Potentially infinite recursion which lead to the crash in #784) ->
fixed too
Migrate CI to use GitHub Actions.
### Motivation:
To migrate to GitHub actions and centralised infrastructure.
### Modifications:
Changes of note:
* Adopt swift-format using rules from SwiftNIO.
* Remove scripts and docker files which are no longer needed.
* Disabled warnings-as-errors on Swift 6.0 CI pipelines for now.
### Result:
Feature parity with old CI.
Motivation:
As an HTTP library, async-http-client should have authentication
support.
Modifications:
This adds a `setBasicAuth()` method to both HTTPClientRequest and
`HTTPClient.Request` and their related unit tests.
Result:
Library users will be able to leverage this method to use basic
authentication on their requests without implementing this in their own
projects.
Note: I also ran the tests (`swift test`) with the
`docker.io/library/swift:6.0-focal` and
`docker.io/library/swift:5.10.1-focal` to ensure linux compatibility.
Signed-off-by: Agam Dua <agam_dua@apple.com>
On modularised platforms, #771 broke things because it changed from
importing `Musl` or `Glibc` to importing just `locale_h`. The latter
understandably doesn't define `errno` or `EOVERFLOW`, so we get a build
failure.
Fixes#773.
### Motivation:
A performance test executing 100,000 sequential requests against a
simple
[`NIOHTTP1Server`](https://github.com/apple/swift-nio/blob/main/Sources/NIOHTTP1Server/README.md)
revealed that 7% of total run time is spent in the setter of the
`request` property in `HTTP1ClientChannelHandler` (GitHub Issue #754).
The poor performance comes from [processing the string interpolation
`"\(self.eventLoop)"`](https://github.com/swift-server/async-http-client/blob/6df8e1c17e68f0f93de2443b8c8cafca9ddcc89a/Sources/AsyncHTTPClient/ConnectionPool/HTTP1/HTTP1ClientChannelHandler.swift#L39C17-L39C75)
which under the hood calls a computed property.
This problem can entirely be avoided by storing `eventLoop.description`
when initializing `HTTP1ClientChannelHandler`, and using that stored
value in `request`'s setter, rather than computing the property each
time.
### Modifications:
- Created a new property `let eventLoopDescription:
Logger.MetadataValue` in `HTTP1ClientChannelHandler` that stores the
description of the `eventLoop` argument that is passed into the
initializer.
- Replaced the string interpolation `"\(self.eventLoop)"` in `request`'s
setter with `self.eventLoopDescription`.
### Result:
`HTTP1ClientChannelHandler.eventLoop`'s `description` property is cached
upon initialization rather than being computed each time in the
`request` property's setter.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Multipath TCP (MPTCP) is a TCP extension allowing to enhance the
reliability of the network by using multiple interfaces. This extension
provides a seamless handover between interfaces in case of deterioration
of the connection on the original one. In the context of iOS and Mac OS
X, it could be really interesting to leverage the capabilities of MPTCP
as they could benefit from their multiple interfaces (ethernet + Wi-fi
for Mac OS X, Wi-fi + cellular for iOS).
This contribution introduces patches to HTTPClient.Configuration and
establishment of the Bootstraps. A supplementary field "enableMultipath"
was added to the configuration, allowing to request the use of MPTCP.
This flag is then used when creating the channels to configure the
client.
Note that in the future, it might also be potentially interesting to
offer more precise configuration options for MPTCP on MacOS, as the
Network framework allows also to select a type of service, instead of
just offering the option to create MPTCP connections. Currently, when
enabling MPTCP, only the Handover mode is used.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
The Darwin module is slowly being split up, and as it gets further
along, it will stop importing some of the split-out modules like the one
for locale.h that provides newlocale() and other locale API. However,
there's a wrinkle that on platforms with xlocale, it's xlocale.h that
provides most of the POSIX locale.h functions and not locale.h, so
prefer the xlocale module when available.
### Motivation
If the channel's writability changed to false just before we finished a
request, we currently run into a precondition.
### Changes
- Remove the precondition and handle the case appropiatly
### Result
A crash less.
Since most of the servers now conform to http2, the change here updates
the behaviour of assuming the connection to be http2 and not http1 by
default. It will migrate to http1 if the server only supports http1.
One can set the `httpVersion` in `ClientConfiguration` to `.http1Only`
which will start with http1 instead of http2.
Additional Changes:
- Fixed an off by one error in the maximum additional general purpose
connection check
- Updated tests
---------
Co-authored-by: Ayush Garg <ayushgarg@apple.com>
Co-authored-by: David Nadoba <d_nadoba@apple.com>
Co-authored-by: Fabian Fett <fabianfett@apple.com>
### Motivation:
When a user wishes to make the connection pool create as many concurrent
connections as possible, a natural way to achieve this would be to set
`.max` to the `concurrentHTTP1ConnectionsPerHostSoftLimit` property.
```swift
HTTPClient.Configuration().connectionPool = .init(
idleTimeout: .hours(1),
concurrentHTTP1ConnectionsPerHostSoftLimit: .max
)
```
The `concurrentHTTP1ConnectionsPerHostSoftLimit` property is of type
`Int`. Setting it to `Int.max` leads to `Int.max` being passed as an
argument to `Array`s `.reserveCapacity(_:)` method, causing an OOM
issue.
Addresses Github Issue #751
### Modifications:
Capped the argument to `self.connections.reserveCapacity(_:)` to 1024 in
`HTTPConnectionPool.HTTP1Connections`
### Result:
Users can now set the `concurrentHTTP1ConnectionsPerHostSoftLimit`
property to `.max` without causing an OOM issue.
### Motivation:
- The properties that store the request body length and the cumulative number of bytes sent as part of a request are of type `Int`.
- On 32-bit devices, when sending requests larger than `Int32.max`, these properties overflow and cause a crash.
- To solve this problem, the properties should use the explicit `Int64` type.
### Modifications:
- Changed the type of the `known` field of the `RequestBodyLength` enum to `Int64`.
- Changed the type of `expectedBodyLength` and `sentBodyBytes` in `HTTPRequestStateMachine` to `Int64?` and `Int64` respectively.
- Deprecated the `public var length: Int?` property of `HTTPClient.Body` and backed it with a new property: `contentLength: Int64?`
- Added a new initializer and "overloaded" the `stream` function in `HTTPClient.Body` to work with the new `contentLength` property.
- **Note:** The newly added `stream` function has different parameter names (`length` -> `contentLength` and `stream` -> `bodyStream`) to avoid ambiguity problems.
- Added a test case that streams a 3GB request -- verified this fails with the types of the properties set explicitly to `Int32`.
### Result:
- 32-bit devices can send requests larger than 2GB without integer overflow issues.
* Make ConnectionPool's `retryConnectionEstablishment` public
* Unified tests to consistently use `enableFastFailureModeForTesting()`
* Add `retryConnectionEstablishment` as optional parameter to the initializer of `HTTPClient.Configuration.ConnectionPool`
* Reverted change to initializer to prevent API stability breakage
* Add parameterless initializer for `HTTPClient.Configuration.ConnectionPool`
* Moved default values for `HTTPClient.Configuration.ConnectionPool` to the property declarations, so they only have to be specified at one point
* Removed superfluous spaces
Co-authored-by: Cory Benfield <lukasa@apple.com>
* Re-added missing line break
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
We would like to make this work for Musl so that we can build fully
statically linked binaries that use AsyncHTTPClient.
Modifications:
Define `_GNU_SOURCE` as a compiler argument; doing it in a source file
doesn't work properly with modular headers.
Add imports of `Musl` in appropriate places.
`Musl` doesn't have `strptime_l`, so avoid using that.
Result:
async-http-client will build for Musl.
## Motivation
Some of the test code was missing availability guards for Apple platforms, resulting in build failures for these platforms, e.g.
```
error: 'AsyncSequence' is only available in iOS 13.0 or newer
```
## Modifications
Add missing availability guards. I've tried to keep them as scoped as possible.
## Result
Tests can now build for run on iOS and other Apple platforms.
Motivation:
Now that Swift 5.9 is GM we should update the supported versions and
remove 5.6
Modifications:
* Update `Package.swift`
* Remove `#if swift(>=5.7)` guards
* Delete the 5.6 docker compose file and make a 5.10 one
* Update docs
Result:
Remove support for Swift 5.6, add 5.10
* ChunkedCollection
* Use swift-algorithms
* SwiftFormat
* test chunking
* add documentation
* SwiftFormat
* fix old swift versions
* fix older swift versions
second attempt
* fix old swift versions
third attempt
* fix old swift versions
fourth attempt
* update documentation
Clinton Nkwocha
Greg Cotten
Johannes Weiss
Allan Shortlidge
Rick Newton-Rogers
Marc Prud'hommeaux
Cory Benfield
Agam Dua
Alastair Houghton
aryan-25
Anthony Doeraene
Ian Anderson
Fabian Fett
Ayush Garg
Andreas Ley
Mahdi Bahrami
Gustavo Cairo
Alastair Houghton
Si Beaumont
Rick Newton-Rogers
David Nadoba
brenno
Corey