Motivation
In machines with more complex network topologies it is possible for us
to have multiple possible NICs we might want to use for a request. Users
may wish to vary this on a per-request or even a per-client basis.
This control can typically be expressed by offering a local address to
bind to before making the connection attempt.
Modifications
Allow users to express a preferred local address at request or client
scope.
Make this part of the connection pool key.
Bind the local address when specified.
Test all of this.
Results
More capable clients.
Add a configuration option to retain the HTTP method and body receiving
301 or 302 responses.
Currently we automatically change the method to GET, and remove the
body, before following a 301 or 302. This is compliant with the fetch
specification: https://fetch.spec.whatwg.org/#http-redirect-fetch
However, it is useful to be able to override this behaviour and retain
the method and body.
Changes
- Add a new struct to encapsulate the (now 4) arguments of the follow
case of the redirect mode
- Add new options `retainHTTPMethodAndBodyOn301` and
`retainHTTPMethodAndBodyOn302`. Defaults to false because thats the
existing behaviour today
- When it is true, do not convert requests to GET after following a
redirect
- Note: this does not affect 307/308 (or any other) redirects. They
always preserve their method
---------
Co-authored-by: Fabian Fett <fabianfett@apple.com>
Motivation:
The tests shouldn't be making sendability violations.
Modifications:
Fix the warnings
Result:
- No warnings
- Strict concurrency is adopted!
# Motivation
The NIO 2.78 release introduced a bunch of new warnings. These warnings
cause us a bunch of trouble, so we should fix them.
# Modifications
Mostly use a bunch of assumeIsolated() and syncOperations.
# Result
CI passes again.
Note that Swift 6 has _many_ more warnings than this, but we expect more
to come and we aren't using warnings-as-errors on that mode at the
moment. We'll be cleaning that up soon.
Enable MemberImportVisibility check on all targets. Use a standard
string header and footer to bracket the new block for ease of updating
in the future with scripts.
Migrate CI to use GitHub Actions.
### Motivation:
To migrate to GitHub actions and centralised infrastructure.
### Modifications:
Changes of note:
* Adopt swift-format using rules from SwiftNIO.
* Remove scripts and docker files which are no longer needed.
* Disabled warnings-as-errors on Swift 6.0 CI pipelines for now.
### Result:
Feature parity with old CI.
### Motivation:
When a user wishes to make the connection pool create as many concurrent
connections as possible, a natural way to achieve this would be to set
`.max` to the `concurrentHTTP1ConnectionsPerHostSoftLimit` property.
```swift
HTTPClient.Configuration().connectionPool = .init(
idleTimeout: .hours(1),
concurrentHTTP1ConnectionsPerHostSoftLimit: .max
)
```
The `concurrentHTTP1ConnectionsPerHostSoftLimit` property is of type
`Int`. Setting it to `Int.max` leads to `Int.max` being passed as an
argument to `Array`s `.reserveCapacity(_:)` method, causing an OOM
issue.
Addresses Github Issue #751
### Modifications:
Capped the argument to `self.connections.reserveCapacity(_:)` to 1024 in
`HTTPConnectionPool.HTTP1Connections`
### Result:
Users can now set the `concurrentHTTP1ConnectionsPerHostSoftLimit`
property to `.max` without causing an OOM issue.
### Motivation:
- The properties that store the request body length and the cumulative number of bytes sent as part of a request are of type `Int`.
- On 32-bit devices, when sending requests larger than `Int32.max`, these properties overflow and cause a crash.
- To solve this problem, the properties should use the explicit `Int64` type.
### Modifications:
- Changed the type of the `known` field of the `RequestBodyLength` enum to `Int64`.
- Changed the type of `expectedBodyLength` and `sentBodyBytes` in `HTTPRequestStateMachine` to `Int64?` and `Int64` respectively.
- Deprecated the `public var length: Int?` property of `HTTPClient.Body` and backed it with a new property: `contentLength: Int64?`
- Added a new initializer and "overloaded" the `stream` function in `HTTPClient.Body` to work with the new `contentLength` property.
- **Note:** The newly added `stream` function has different parameter names (`length` -> `contentLength` and `stream` -> `bodyStream`) to avoid ambiguity problems.
- Added a test case that streams a 3GB request -- verified this fails with the types of the properties set explicitly to `Int32`.
### Result:
- 32-bit devices can send requests larger than 2GB without integer overflow issues.
## Motivation
Some of the test code was missing availability guards for Apple platforms, resulting in build failures for these platforms, e.g.
```
error: 'AsyncSequence' is only available in iOS 13.0 or newer
```
## Modifications
Add missing availability guards. I've tried to keep them as scoped as possible.
## Result
Tests can now build for run on iOS and other Apple platforms.
* save progress
* Replace `TransactionBody` with `NIOAsyncSequenceProducer`
* test
* revert unnesscary changes
* Add end-to-end test
which currently fails because of https://github.com/apple/swift-nio/issues/2398
* soundness
* Use latest swift-nio release
* throw CancellationError on task cancelation
* Fix Swift 5.5 & 5.6
Motivation:
not accumulate too many bytes
Modifications:
Implementing collect function to use NIOCore version to prevent overflowing
Co-authored-by: Cory Benfield <lukasa@apple.com>
Sometimes it can be useful to connect to one host e.g. `x.example.com` but request and validate the certificate chain as if we would connect to `y.example.com`. This is what this PR adds support for by adding a `dnsOverride` configuration to `HTTPClient.Configuration`. This is similar to curls `—resolve-to` option but only allows overriding host and not ports for now.
Motivation
Task.wait() is a convenience method to avoid needing to wait for the
response future. This has had the effect of "laundering" a noasync
warning from EventLoopFuture.wait(), hiding it within a purely sync call
that may itself be used in an async context.
We should discourage using these and prefer using .get() instead.
Modifications
Mark Task.wait() noasync.
Add Task.get() for backward compatibility.
Result
Safer migration to Swift concurrency
* Add Swift 5.8 CI
* Update nightly CI to Ubuntu 22.04
* Fix warnings in test target with latest NIO release
Replace calls to `.wait()` with calls to `.get()`
Motivation
Allowing arbitrary data in outbound header field values allows for the
possibility that users of AHC will accidentally pass untrusted data into
those values. That untrusted data can substantially alter the parsing
and content of the HTTP requests, which is extremely dangerous. The
result of this is vulnerability to CRLF injection.
Modifications
Add validation of outbound header field values.
Result
No longer vulnerable to CRLF injection
* Tollerate more data after request body is cancelled
* wait is not needed if we shutdown the server first
* Remove test that depends on external resources
* Remove unused conformance to Equatable
* SwiftFormat
* run generate_linux_tests.rb
* Increase timeout for CI
Motivation
We should apply the connect timeout to the complete set of connection
attempts, rather than the request deadline. This allows users
fine-grained control over how long we attempt to connect for. This is
also the behaviour of our old-school interface.
Modifications
- Changed the connect deadline calculation for async/await to match that
of the future-based code.
- Added a connect timeout test.
Result
Connect timeouts are properly handled
### Motivation
If we follow a redirect which changes the origin e.g. from `127.0.0.1` to `localhost` we didn't change the `Host` header to the appropriate new origin and port combination.
### Changes
Use the original request which does not include the host instead of the prepared request to form a new request to the redirect URL.
### Alternatives
If the user defines a `Host` header themselves on the original `HTTPClientRequest` we currently never touch it, even in the redirect case. Maybe we should change our strategy and do one of the following:
1. We could always override the user defined `Host` header
2. We could only remove the user defined `Host` header on redirect and set it to the new origin and port combination
### Motivation
With Xcode 13.2, and therefore Swift 5.5.2, Swift Concurrecy is supported on older Apple OSs. async/await suport will no longer be available on Swift before `5.5.2` but this isn't a breaking change because we have not yet made anything of it public.
### Changes
- replace all `#if compiler(>=5.5) && canImport(_Concurrency)` with `#if compiler(>=5.5.2) && canImport(_Concurrency)`
- replace all `available(macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0, *)` with `available(macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0, *)`
* Fix HTTP1 to HTTP2 migration while shutdown is in progress
### Motivation
Calling `HTTPClient.shutdown()` may never return if connections are still starting and one new established connection results in a state migration (i.e. from HTTP1 to HTTP2 or vice versa). We forgot to migrate the shutdown state. This could result in a large dealy until `.shutdown()` returns because we wait until connections are closed because of idle timeout. Worse, it could also never return if more requests are queued because the connections would not be idle and therefore not close itself.
###Changes
- Mirgrate shutdown state too
- add tests for this specific case
* simplify testMigrationFromHTTP1ToHTTP2WhileShuttingDown
* add http2 to http1 migration test
Cory BenfieldandGitHub
George BarnettandGitHub
Gus CairoandGitHub
Greg CottenandGitHub
Rick Newton-RogersandGitHub
aryan-25andGitHub
hamzahrmalikandGitHub
Si BeaumontandGitHub
David NadobaandGitHub
