Replaces all the foundation imports.
One issue is that `HTTPClient.init?(httpsURLWithSocketPath socketPath:
String, uri: String = "/")` uses `addingPercentEncoding()` from
Foundation. So instead, we use a pure Swift impl. that does the same.
We also need to disable default traits from `swift-configuration` to
prevent linking Foundation, because the `JSON` trait does that.
This also adds a linkage test to prevent regressions to CI.
Add a configuration option to retain the HTTP method and body receiving
301 or 302 responses.
Currently we automatically change the method to GET, and remove the
body, before following a 301 or 302. This is compliant with the fetch
specification: https://fetch.spec.whatwg.org/#http-redirect-fetch
However, it is useful to be able to override this behaviour and retain
the method and body.
Changes
- Add a new struct to encapsulate the (now 4) arguments of the follow
case of the redirect mode
- Add new options `retainHTTPMethodAndBodyOn301` and
`retainHTTPMethodAndBodyOn302`. Defaults to false because thats the
existing behaviour today
- When it is true, do not convert requests to GET after following a
redirect
- Note: this does not affect 307/308 (or any other) redirects. They
always preserve their method
---------
Co-authored-by: Fabian Fett <fabianfett@apple.com>
Motivation
We have some Task error handling functions that are generic for no
apparent reason. They're also typically called from contexts where they
also report the error to the delegate, but one of the call sites doesn't
do that. So add a test for that as well.
Modifications
- Rewrite Task.fail(with:delegate:) to be non-generic.
- Add a call to the delegate error handler on the path that is missing
it.
- Add a test for that call
Results
Cleaner, easier to follow code
Motivation:
The response accumulator is a delegate which must be sendable as it's
passed across isolation domains.
Modifications:
- Make delegates have a sendable requirement
- Make the response accumulator sendable
Result:
Delegates, and the response accumulator, are sendable
Motivation:
The body stream writer can be sent across isolation domains so should be
sendable.
Modifications:
- Make it explicitly sendable
- Add appropriate preconcurrency annotations
- Wrap an iterator from swift-algorithms as it hasn't yet annotated its
types with Sendable
Result:
Body stream writer is sendable
Trying to pave the way for closing
https://github.com/swift-server/async-http-client/issues/790 with some
direction from @Lukasa.
I have no idea where is best to insert this new delegate method. I'm
currently doing it first thing in `receiveResponseHead0`, and not using
an EventLoopFuture for back pressure management. The state machine seems
pretty fragile and I don't want to leave too much of an imprint. Trying
to be a part of an EventLoopFuture chain seems really complicated and
would really leave a mark on the codebase, so I'm wondering if it's
possible to just warn the user "do not block"?
Anyway, just a jumping-off point and happy to take direction!
At the moment, `HTTPClient`'s entire API surface violates Structured
Concurrency. Both the creation & shutdown of a HTTP client as well as
making requests (#807) doesn't follow Structured Concurrency. Some of
the problems are:
1. Upon return of methods, resources are still in active use in other
threads/tasks
2. Cancellation doesn't always work
This PR is baby steps towards a Structured Concurrency API, starting
with start/shutdown of the HTTP client.
Co-authored-by: Johannes Weiss <johannes@jweiss.io>
fixes#784
`writeChunks` had 3 bugs:
1. An actually wrong `UnsafeMutableTransferBox` -> removed that type
which should never be created
2. A loooong future chain (instead of one final promise) -> implemented
3. Potentially infinite recursion which lead to the crash in #784) ->
fixed too
Migrate CI to use GitHub Actions.
### Motivation:
To migrate to GitHub actions and centralised infrastructure.
### Modifications:
Changes of note:
* Adopt swift-format using rules from SwiftNIO.
* Remove scripts and docker files which are no longer needed.
* Disabled warnings-as-errors on Swift 6.0 CI pipelines for now.
### Result:
Feature parity with old CI.
Motivation:
As an HTTP library, async-http-client should have authentication
support.
Modifications:
This adds a `setBasicAuth()` method to both HTTPClientRequest and
`HTTPClient.Request` and their related unit tests.
Result:
Library users will be able to leverage this method to use basic
authentication on their requests without implementing this in their own
projects.
Note: I also ran the tests (`swift test`) with the
`docker.io/library/swift:6.0-focal` and
`docker.io/library/swift:5.10.1-focal` to ensure linux compatibility.
Signed-off-by: Agam Dua <agam_dua@apple.com>
### Motivation:
- The properties that store the request body length and the cumulative number of bytes sent as part of a request are of type `Int`.
- On 32-bit devices, when sending requests larger than `Int32.max`, these properties overflow and cause a crash.
- To solve this problem, the properties should use the explicit `Int64` type.
### Modifications:
- Changed the type of the `known` field of the `RequestBodyLength` enum to `Int64`.
- Changed the type of `expectedBodyLength` and `sentBodyBytes` in `HTTPRequestStateMachine` to `Int64?` and `Int64` respectively.
- Deprecated the `public var length: Int?` property of `HTTPClient.Body` and backed it with a new property: `contentLength: Int64?`
- Added a new initializer and "overloaded" the `stream` function in `HTTPClient.Body` to work with the new `contentLength` property.
- **Note:** The newly added `stream` function has different parameter names (`length` -> `contentLength` and `stream` -> `bodyStream`) to avoid ambiguity problems.
- Added a test case that streams a 3GB request -- verified this fails with the types of the properties set explicitly to `Int32`.
### Result:
- 32-bit devices can send requests larger than 2GB without integer overflow issues.
Motivation:
Now that Swift 5.9 is GM we should update the supported versions and
remove 5.6
Modifications:
* Update `Package.swift`
* Remove `#if swift(>=5.7)` guards
* Delete the 5.6 docker compose file and make a 5.10 one
* Update docs
Result:
Remove support for Swift 5.6, add 5.10
* ChunkedCollection
* Use swift-algorithms
* SwiftFormat
* test chunking
* add documentation
* SwiftFormat
* fix old swift versions
* fix older swift versions
second attempt
* fix old swift versions
third attempt
* fix old swift versions
fourth attempt
* update documentation
Motivation
Task.wait() is a convenience method to avoid needing to wait for the
response future. This has had the effect of "laundering" a noasync
warning from EventLoopFuture.wait(), hiding it within a purely sync call
that may itself be used in an async context.
We should discourage using these and prefer using .get() instead.
Modifications
Mark Task.wait() noasync.
Add Task.get() for backward compatibility.
Result
Safer migration to Swift concurrency
Motivation:
This change was proposed in issue
[#389](https://github.com/swift-server/async-http-client/issues/389).
Users writing their own `HttpClientResponseDelegate` implementation
might want to emit log messages to the `task`'s `logger`.
Modifications:
Changed the access level of `Task`'s `logger` property from `internal`
to `public`.
Result:
Users can access the `logger` property of a `Task` outside of the
`async-http-client`.
* Handle ResponseAccumulator not being able to buffer large response in memory
Check content length header for early exit
* Add test which currently hangs indefinitely
* Run `generate_linux_tests.rb` and `SwiftFormat`
* Print type and value if assert fails
* Run `generate_linux_tests.rb` and `SwiftFormat`
* Remove duplicate test due too merge conflict
* Validate that maxBodySize is positive
* Address review comments
SwiftNIO 2.42.0 has deprecated Lock and replaced it with a new NIOLock. This commit removes all uses of Lock and replaces them with NIOLock. Further, now, we must require SwiftNIO 2.42.0
* Fix thread leak in `FileDownloadDelegate`
* `SwiftFormat`
* Add a shared file IO thread pool per HTTPClient
* User bigger thread pool and initlize lazily during first file write
* thread pool is actually not used in tests
* Update documentation
* fix review comments
* make `fileIOThreadPool` internal
* Add test to verify that we actually share the same thread pool across all delegates for a given HTTPClient
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation
Documentation is nice, and we can help support users by providing useful
clear docs.
Modifications
Add Docc to 5.6 and later builds
Make sure symbol references work
Add overview docs
Result
Nice rendering docs
* refactor RedirectHandler
- `redirectState` is no longer a property of `HTTPClient.Request`. RedirectHandler now stores this state directly and therefore no longer optional.
- we no longer count the number of allowed redirects down. Instead the number of redirects is dervied from `self.visited.count` and we compare it to the maxRedirect to check if we git the limit.
* `HTTPClient.Configuration.RedirectConfiguration.Configuration` is now called `HTTPClient.Configuration.RedirectConfiguration.Mode`
only two `Configuration`s left in the type name
* add redirect logger test
* make `Scheme` a type
* introduce new Endpoint type
* use endpoint as storage in `HTTPClient.Request`
* fix merge conflicts
* rename Endpoint to DeconstructedURL
* swift-format
* make `DeconstructedURL` properties `var`'s
* move scheme into global namespace
- rename `useTLS` to `usesTLS` where posible without breaking public API
- only import Foundation.URL
* fix review comments
- Refactored code from `TaskHandler` into new method `HTTPClient.Request.createRequestHead` that creates an `HTTPRequestHead` and matching `RequestFramingMetadata`
- Added required property `requestFramingMetadata` to `HTTPExecutingRequest`
- Added property `requestFramingMetadata` to `RequestBag`
Co-authored-by: Cory Benfield <lukasa@apple.com>
- Adding four new protocols:
- `HTTPExecutingRequest` a protocol representing an HTTP task that is executed on a ChannelHandler.
- `HTTPScheduledRequest` a protocol representing an HTTP task that is scheduled for execution, but in a waiting state (example: Waiting for an idle connection in the connection pool).
- `HTTPRequestExecutor` a protocol that can be used from the `HTTPExecutingRequest` abstracting away functionality that will normally be implemented by a `ChannelHandler`
- `HTTPRequestScheduler` a protocol that can be used from the `HTTPScheduledRequest` abstracting away functionality that will normally be implemented by a `ConnectionPool`
- An implementation of the `HTTPExecutingTask` and `HTTPScheduledRequest` called `RequestBag`. It implements our current API using the new protocols
Co-authored-by: George Barnett <gbarnett@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
Mads Odgaard
hamzahrmalik
Cory Benfield
Konrad `ktoso` Malawski
George Barnett
Greg Cotten
Johannes Weiss
Rick Newton-Rogers
Agam Dua
aryan-25
Rick Newton-Rogers
David Nadoba
Fabian Fett
Felix Schlegel
Karl
David Evans
Mads Odgaard