Enable MemberImportVisibility check on all targets. Use a standard
string header and footer to bracket the new block for ease of updating
in the future with scripts.
Migrate CI to use GitHub Actions.
### Motivation:
To migrate to GitHub actions and centralised infrastructure.
### Modifications:
Changes of note:
* Adopt swift-format using rules from SwiftNIO.
* Remove scripts and docker files which are no longer needed.
* Disabled warnings-as-errors on Swift 6.0 CI pipelines for now.
### Result:
Feature parity with old CI.
Motivation:
We would like to make this work for Musl so that we can build fully
statically linked binaries that use AsyncHTTPClient.
Modifications:
Define `_GNU_SOURCE` as a compiler argument; doing it in a source file
doesn't work properly with modular headers.
Add imports of `Musl` in appropriate places.
`Musl` doesn't have `strptime_l`, so avoid using that.
Result:
async-http-client will build for Musl.
Sometimes it can be useful to connect to one host e.g. `x.example.com` but request and validate the certificate chain as if we would connect to `y.example.com`. This is what this PR adds support for by adding a `dnsOverride` configuration to `HTTPClient.Configuration`. This is similar to curls `—resolve-to` option but only allows overriding host and not ports for now.
* make `Scheme` a type
* introduce new Endpoint type
* use endpoint as storage in `HTTPClient.Request`
* fix merge conflicts
* rename Endpoint to DeconstructedURL
* swift-format
* make `DeconstructedURL` properties `var`'s
* move scheme into global namespace
- rename `useTLS` to `usesTLS` where posible without breaking public API
- only import Foundation.URL
* fix review comments
- The connection creation logic has been refactored into a number of smaller methods that can be combined
- Connection creation now has a logical home. It is moved from `Utils.swift` into a `ConnectionFactory`
- There are explicit `ChannelHandlers` that are used for connection creation:
- `TLSEventsHandler` got its own file and unit tests
- `HTTP1ProxyConnectHandler` got its own file and unit tests
- `SOCKSEventsHandler` got its own file and unit tests
- Some small things are already part of this pr that will get their context later. For example:
- `HTTPConnectionPool` is added as a namespace to not cause major renames in follow up PRs
- `HTTPConnectionPool.Connection.ID` and its generator were added now. (This will be used later to identify a connection during its lifetime)
- the file `HTTPConnectionPool+Manager` was added to give `HTTPConnectionPool.Connection.ID.Generator` already its final destination.
Motivation:
In the vast majority of cases, we'll only ever create one and only one
`NIOSSLContext`. It's therefore wasteful to keep around a whole thread
doing nothing just for that. A `DispatchQueue` is absolutely fine here.
Modification:
Run the `NIOSSLContext` creation on a `DispatchQueue` instead.
Result:
Fewer threads hanging around.
Motivation:
At the moment, AHC assumes that creating a `NIOSSLContext` is both cheap
and doesn't block.
Neither of these two assumptions are true.
To create a `NIOSSLContext`, BoringSSL will have to read a lot of
certificates in the trust store (on disk) which require a lot of ASN1
parsing and much much more.
On my Ubuntu test machine, creating one `NIOSSLContext` is about 27,000
allocations!!! To make it worse, AHC allocates a fresh `NIOSSLContext`
for _every single connection_, whether HTTP or HTTPS. Yes, correct.
Modification:
- Cache NIOSSLContexts per TLSConfiguration in a LRU cache
- Don't get an NIOSSLContext for HTTP (plain text) connections
Result:
New connections should be _much_ faster in general assuming that you're
not using a different TLSConfiguration for every connection.
Motivation:
Currently when either we or the server send Connection: close, we
correctly do not return that connection to the pool. However, we rely on
the server actually performing the connection closure: we never call
close() ourselves. This is unnecessarily optimistic: a server may
absolutely fail to close this connection. To protect our own file
descriptors, we should make sure that any connection we do not return
the pool is closed.
Modifications:
If we think a connection is closing when we release it, we now call
close() on it defensively.
Result:
We no longer leak connections when the server fails to close them.
Fixes#324.
Motivation:
Thread safety analyzer reports warning about observable state
leaving lock guarded code blocks. This PR address some of the warnings.
Analysis of the warning does not prove that we have a real bug and all the
warnings are considered as potential bugs.
Analyzer used is:
`docker run --rm -v $(pwd):/src -w /src swift:5.3.1 swift test -c
release --sanitize=thread --enable-test-discovery`
Modifications:
* accessor to count of connections in connection pool is guarded by lock
Result:
Most of thread safety warnings are addressed without modification of observable
code behaviour.
allowed only using exposed API.
Motivation:
Having a setter for internal state of ConnectionsState led to a subset
of test testing invalid invariants, for example when we have at the same
time an available connecion and a waiter, which is an invalid state of
the system.
Modifications:
* test of ConnectionsState
* ConnectionsState
* ConnectionPool
are modified so the state under tests is achieved only by a sequence of
modifications invoked by state API.
During modification some tests are eliminated as they were testing
artificial state, which can not be achieved by exposed APIs.
ConnectionsState is pruned from "replace" as in no valid state we can
have a situation when we can "replace" a connection.
Invalid invariants and tests are removed.
Result:
We do not have a way to modify state of the ConnectionsState by direct
interaction with private state of the object. Getter on the state is
considered harmless and used for tests only.
* Introduce helper methods for test of ConnectionsState
Motivation:
Issue #234 highlights that we directly manipulate ConnectionsState and
this commit prepares tests to be refactored to manipulate the state
by exposed APIs instead.
Modifications:
* introduce helper methods in ConnectionPoolTestsSupport.swift
Result:
* no observable changes
* Move Connection tests out of ConnectionsState tests into separate file.
Motivation:
Clean up of code to address issue #234 - here we move away
connection tests to separate files outside of ConnectionsState tests
so we will be able to work on the ConnectionsState in focussed mode.
Modifications:
Connection tests moved to separate files.
Result:
No observable changes.
* Gather Connection code into Connection.swift
Motivation:
For tests we will need a simple version of Connection, so here I gather
Connection code in one place and will generify ConnectionsState on next
commit.
Modifications:
Code of Connection is moved from multiple files into single
Connections.swift.
Result:
All tests are passing, no observable behaviour change.
* Introduce generic type ConnectionType into ConnectionsState
Motivation:
To rework tests of ConnectionsState we want to have a "simpler" version
of Connection to be used, therefore here we convert ConnectionsState to
support generic type ConnectionType. We will substitute current
Connection with a test version in follow up commit.
Modifications:
ConnectionsState is altered to work on generic type ConnectionType
instead of solid type Connection.
Users of ConnectionsState are modified to provide type Connection into
ConnectionType in this commit.
Result:
Test are passing, no observable behaviour change.
Motivation:
AsyncHTTPClient is not a simple piece of software and nowadays also
quite stateful. To debug issues, the user may want logging.
Modification:
Support passing a logger to the request methods.
Result:
Debugging simplified.
* Added tests for http+unix and https+unix url schemes
Motivation:
Using a base URL as the socket path only works when the URL object is maintained as long as possible through the stack. Additionally, it doesn't currently provide a way to use TLS over UNIX sockets.
Modifications:
Added two tests to test out the to-be supported URL schemes, http+unix, and https+unix, which encode the socket path as a %-escaped hostname, as some existing services already do.
Result:
Better UNIX domain socket support.
This is the continuation of the good work of @Yasumoto and @weissi in #135
The following code adds support for NIO Transport services. When the ConnectionPool asks for a connection bootstrap it is returned a NIOClientTCPBootstrap which wraps either a NIOTSConnectionBootstrap or a ClientBootstrap depending on whether the EventLoop we are running on is NIOTSEventLoop.
If you initialize an HTTPClient with eventLoopGroupProvider set to .createNew then if you are running on iOS, macOS 10.14 or later it will provide a NIOTSEventLoopGroup instead of a EventLoopGroup.
Currently a number of tests are failing. 4 of these are related to the NIOSSLUncleanShutdown error the others all seem related to various race conditions which are being dealt with on other PRs. I have tested this code with aws-sdk-swift and it is working on both macOS and iOS.
Things look into:
The aws-sdk-swift NIOTS HTTP client had issues with on Mojave. We should check if this is the case for async-http-client as well.
Co-authored-by: Joe Smith <yasumoto7@gmail.com>
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
motivation:
TLS handshake promise was leaked in some cases of failure (see #179)
changes:
- Avoid leaking promise
- Clearer completion flow for related futures
- Add testAvoidLeakingTLSHandshakeCompletionPromise test
* Close idle pool connections
Motivation: Pooled connections should close at some point (see #168)
Changes:
- Add new poolingTimeout property to HTTPClient.Configuration, it's
default value is .seconds(60), it can be set to nil if one wishes to
disable this timeout.
- Add relevant unit test
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
* Remove activity assertion in prepareForClose
motivation: assertion on activity sometimes fails but it turns out it is overly restrictive
changes:
- remove assertion and accept any given activity in prepareForClose
* test closing connections whilst syncShutdown
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
motivation: Better performance thanks to connection reuse
changes:
- Added a connection pool for HTTP/1.1
- All requests automatically use the connection pool
- Up to 8 parallel connections per (scheme, host, port)
- Multiple additional unit tests
Marc Prud'hommeaux
Rick Newton-Rogers
Alastair Houghton
David Nadoba
Karl
Fabian Fett
Johannes Weiss
Mads Odgaard
Cory Benfield
Ilya Teterin
Fabian Fett
Artem Redkin
Dimitri Bouniol
Artem Redkin
Adam Fowler
Trevör