Commit Graph

50 Commits

Author SHA1 Message Date
Artem Redkin 4b4d6605aa Fix state reverting (#298)
* fail if we get part when state is endOrError

* Prevent TaskHandler state change after `.endOrError`

Motivation:
Right now if task handler encounters an error, it changes state to
`.endOrError`. We gate on that state to make sure that we do not
process errors in the pipeline twice. Unfortunately, that state
can be reset when we upload body or receive response parts.

Modifications:
Adds state validation before state is updated to a new value
Adds a test

Result:
Fixes #297
2020-08-24 11:28:28 +01:00
Artem Redkin ffcd1e1a1c Fixes double-release of a connection. (#295)
Motivation:
TaskHandler unconditionally releases it's connection on error,
this can lead to double release. This issue actually indicates
a more general issue where handler continues to handle errors
even after its state is `.endOrError`. We need to fix this by
ignoring all subsequent errors.

Modifications:
1. Check state before calling out delegate and pool
2. Replace all error callouts with call to `errorCaught`

Result:
Fixes #294
2020-08-20 15:35:06 +01:00
Tim Condon 1432843305 Fix typo in Task<Response> (#293) 2020-08-20 12:12:18 +01:00
Artem Redkin f69b68ffa8 fail if user tries writing bytes after request is sent (#270) 2020-07-30 11:22:55 +01:00
Dimitri Bouniol 5c7a317f5b Fixed an issue where redirects to socket path-based servers from any server was always allowed (#259)
* Added tests that ensure redirects to unix socket paths from a regular HTTP server are disallowed.

Motivation:

Currently, redirects to any supported URL scheme will always be allowed, despite code being in place to seemingly prevent it. See #230.

Modifications:

- Added a method to HTTPBin to redirect to the specified target.
- Added failing tests that perform redirects from a regular server to a socket-based server and vice versa.

Result:

Failing tests that show that the existing redirect checks were inadequate.

* Fixed an issue where redirects to socket path-based servers from any server was always allowed.

Motivation:

An arbitrary HTTP(S) server should not be able to trigger redirects, and thus activity, to a local socket-path based server, though the opposite may be a valid scenario. Currently, requests in either direction are allowed since the checks don't actually check the destination scheme.

Modifications:

- Refactored `hostSchemes`/`unixSchemes` to `hostRestrictedSchemes`/`allSupportedSchemes`, which better describes what they do.
- Refactored `Request.supports()` to `Request.supportsRedirects(to:)` since it is only used by Redirects now.
- Check the destination URL's scheme rather than the current URL's scheme when validating a redirect.

Result:

Closes #230

Co-authored-by: Artem Redkin <artem@redkin.me>
2020-06-22 17:56:01 +01:00
Artem Redkin c097c17e3e fix flaky testContentLengthTooLongFails test (#269) 2020-06-22 15:42:37 +01:00
Dimitri Bouniol ec48f4f114 Convenience methods for socket paths (#235)
* Added additional tests for socketPath-based requests

Motivation:

While going through the existing tests, I identified a few more instances where we could add some testing.

Modifications:

Added one test that verifies Requests are being decoded correctly, and improved three others to check for path parsing, error throwing, and schema casing respectively.

Result:

Tests that continue to pass, but that will also catch any incompatible changes in the future.

* Added some convenience initializers to URL and methods to Request for making requests to socket paths

Motivation:

Creating URLs for connecting to servers bound to socket paths currently requires some additional code to get exactly right. It would be nice to have convenience methods on both URL and Request to assist here.

Modifications:

- Refactored the get/post/patch/put/delete methods so they all call into a one line execute() method.
- Added variations on the above methods so they can be called with socket paths (both over HTTP and HTTPS).
- Added public convenience initializers to URL to support the above, and so socket path URLs can be easily created in other situations.
- Added unit tests for creating socket path URLs, and testing the new suite of convenience execute methods (that, er, test `HTTPMETHOD`s). (patch, put, and delete are now also tested as a result of these tests)
- Updated the read me with basic usage instructions.

Result:

New methods that allow for easily creating requests to socket paths, and passing tests to go with them.

* Removed some of the new public methods added for creating a socket-path based request

Motivation:

I previously added too much new public API that will most likely not be necessary, and can be better accessed using a generic execute method.

Modifications:

Removed the get/post/patch/put/delete methods that were specific to socket paths.

Result:

Less new public API.

* Renamed execute(url:) methods such that the HTTP method is the first argument in the parameter list

Motivation:

If these are intended to be general methods for building simple requests, then it makes sense to have the method be the first parameter in the list.

Modifications:

Moved the `method: HTTPMethod` parameter to the front of the list for all `execute([...] url: [...])` methods, and made it default to .GET. I also changed the url parameter to be `urlPath` for the two socketPath based execute methods.

Result:

A cleaner public interface for users of the API.

* Fixed some minor issues introduces with logging

Motivation:

Some of the convenience request methods weren't properly adapted for logging.

Modifications:

- Removed a doc comment from patch() that incorrectly referenced a logger.
- Fixed an issue where patch() would call into post().
- Added a doc comment to delete() that references the logger.
- Tests for the above come in the next commit...

Result:

Correct documentation and functionality for the patch() and delete() methods.

* Updated logging tests to also check the new execute methods

Motivation:

The logging tests previously didn't check for socket path-based requests.

Modifications:

Updated the `testAllMethodsLog()` and `testAllMethodsLog()` tests to include checks for each of the new `execute()` methods.

Result:

Two more tests that pass.
2020-06-19 10:51:03 +01:00
Johannes Weiss 8e60b94178 use standard ByteBuffer construction methods (#262) 2020-06-17 15:48:11 +01:00
Artem Redkin 606ab0eaea check body length (#255) 2020-06-16 09:17:09 +01:00
Adam Fowler 785ced571c The host header should also include the port (#237)
See https://tools.ietf.org/html/rfc7230#section-5.4

If port is not 80 or 443 then add to host header.
Fixed up tests
2020-06-15 15:46:29 +01:00
Johannes Weiss 86db162a11 logging support (#227)
Motivation:

AsyncHTTPClient is not a simple piece of software and nowadays also
quite stateful. To debug issues, the user may want logging.

Modification:

Support passing a logger to the request methods.

Result:

Debugging simplified.
2020-06-09 15:55:23 +01:00
Dimitri Bouniol 364d1069a4 Better support for UNIX Domain sockets (#228)
* Added tests for http+unix and https+unix url schemes

Motivation:

Using a base URL as the socket path only works when the URL object is maintained as long as possible through the stack. Additionally, it doesn't currently provide a way to use TLS over UNIX sockets.

Modifications:

Added two tests to test out the to-be supported URL schemes, http+unix, and https+unix, which encode the socket path as a %-escaped hostname, as some existing services already do.

Result:

Better UNIX domain socket support.
2020-06-02 15:22:20 +01:00
Johannes Weiss 070c1e5f37 cpool: don't reuse connection if we sent close (#225)
Motivation:

Previously, we'd only use the server's connection header to determine if
we should close the connection or not. That's wrong because if we set
`connection: close` ourselves, we must not reuse again.

Modification:

Set `TaskHandler.closing = false` if we send a close header.

Result:

More HTTP correctness.
2020-05-21 15:43:38 +01:00
Artem Redkin f81d0fec12 draft for streaming el fixes (#215) 2020-05-20 15:09:49 +01:00
Artem Redkin ce82178164 fix validation error propagation (#221) 2020-05-20 11:50:13 +01:00
Artem Redkin 8adfdb9bc5 refactor pool (#192) 2020-05-18 17:47:21 +01:00
Fabian Fett b72756af52 Typo fix: Trasfer-Encoding -> Transfer-Encoding (#211) 2020-05-15 14:30:00 +01:00
Iain Smith be517e3cc1 Enable clients to call URLs that include %2F as an escaped backslash (#201)
* Enable clients to call URLs that include %2F as an escaped backslash

Previously `percentEncodedPath` was using `path.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed)`
which converts %2F to a literal '/'. This prevented users fetching URLs like https://api.travis-ci.org/repo/github/rails%2Frails
which use %2F as part of a path segment.

Migrating to `URLComponents(url: self, resolvingAgainstBaseURL: false)?.percentEncodedPath` has the desired behaviour
for the couple of test cases that exist.

Updated the test server to switch on the `percentEncodedPath` so it's easier
to understand the desired behaviour.
2020-04-15 11:24:30 +01:00
Fabian Fett 2d88de3eb6 Verify header field names (#191)
* HTTPRequest without body: Content-Length shall not be send
* Verify field names comply with RFC7230
2020-04-01 16:48:01 +01:00
Artem Redkin 2fcf0a9fdd fix location header lookup (#186) 2020-03-31 09:56:44 +01:00
Artem Redkin 7211025f2a add version to response (#182)
* add version to response

* fix API breakage by deprecating old init

* review fix: use renamed instead of message
2020-03-26 16:46:59 +00:00
Trevör 9cdd1dc389 Close idle pool connections (#170)
* Close idle pool connections

Motivation: Pooled connections should close at some point (see #168)

Changes:
- Add new poolingTimeout property to HTTPClient.Configuration, it's
default value is .seconds(60), it can be set to nil if one wishes to
disable this timeout.
- Add relevant unit test

Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
2020-03-03 21:51:50 +00:00
Trevör 19e2ea727e Add an HTTP/1.1 connection pool (#105)
motivation: Better performance thanks to connection reuse

changes:
- Added a connection pool for HTTP/1.1
- All requests automatically use the connection pool
- Up to 8 parallel connections per (scheme, host, port)
- Multiple additional unit tests
2020-02-25 15:43:16 +00:00
Johannes Weiss de7421906c fix UDS without a baseURL (#165)
Previously, UNIX Domain Sockets would only work if the URL also had a
"base URL". If it didn't have a base URL, it would try to connect to the
empty string which would fail :).

Now, we support both cases:
- URLs with a baseURL (the path to the UDS) and a path (actual path)
- URLs that just have an actual path (path to the UDS) where we'll just
  use "/" as the URL's path
2020-02-24 17:52:36 +00:00
Johannes Weiss f648c04d18 some lets in Requests were vars by accident (#163) 2020-02-11 12:33:25 +00:00
Artem Redkin 70ad494183 add a public init to StreamWriter (#161) 2020-02-05 22:31:35 +00:00
Marcin Krzyzanowski e90f5fd03d Support UNIX Domain Sockets (#151)
Adds support for UNIX Domain Socket requests.

Usage:

```
let httpClient = HTTPClient(eventLoopGroupProvider: .createNew)
let socketURL = URL(string: "unix:///var/run/docker.sock")!
let req = try HTTPClient.Request(url: URL(string: "/users/list", relativeTo: socketURL)!, method: .GET)
let response = try httpClient.execute(request: req).wait()
```
2020-01-27 15:55:12 +00:00
Fabian Fett 84026a7ba0 ResponseAccumulator is now public (#155)
As discussed in #128. We make the ResponseAccumulator public to give developers an easy time to create a Task. With the ResponseAccumulator the developer using this does not have to worry, about implementing the HTTPClientResponseDelegate all by himself.
2020-01-20 17:11:39 +00:00
Johannes Weiss 7aeaf6f0e1 a set of extra test cases (#132)
Motivation:

More unit tests are good and now that SwiftNIO shipd
`NIOHTTP1TestServer`, writing integration tests for async-http-client is also
more straightforward.

Modification:

Demonstrate some tests using NIOHTTP1TestServer.

Result:

More tests.
2019-11-18 16:11:30 +01:00
tomer doron 018e139eef update copyrights note (#125)
motivation: the Swift Server Workgroup is not a legal entity and cannot hold copyrights. with this change, code authors continue and retain their copyrights under the apache license and previous copyrights note, but Apple steps up instead of the workgroup which has no legal status

changes:
* update header files to say "Apple Inc. and the AsyncHTTPClient project authors" instead of "Swift Server Workgroup and the AsyncHTTPClient project authors"
* update validation scripts to check for the correct header
* add CONTRIBUTING.md file to explain how to make contributions and include a legal notice about licensing the contribution to Apple and the project
* regenerate CONTRIBUTORS.md to reflect most recent contributions
2019-11-06 11:20:30 -08:00
Artem Redkin 51dc885a30 add support for redirect limits (#113) 2019-10-23 19:29:25 +03:00
Johannes Weiss c1c3da3990 remove all deprecations (#116) 2019-10-22 17:52:11 -07:00
Artem Redkin 191c4ba506 add response decompression support (#86)
fixes #44
2019-10-22 17:45:12 -07:00
Artem Redkin 54b0b96b74 cancel should be done with outbound user event (#110)
cancel should be done with outbound user event
2019-09-23 16:43:15 +01:00
Johannes Weiss 513be15050 EventLoop preference overhaul (#102) 2019-09-21 21:33:15 +02:00
Artem Redkin 79cd718c35 preserve trailing slash in uri path (#107) 2019-09-20 15:32:49 +01:00
Johannes Weiss 2d7ae2bbae minor: improve code style (#103) 2019-09-15 11:04:11 +01:00
Johannes Weiss 1b31c5a4e3 give generic parameters nice names (#99) 2019-09-09 17:54:21 +01:00
vkill 47de4bb5ec Add authorization to proxy (#94) 2019-09-08 14:30:24 +01:00
Trevör 244aea673e Tolerate futures from arbitrary event loops (#96)
This commit fixes #95 by always hopping event loop futures received from
the delegate to the right event loop. This could be a source of bugs if
the library users forgot to hop(to:) futures from their delegates
implementations.
2019-09-06 07:40:49 +01:00
Artem Redkin e9d0dd2cc9 rename didReceivePart to didReceiveBodyPart (#84) 2019-08-17 13:32:51 +01:00
vkill e0eeb0481c Ignore uncleanShutdown error when state is .head or .body (#77)
* Ignore uncleanShutdown error when state is head or body

* Add ignoreNIOSSLUncleanShutdownError to Configuration

* Revert old HTTPClient.init founctions

* Run generate_linux_tests.rb

* Rename ignoreNIOSSLUncleanShutdownError to ignoreUncleanSSLShutdown

* Make tests compatible with swift 5.0
2019-08-15 11:55:33 +01:00
Trevör 690e8ee797 Add @discardableResult to HTTPHandler setChannel method (#81)
The only place where this method is currently called uses its return
value, but that's specific to the context where it's used (inside of
chained calls to map/flatMap). Future uses might not always need to use
the return value since it is the same as what's passed as the method
argument.
2019-08-14 11:06:32 +01:00
vkill 912005b4f1 Add init to HTTPClient.Response (#75) 2019-08-12 15:15:00 +01:00
tomer doron e64448e1ef improve request validation (#67)
motivation: safer handling of request validation and mutation

changes:
* drop request version
* made request method and url immutable
* made request scheme and host internal
* fix scheme logic to be non-case sensitive
* adjusted redirect handler implementation to stricter request immutabllity
* adjust and add tests
2019-07-30 09:55:38 -07:00
tomer doron bab22d0ebd test formatting issues as part of sanity check (#73)
motivaiton: unified format

changes:
* fix outstanding formatting issues
* add a call to swiftformat as part of sanity script
* fix sanity script language check debugging statements
2019-07-30 10:00:10 +01:00
Artem Redkin 53d5bce8ee call didSendRequest after write future is fullfilled (#65) 2019-07-24 12:53:35 +01:00
Artem Redkin 0c617a4d7f document public API (#62)
Co-Authored-By: Joe Smith <yasumoto7@gmail.com>
2019-07-17 10:56:54 +01:00
Tobias daf66bde06 Make sure that event loop of task is the same as event loop of channel (#55) 2019-07-03 12:39:42 +01:00
Artem Redkin 27fe9265b2 rename to async-http-client (#58) 2019-07-02 14:00:34 +03:00