Motivation:
Swift 5.9 is no longer supported, we should bump the tools version and
remove it from our CI.
Modifications:
* Bump the Swift tools version to Swift 5.10
* Remove Swift 5.9 jobs where appropriate in main.yml, pull_request.yml
Result:
Code reflects our support window.
### Motivation
The `write(webSocketErrorCode:)`, `readWebSocketErrorCode()`, and
`getWebSocketErrorCode(at:)` methods in ByteBuffer were not covered by
unit tests
https://github.com/apple/swift-nio/pull/3174#pullrequestreview-2743588010.
Since a peekWebSocketErrorCode() method was recently added, I figured I
should verify both the new peek API and the existing WebSocket error
code APIs behave correctly.
### Modifications
Added tests for:
write(webSocketErrorCode:)
getWebSocketErrorCode(at:)
readWebSocketErrorCode()
Tests verify:
Correct encoding and decoding of WebSocket error codes
Reader index behavior (no movement for get, movement for read)
Handling of insufficient bytes
Consistency between peek-before-read and post-read
### Result
These tests ensure the WebSocket error code APIs in ByteBuffer are
correct, and safe to use.
Co-authored-by: Cory Benfield <lukasa@apple.com>
### Motivation:
Existing get APIs require passing an explicit index and can be misused,
leading to verbose and error-prone code. Adding peek variants that
automatically use the current readerIndex improves safety and clarity.
This aims to address issue #2034 and issue #2736, and is a continuation
of PR #3157
### Modifications:
Introduced peekSlice(), peekLengthPrefixedSlice(), peekData(),
peekUUIDBytes() and peekWebSocketErrorCode().
Added tests for each peek API covering normal, empty and repeated peek
scenarios.
### Result:
Developers can now use nonmutating peek APIs to inspect ByteBuffer
contents without altering the reader index.
### Motivation:
To ensure NIOHTTP1 concurrency safety.
### Modifications:
* Enable strict concurrency checking in the package manifest.
* Mark several objects `Sendable` with `@preconcurrency` annotations
where they are returned in futures which may execute in arbitrary
concurrency domains.
* `NIOTypedHTTPClientProtocolUpgrader`
* `NIOTypedHTTPClientUpgradeConfiguration`
* `NIOUpgradableHTTPServerPipelineConfiguration`
* `NIOUpgradableHTTPClientPipelineConfiguration`
* `NIOTypedHTTPServerProtocolUpgrader`
* `NIOTypedHTTPServerUpgradeConfiguration`
* Mark handlers as explicitly not sendable
* `NIOTypedHTTPClientUpgradeHandler`
* `NIOTypedHTTPServerUpgradeHandler`
* Added new Sendable type aliases:
* `NIOHTTPClientUpgradeSendableConfiguration`
* `NIOHTTPServerUpgradeSendableConfiguration`
### Result:
No more concurrency warnings. Builds will warn and CI will fail if
regressions are introduced.
Enhance `WebSocketProtocolErrorHandler` to correctly add masking key for
client/server.
### Motivation:
In `NIOWebSocket`, the automatic error handling provided by
`WebSocketProtocolErrorHandler` offers a convenient way for both clients
and servers to handle protocol-level errors. However, the
`WebSocketFrame` used in `WebSocketProtocolErrorHandler` does not fully
adhere to the RFC 6455 standard. Specifically, as per [RFC 6455 Section
5.1](https://datatracker.ietf.org/doc/html/rfc6455#section-5.1), a
client *must* mask all frames it sends to the server, while a server
*must not* mask any frames it sends to the client. This PR addressed
this discrepancy to ensure compliance with the WebSocket protocol.
### Modifications:
In the `WebSocketProtocolErrorHandler` initializer, the user can specify
whether the handler is used for a WebSocket client or server. Within the
`errorCaught` method, the `WebSocketFrame` will include a maskingKey if
the handler is used by a client, or nil if it is used by a server.
Additionally, the `@unchecked` annotation is removed from
`NIOWebSocketServerUpgrader` since the project is now using swift 5.9
toolchain.
### Result:
The `WebSocketProtocolErrorHandler` is more robust and can correctly
close the connection is error occurs.
Motivation:
6.0.3 added new warnings about Sendable issues, so let's fix those to
keep things warnings free.
Modifications:
A bunch of Sendable usage fixes.
Result:
Warnings free builds again!
Add support for treating WebSocketFrame reserved bits as an option set
### Motivation:
I would like to do mask operations on the reserved bits of the
WebSocketFrame eg check none are set, check only bits in a mask are set.
With the current public interface it is only possible to check the state
of one bit at a time.
### Modifications:
Add `WebSocketFrame.ReservedBits` OptionSet
Add computed member `WebSocketFrame.reservedBits`
### Result:
I can now check the status of multiple bits in one operation eg
`frame.reservedBits.isEmpty`
`frame.reservedBits.itersection([.rsv1, .rsv2]).isEmpty`
### Motivation:
Resolving the following issue:
https://github.com/apple/swift-nio/issues/2828
### Modifications:
Making `WebSocketFrame` conform to `CustomStringConvertible`.
### Result:
A nicer description for `WebSocketFrame`.
---------
Co-authored-by: Franz Busch <f.busch@apple.com>
* Apply formatting
* Apply no block comments rule
* Apply OmitExplicitReturns
* Apple OnlyOneTrailingClosureArgument
* Apply NoAssignmentInExpressions
* Fix up DontRepeatTypeInStaticProperties lint errors
* Apply `OrderedImports`
* Apply `ReplaceForEachWithForLoop`
* format file
* Enable the formatting pipeline
* Adopt `AmbiguousTrailingClosureOverload`
* Fix license header
* Fix format check
* Fix `EndOfLineComment`
* Fix CI
* Adapt CI script to check if changes when running formatting
* Separate lint and format into to steps
* Fix format
* Adopt `UseEarlyExits`
* Revert "Adopt `UseEarlyExits`"
This reverts commit d1ac5bbe12.
# Motivation
Adding a handler via the regular `pipeline.addHandler` APIs can lead to transferring the handler over isolation regions. To avoid running into `Sendable` warnings here we can use the `syncOperations` of the pipeline to avoid transferring the handlers.
# Modification
This PR is mostly changing code to avoid transferring handlers to and from the event loops in tests. This should be a no-op mostly.
Co-authored-by: Cory Benfield <lukasa@apple.com>
* Revert "Back out new typed HTTP protocol upgrader (#2579)"
# Motivation
We have reverted the typed HTTP protocol upgrader pieces since adopters were running into a compiler bug (https://github.com/apple/swift/pull/69459) that caused the compiler to emit strong references to `swift_getExtendedExistentialTypeMetadata`. The problem is that `swift_getExtendedExistentialTypeMetadata` is not available on older runtimes before constrained existentials have been introduced. This caused adopters to run into runtime crashes when loading any library compiled with this NIO code.
# Modifications
This PR reverts the revert and guard all new code in a compiler guard that checks that we are either on non-Darwin platforms or on a new enough Swift compiler that contains the fix.
# Result
We can offer the typed HTTP upgrade code to our adopters again.
* Add compiler guards
# Motivation
We got reports in https://github.com/apple/swift-nio/issues/2574 that our new typed HTTP upgrader are hitting a Swift compiler bug which manifests in a runtime crash on older iOS/macOS/etc.
# Modification
This PR backs out the new typed HTTP protocol upgrader APIs so that we can unblock our users until the Swift compiler bug is fixed.
# Result
No more crashes for our users.
# Motivation
Over the past months, we have been working on new async bridges to make using NIO's `Channel` from Swift Concurrency possible. Since this work was far reaching we have opted to land all of it as SPI. Now the time has come and we feel confident enough to make the SPI official API. This comes after testing the new APIs in various scenarios such as HTTP 1&2, HTTP upgrades, protocol negotiation and in benchmarks.
# Modification
This PR removes the SPI from the `NIOAsyncChannel`, the bootstrap methods, protocol negotiation and HTTP upgrade.
# Result
Everyone can use the our new APIs🚀
* Introduce new typed `HTTPClientUpgrader` and `WebSocketClientUpgrader`
# Motivation
In my previous PR https://github.com/apple/swift-nio/pull/2517, I added a new typed `HTTPServerUpgrader` and corresponding implementation for the `WebSocketServerUpgrader`. The goal of those is to carry type information across HTTP upgrades which allows us to build fully typed pipelines.
# Modification
This PR adds a few things:
1. A new `NIOTypedHttpClientUpgradeHandler` + `NIOTypedHttpClientProtocolUpgrader`. I also moved the state handling to a separate state machine. Similar to the server PR I did not unify the state machine between the newly typed and untyped upgrade handlers since they differ in logic.
2. A new `NIOTypedWebSocketClientUpgrader`
3. An overhauled WebSocket client example.
# Result
This is the last missing piece of dynamic pipeline changing where we did not carry around the type information. After this PR lands, we can finalize the `AsyncChannel` and async typed NIO pieces.
* Remove availability on the protocols
* Introduce new typed `HTTPServerUpgrader` and `WebSocketServerUpgrader`
# Motivation
With our new `NIOAsyncChannel` and typed bootstrap APIs we want to be able to let users spell out their pipeline in a typed way. Pipelines can contain handlers that have to make a forking decision such as HTTP upgrading. Our current `HTTPServerUpgradeHandler` is one of those handlers but it lacks strict typing. To interact nicely with our new typed APIs we need to have a new variant of the `HTTPServerUpgradeHandler` that can carry type information.
# Modification
This PR adds a few things:
1. A new `NIOTypedHTTPServerUpgradeHandler` + `NIOTypedHTTPServeProtocolUpgrader`. I also moved the state handling logic to a separate state machine. I thought about unifying the state machines of the _old_ handler and the new one but they differ in behaviour which makes the state machine more complicated.
2. A new `NIOTypedWebSocketServerUpgrader` that conforms to `NIOTypedHTTPServerProtocolUpgrader`
3. An overhauled WebSocket server example that fully uses Concurrency.
# Result
We now have a way to fully type the server side of HTTP protocol upgrading.
Code review
Update parameter names for new API and fix example
* Introduce new configuration struct and rename to `UpgradablePipeline`
* Review comments
This commit adds Codable to ByteBuffer. It encodes in Base64 into a singleValue
container.
- Base64 implementation moved into _NIODataStructures
- Base64 now supports decoding.
Motivation:
#fileID introduced in Swift 5.3, so no longer need to use #file anywhere
Modifications:
Changed #file to #filePath or #fileID depending on the situation
* Adopt `Sendable` for closures in `HTTPPipelineSetup.swift`
* make `UnsafeTransfer` and `UnsafeMutableTransferBox` available in Swift 5.4 too
* fix code duplication
* Copy `UnsafeTransfer` to `NIOHTTP1Tests` to be able to remove `@testable` from `NIOCore` import
Motivation:
As we've largely completed our move to split out our core abstractions,
we now have an opportunity to clean up our dependencies and imports. We
should arrange for everything to only import NIO if it actually needs
it, and to correctly express dependencies on NIOCore and NIOEmbedded
where they exist.
We aren't yet splitting out tests that only test functionality in
NIOCore, that will follow in a separate patch.
Modifications:
- Fixed up imports
- Made sure our protocols only require NIOCore.
Result:
Better expression of dependencies.
Co-authored-by: George Barnett <gbarnett@apple.com>
Motivation:
Our basic Channel Handlers don't need to be in NIO anymore: they aren't
realistically tied to that code. So we can move them to NIOCore.
Modifications:
- Move Codecs to NIOCore
- Move SingleStepByteToMessageDecoder to NIOCore.
Result:
Even more stuff in NIOCore
* issue-1036: Fix for dates on Linux test generation script.
Motivation:
Fix for issue-1036 to add a date or date range for the header.
These dates are derived from the git log on the file.
Modifications:
Parsing method to extract the date or date range from the git log
of the file. This was added to generate_linux_tests.
Result:
Dynamic date or date range added to the file header.
* issue-1036: Added revised headers from generation script
* Regenerating dates on XCTests.
Motivation:
Addressing feedback from PR.
Modifications:
All of the XCTests now have -2021 as a header date attached to them.
Result:
Dates modified on the XCTests.
Co-authored-by: Cory Benfield <lukasa@apple.com>
* add randomRequestKey method
* fix some typos
* fix compilation on 5.0
* run generate_linux_tests.rb
* add test with default random number generator
* @inlineable
* base64Encoding @inlineable
* add static method to generate a random mask
* use SystemRandomNumberGenerator by default and add documentation
* use WebSocketMaskingKey instead of Self to support Swift 5.0
* add tests for random masking key
* add return keyword to support Swift 5.0
* run scripts/generate_linux_tests.rb
* rename T to Generator
* test SystemRandomNumberGenerator
* Revert "test SystemRandomNumberGenerator"
This reverts commit d9bdbe57ac.
* work around thread sanitizer bug on Swift 5.3
* implement and test WebSocketFrameAggregator
* add documentation
* fix review comments
* run scripts/generate_linux_tests.rb
* fix Swift 5.0
* move redundant channel creation into init
* create channel in setUp
* wrap all trys in XCTAssertNoThrow
* cache accumulatedFrameSize
* accumulate buffered frames into the first frames data buffer
* Revert "accumulate buffered frames into the first frames data buffer"
This reverts commit 0f83823f95.
* use channel allocator
Motivation:
I'm sick of typing `.init(major: 1, minor: 1)`.
Modifications:
- Added static vars for common HTTP versions.
Result:
Maybe I'll never type `.init(major: 1, minor: 1)` ever again.
Motivation:
The fullFilePath() hack doesn't actually work
(https://bugs.swift.org/browse/SR-12934). This is expected behaviour so
currently, the only workaround would be to `#if` whole function bodies
which is a lot of work. Given that in Swift 5.3 `#file` == `#filePath`
still, we can just silence the warning for now.
Modifications:
- Replace `fullFilePath()` by `(#file)`.
Result:
- file locations working properly again.
Co-authored-by: George Barnett <gbarnett@apple.com>
Motivation:
There are multiple sub-optimal ByteBuffer creation patterns that occur
in the wild. Most often they happen when people don't actually have
access to a `Channel` just want to "convert" a `String` into a
`ByteBuffer`. To do this, they are forced to type
var buffer = ByteBufferAllocator().buffer(capacity: string.utf8.count)
buffer.writeString(string)
Sometimes, they don't get the capacity calculation right or just put a
`0`.
Similar problems happen if NIO users want to cache a ByteBuffer in their
`ChannelHandler`. You will then find this code:
```swift
if self.buffer == nil {
self.buffer = receivedBuffer
} else {
var receivedBuffer = receivedBuffer
self.buffer!.writeBuffer(&receivedBuffer)
}
```
And lastly, sometimes people want to append one `ByteBuffer` to another
without mutating the appendee. That's also cumbersome because we only
support a mutable version of `writeBuffer`.
Modifications:
- add `ByteBuffer` convenience initialisers
- add convenience `writeBuffer` methods to `Optional<ByteBuffer>`
- add `writeBufferImmutable` which doesn't mutate the appendee.
Result:
More convenience.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
Very recent versions of the Swift compiler complain if you pass `#file`
to a parameter that is defaulted to `#filePath`. You can silence the
warning by using `(#file)`. We cannot migrate to `#filepath` because we
support Swift 5.0 and 5.1 which don't have `#filepath` yet.
Modifications:
- replace all occurances of `#file` by a special (test `internal`)
function `fullFilePath()` which gives you the full path and is defined
differently depending on the compiler version.
Result:
No warnings even on the latest compiler versions.
Motivation:
The
do {
try someOperation()
XCTFail("should throw") // easy to forget
} catch error as SomethingError {
XCTAssertEqual(.something, error as? SomethingError)
} catch {
XCTFail("wrong error")
}
pattern is not only very long, it's also very error prone. If you forget
any of the XCTFails, you might not tests what it looks like
XCTAssertThrowsError(try someOperation) { error in
XCTAssertEqual(.something, error as? SomethingError)
}
is much safer and shorter.
Modifcations:
Do many of the above replaces.
Result:
Cleaner, shorter, and safer tests.
Motivation:
The class name was a typo.
It is a client upgrader that upgrades WebSocket connections, so the updated name suits better.
Modifications:
Change class name of ‘NIOHTTPWebClientSocketUpgrader’ to ‘NIOHTTPWebSocketClientUpgrader’.
Update the WebSocket client example that uses this class.
Add a typealias to allow the old class name to be used.
Result:
Improved naming.
Motivation:
In some cases it may be possible to write the (usually small) web socket
frame header to the buffer provided by the user. If we do this we can
avoid an extra pipeline traversal for the small write. This is only
going to be a performance win in cases where we can avoid a
copy-on-write operation on the buffer, but if we can then it's a useful
small win to achieve.
Modifications:
- Refactored the WebSocketFrameEncoder to potentially prepend the frame
header.
- Added a missing @inlinable attribute.
- Tests.
Result:
Potentially improved performance.
Motivation:
It's important to also test deprecated functionliaty. One way of
achieving this without warnings is to also deprecate the tests that test
this deprecated functionality. Unfortunately, on Linux we need to
generate lists of tests which would then reference deprecated tests
(which gives us a warning).
Modifications:
Deprecate test suites and the main test runner all the way to the top so
never get warnings.
Result:
Possible to test deprecated functionlity without warnings.
Motivation:
ByteBufferView isn't a mutable collection, but it probably should be.
Modifications:
- Add `copyBytes(at:to:length)` to `ByteBuffer` to copy bytes from a
readable region of a buffer to another part of the buffer
- Conform `ByteBufferView` to `MutableCollection`, `RangeReplaceableCollection`
and `MutableDataProtocol`
- Add an allocation counting test
Result:
`ByteBufferView` is now mutable.
Motivation:
The WebSocketFrameEncoder naively allocated a new block to write the
frame header into every time it wrote. This is excessive: in many cases
it would be able to re-use the same buffer as last time.
Modifications:
- Attempt to re-use the buffer we used for the last header.
Result:
Fewer allocations in some applications.
Motivation:
There is a client protocol upgrader but, unlike the server protocol upgrader, it does not have a WebSocket protocol as part of the project.
Modifications:
Made the magic WebSocket GUID public to the WebSockets project.
Added a NIOWebSocketClientUpgrader.
Added tests for the upgrader.
Updated the Linux test script files.
Result:
The project now has a WebSocket client upgrader to match the WebSocket server upgrader.
Motivation:
To ensure the swift-NIO style for self is followed within the WebSocket tests.
Modifications:
Altered the style within the WebSocketServerEndToEnd tests.
Result:
Slightly more readable code.
Motivation:
To make the web socket server end to end tests class more descriptive.
This will allow ‘WebSocketClientEndToEndTests’ or other end to end tests to be added at a later date.
Modifications:
Renames the class and filename for ‘EndToEndTests’ to ‘WebSocketServerEndToEndTests’
Result:
The nomenclature now allow for other end to end tests to be added.
Motivation:
To make the web socket upgrader naming clearer, particularly once we add a client version.
Modifications:
Rename WebSocketUpgrader too WebSocketServerUpgrader.
Result:
Improved clarity of naming on the web socket upgrader.
Motivation:
Spotted a couple of issues with the documentation and fixed them.
Modifications:
- explicitly marked all `public class`es as `public final class` to get
consistency in the Jazzy documentation.
- removed `public` extension methods of the `internal struct
PriorityQueue` which Jazzy showed in the docs.
- improved the largely missing `EmbeddedChannel` and `EmbeddedEventLoop`
documentation.
- clear up lies in the B2MD docs
- add some other missing docs
Result:
better docs makes happier users
Motivation:
EmbeddedChannel.finish/writeInbound/writeOutbound returned some mystery
bools. I always had to check the code to remember what they actually
meaned.
Whilst this is technically a breaking change, I couldn't find any users
of the return values on Github that are using the convergence releases.
Modifications:
Replace them by enums giving you all the information.
Result:
- fixes#916
- clearer APIs
Motivation:
Follows on from the work done in #528 for #527: we have moved the the
default error handling out of WebSocketFrameDecoder, but had to leave
the code there for backward compatibility reasons. We can remove that
code now.
Modifications:
Removed automatic error handling code in WebSocketFrameDecoder.
Result:
- fixes#534
Rick Newton-Rogers
George Barnett
Raghav RoyandCory Benfield
John Zhou
Adam Fowler
Mirza UčanbarlićandFranz Busch
Johannes Weiss
Natik Gadzhi
Gwynne Raskind
David Nadoba
carolinacass
Matt EatonandCory Benfield
David Nadoba
Liam Flynn
Liam Flynn