### Motivation:
Opening the `swift-nio` repository made me warning blind because there
were always so many trivially fixable warnings about things that were
correct but cannot be understood by the compiler.
### Modifications:
Fix all the sendable warnings that popped up, except for one test where
`NIOLockedValueBox<Thread?>` isn't sendable because `Foundation.Thread`
seemingly isn't `Sendable` which is odd. Guessing that'll be fixed on
their end.
### Result:
- Fewer warnings
- Less warning-blindness
- More checks
Motivation:
`IOData` is a legacy but alas also core type that needs to be
`Sendable`. Before this PR however it can't be `Sendable` because it
holds a `FileRegion` which holds a `NIOFileDescriptor`. So let's make
all of these `Sendable` but let's also start the deprecation journey for
the following types:
- `IOData`, now soft-deprecated (no warnings) because on its reliance on
`FileRegion`
- `FileRegion`, now soft-deprecated (no warnings) because on its
reliance on `NIOFileHandle`
- `NIOFileHandle`, now soft-deprecated (warnings on the
`NIOFileHandle(descriptor:)` constructor but with a
`NIOFileHandle(_deprecatedTakingOwnershipOfDescriptor:)` alternative
- `NonBlockingFileIO`, now soft-deprecated (warnings on the `openFile`
functions (but with `_deprecated` alternatives) because of their
reliance on `NIOFileHandle)
Modification:
- Make `NIOFileDescriptor`, `FileRegion` and `IOData` `Sendable` by
tracking the fd number and the usage state in an atomic
- Enforce singular access by making the `withFileDescriptor { fd ... }`
function atomically exchange the fd number for a "I'm busy" sentinel
value
- Start deprecating `IOData`, `NIOFileHandle`, `NonBlockingFileIO`,
`FileRegion`
Result:
- `NIOFileDescriptor`, `FileRegion` and `IOData` can be `Sendable`
Motivation:
The ChannelInvoker protocols are an awkward beast. They aren't really
something that people can do generic programming against. Instead, they
were designed to do API sharing. Of course, they didn't do that very
well, and the strict concurrency checking world has revealed this.
Much of the API surface on ChannelInvoker is confused. There are
NIOAnys, which aren't Sendable. We allow sending user events without
requiring Sendable. And our two main conforming types are
ChannelPipeline and ChannelHandlerContext, two types with wildly
differing thread-safety semantics.
This PR aims to clean that up.
Modifications:
- Deprecated all API surface on ChannelInvoker protocols that uses
NIOAny.
ChannelInvoker has to be assumed to be a cross-thread protocol,
and that requires that it only use Sendable types. NIOAny isn't,
so these methods are no longer sound.
- Re-add non-deprecated versions on ChannelHandlerContext.
While it's not safe to use the NIOAny methods on Channel or
ChannelPipeline, it's totally safe to use them on
ChannelHandlerContext. So we keep those available and
undeprecated.
- Provide typed generic replacements on ChannelPipeline and on Channel
To replace the NIOAny methods on ChannelPipeline and Channel
we can use some typed generic ones instead. These are not
defined on ChannelInvoker, as the methods are useless on
ChannelHandlerContext. This begins the acknowledgement that
ChannelHandlerContext should not have conformed to these
protocols at all.
- Add Sendable constraints to the user event witnesses on ChannelInvoker
Again, these were missing, but must be there for Channel and
ChannelPipeline.
- Provide non-Sendable overloads on ChannelHandlerContext
ChannelHandlerContext is thread-bound, and so may safely pass
non-Sendable user events.
Result:
One step closer to strict concurrency cleanliness for NIOCore.
Motivation:
Since Swift 5.5 and [SE-0299](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0299-extend-generic-static-member-lookup.md) it is possible to add static members to protocols which are discoverable through the shorthand dot syntax.
This change reduces type repetition and improves call-site legibility.
Modifications:
Added extensions for ChannelOption with static members where Self is bound to a concrete type.
Result:
ChannelOption types can be used with the leading dot syntax. For eg:
```
//before
.channelOption(ChannelOptions.explicitCongestionNotification, value: true)
.channelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)
//after
.channelOption(.explicitCongestionNotification, value: true)
.channelOption(.socketOption(.so_reuseaddr), value: 1)
```
* Apply formatting
* Apply no block comments rule
* Apply OmitExplicitReturns
* Apple OnlyOneTrailingClosureArgument
* Apply NoAssignmentInExpressions
* Fix up DontRepeatTypeInStaticProperties lint errors
* Apply `OrderedImports`
* Apply `ReplaceForEachWithForLoop`
* format file
* Enable the formatting pipeline
* Adopt `AmbiguousTrailingClosureOverload`
* Fix license header
* Fix format check
* Fix `EndOfLineComment`
* Fix CI
* Adapt CI script to check if changes when running formatting
* Separate lint and format into to steps
* Fix format
* Adopt `UseEarlyExits`
* Revert "Adopt `UseEarlyExits`"
This reverts commit d1ac5bbe12.
* Tolerate empty HTTP response body parts
### Motivation
Empty HTTP response body parts currently eagerly end a response. This is unexpected and against NIOs documented behaviour.
### Modification
- add test which send an empty response body part and would previously fail
- skip empty response body parts
### Result
Users can send empty body parts without ending the response.
* add comment explaining why we do it here instead of fixing it at the source
* move empty body part logic to `writeChunk`
Motivation:
The remaining NIO code really conceptually belongs in a module called
NIOPosix, and NIOCore should really be called NIO. We can't really do
that last step, but we can prepare by pushing the bulk of the remaining
code into a module called NIOPosix.
Modifications:
- Move NIO to NIOPosix
- Make NIO an umbrella module.
Result:
NIOPosix exists.
Motivation:
As we've largely completed our move to split out our core abstractions,
we now have an opportunity to clean up our dependencies and imports. We
should arrange for everything to only import NIO if it actually needs
it, and to correctly express dependencies on NIOCore and NIOEmbedded
where they exist.
We aren't yet splitting out tests that only test functionality in
NIOCore, that will follow in a separate patch.
Modifications:
- Fixed up imports
- Made sure our protocols only require NIOCore.
Result:
Better expression of dependencies.
Co-authored-by: George Barnett <gbarnett@apple.com>
Motivation:
I'm sick of typing `.init(major: 1, minor: 1)`.
Modifications:
- Added static vars for common HTTP versions.
Result:
Maybe I'll never type `.init(major: 1, minor: 1)` ever again.
Motivation:
The names of most of the BSD socket option values were brought over into
their NIO helper properties, with their namespace removed. This vastly
increases the risk of collision, particularly for things like TCP_INFO,
which just became .info.
Modifications:
- Added the prefixes back, e.g. `.info` is now `.tcp_info`.
- Renamed socket type `.dgram` to `.datagram`, as this is the nice clean
API and we should use nice clean names.
Result:
Better APIs
* NIO: Add new `BSDSocket` namespace
This starts the split of the POSIX/Linux/Darwin/BSD interfaces and the
BSD Socket interfaces in order to support Windows. The constants on
Windows are not part of the C standard library and need to be explicitly
prefixed. Use the import by name to avoid the `#if` conditions on the
wrapped versions. This will allow us to cleanly cleave the dependency
on the underlying C library.
This change adds a `BSDSocket.OptionLevel` and `BSDSocket.Option` types
which allow us to get proper enumerations for these values and pipe them
throughout the NIO codebase.
* Apply suggestions from code review
Co-Authored-By: Cory Benfield <lukasa@apple.com>
Motivation:
writeWithUnsafeMutableBytes cannot tolerate arbitrarily large writes because it is not possible to resize the buffer by the time body receives the buffer pointer. This change provides an option for the user to reserve writable capacity before the pointer is passed to body.
Modifications:
Added a backward compatible optional parameter to writeWithUnsafeMutableBytes which can reserve writable capacity before the pointer is passed to body.
Result:
Additive change only. Users can optionally specify a minimum writable capacity when calling writeWithUnsafeMutableBytes.
Motivation:
With HTTP/1.0 and EOF framing it's possible to have a HTTP response that
doesn't have any headers. HTTPDecoder unfortunately fell flat on its
face and crashed the program if that happened.
Modifications:
- don't expect headers on responses
- add tests for that
- rename the state machine cases to reflect a request and response
parsing
Result:
fewer crashes.
Motivation:
`ctx` was always an abbreviation was 'context` and in Swift we don't
really use abbreviations, so let's fix it.
Modifications:
- rename all instances of `ctx` to `context`
Result:
- fixes#483
Motivation:
- `ChannelPipeline.add(name:handler:...)` had a strange order of arguments
- `remove(handler:)` and `remove(ctx:)` both remove `ChannelHandler`s
but they read like they remove different things
So let's just fix the argument order and name them `addHandler` and
`removeHandler` making clear what they do.
Modifications:
- rename all `ChannelPipeline.add(name:handler:...)`s to `ChannelPipeline.addHandler(_:name:...)`
- rename all `ChannelPipeline.remove(...)`s to `ChannelPipeline.removeHandler(...)`
Result:
more readable and consistent code
Motivation:
Collisions with Foundation and other standard libraries is painful.
Modifications:
- rename `FileHandle` to `NIOFileHandle`
- rename `Thread` to `NIOThread` (only exposed in docs)
Result:
fewer collisions
Motivation:
ByteBuffer methods like `set(string:)` never felt very Swift-like and
also didn't look the same as their counterparts like `getString(...)`.
Modifications:
- rename all `ByteBuffer.set/write(<type>:,...)` methods to
`ByteBuffer.set/write<Type>(...)`
- polyfill the old spellings in `_NIO1APIShims`
Result:
code more Swift-like
Motivation:
The name `mapIfError` implied that you can transform the error value but
really that's never what it did. Instead, it allowed you to recover
from a failure.
Modifications:
- rename `mapIfError` to `recover`
Result:
cleaner method names
Motivation:
ELF's API should be as close as possible to the new Result's API.
Therefore, we should rename `then` to `flatMap`
Modifications:
- renamed `then` to `flatMap`
- renamed `thenIfError` to `flatMapError`
- renamed ELF's generic parameter from `T` to `Value`
Result:
- more like Result
- fixes#688
Motivation:
Now that the stdlib has introduced the Result type, we can use it in the
implementation (and the whenComplete) function of EventLoopFuture
Modifications:
- replace EventLoopValue with Result
- make whenComplete provide the Result
Result:
use the new shiny stuff
Motivation:
NIO2 development starts now.
Modifications:
Made NIO Swift 5-only for everything else see docs/public-api-changes-NIO1-to-NIO2.md
Result:
NIO2 development can start.
### Motivation:
make swift-nio compatible with Android, and pass all test in Android.
### Modifications:
* add missed `ifaddrs` implementation
* add missed thread related functions - `CNIOLinux_pthread_getname_np`, `CNIOLinux_pthread_setaffinity_np` and `CNIOLinux_pthread_getaffinity_np`
* fix types incositency between Linux and Android, e.g. `Epoll.swift` class
* make bytes counter explicitely 64bits to avoid overflow on 32bit, in `PendingWritesManager.swift` and `PendingDatagramWritesState.swift`
* fix issue with Unix domain socket, first byte need to be zero in Android
* several incosistency fixes between Linux and Android api.
### Result:
now swift-nio works on Android. All tests passed!
Motivation:
Naked `try`s in tests are a problem because if they're hit you'll get a
bad error message and likely no file/line information. That has made
many flaky tests harder to debug than necessary.
Modifications:
- fix a lot of naked `try`s
- symlink `TestUtils` from `NIOTests` into `NIOHTTP1Tests`
- make a bunch of `wait()`s synchronous that were asynchronous by
accident
Result:
- better code
- easier to debug test failures
- setting better example
Motivation:
We should be consistent with naming and also choose descriptive names.
Modifications:
- Deprecate old init method that uses numThreads
- Add new init with numberOfThreads param name
- Everywhere use the new init
Result:
More consistent and descriptive naming. Fixes https://github.com/apple/swift-nio/issues/432.
Motivation:
The current HTTP server helpers are not modular, meaning that each
time we add a new first-party HTTP handler we need to add a brand new
method. Additionally, they do not currently compose, so if you want
assistance with HTTP pipelining *and* to support HTTP upgrade you are
out of luck.
Modifications:
Deprecate the two existing server helpers.
Add a new server helper that can be extended to support all of the
possible features that users may want in their HTTP pipelines.
Result:
Users will have a single place to go that can be used to configure
their HTTP server pipeline, and that understands how the different
handlers fit together in a complete pipeline.
Motivation:
guard !self.open else { ... } is a double negation and can be confusing,
guard self.isOpen else { ... } is much better
Modifications:
replaced all `var closed: Bool` with `var isOpen: Bool`
Result:
we're more positive
Motivation:
We have multiple reasons why flush promises weren't great, for example:
- writeAndFlush sounds like it's a write + flush optimisation but in
reality it was more expensive (2 extra promises)
- the semantics were never quite clear
- lots of implementation complexity, especially for the datagram channel
Modifications:
- removed the flush promises in the API
- this deliberately doesn't do the PendingWritesManager simplifications
that this unlocks
Result:
flush doesn't have a promise anymore.
Motivation:
There were a couple of places in the Channel implementations that just
weren't thread-safe at all, namely:
- localAddress
- remoetAddress
- parent
those are now fixed. I also re-ordered the code so it should be easier
to maintain in the future (the `// MARK` markers were totally
incorrect).
Modifications:
- made Channel.{local,remote}Address return a future
- made Channel.parent a `let`
- unified more of `SocketChannel` and `DatagramSocketChannel`
- fixed the `// MARK`s by reordering code
- annotated methods/properties that are in the Channel API and need to
be thread-safe
Result:
slightly more thread-safety :)
Motivation:
Foundation is problematic for a few reasons:
- its implementation is different on Linux and on macOS which means our
macOS tests might be inaccurate
- on macOS it uses ObjC Foundation which means the autorelease pool
might get populated
- it links the world on Linux which means we can't do static
binaries at all
Modifications:
removed the last bits of Foundation dependency
Result:
no Foundation dependency
to not return a value
Motivation:
We recently had a bug where we had `EventLoopFuture<EventLoopFuture<()>>` which didn't make any sense. The compiler couldn't catch that problem because we just ignored a closure's argument like this:
future.then { _ in
...
}
which is dangerous. For closures that take an empty tuple, the `_ in`
isn't actually required and the others should state the type they want
to ignore.
And most whenComplete calls can be better (and often shorter) expressed
by other combinators.
Modifications:
remove pretty much all closures which just blanket ignore their
parameter.
Result:
- no closures which just ignore their parameter without at least stating
its type.
- rewrote all whenCompletes that actually used the value
Motivation:
Previously `FileRegion` was a special snow flake, a bit like
`ByteBuffer` but also totally different. That caused surprise and made
`PendingWritesManager` even harder. It was also used to manage the
lifetime of a file descriptor but only sort of.
Modifications:
We now have a `FileHandle` which is a one-to-one mapping with a file
descriptor and its lifetime must be managed appropriately.
Result:
hopefully less bugs and fd leaks.
Motivation:
Lots of our most important operations had redundant labels like
func write(data: NIOAny)
the `data: ` label doesn't add anything meaningful and therefore it
should be removed.
Modifications:
removed lots of redundant labels
Result:
less redundant labels
Max Desiatov
George BarnettandRick Newton-Rogers
Johannes Weiss
Cory Benfield
Ayush Garg
Franz Busch
David Nadoba
Saleem AbdulrasoolandCory Benfield
Jake
Albert Aleksieiev
Johannes Weiss
Norman Maurer
Cory Benfield
adamnemecek
Cory Benfield