## Motivation
NIO channels abstract over an underlying transport mechanisms (sockets,
pipes, etc.), which users typically need not interact with. However,
there are scenarios where users need direct access to the underlying
transport for low-level operations that work outside NIOs abstraction.
One example is performing out out-of-band operations on the underlying
file descriptor for a socket-based channel.
This PR adds a structured way to access the underlying transport of a
channel, for channels that choose to implement it.
## Modifications
- Add a new `public protocol
NIOTransportAccessibleChannelCore<Transport>` which provides a scoped
`withUnsafeTransport(_:)`, used for channel implementations to opt-in.
- Add conformance to
`NIOTransportAccessibleChannel<NIOBSDSocket.Handle>` for
`BaseSocketChannel`, to make this API available for all socket-based
channels, including channels returned from the socket-based bootstrap
public APIs.
- Add public API
`ChannelPipeline.SynchronousOperations.withUnsafeTransportIfAvailable(_:)`
Note that not all channels need to or should their transport, which is
why this was added as an additional protocol that refines `ChannelCore`,
vs. extending `Channel` or `ChannelCore` with a default implementation.
The protocol uses a primary associated type allowing channels to provide
typed access to the transport. E.g. this could be used in NIO Transport
Services to expose the underlying `NWConnection`, if desired.
The method itself is spelled with "unsafe", uses scoped access, and has
clear documentation that users must not violated any of NIOs assumptions
about the state of the underlying transport. It's very much not intended
for every day use. It being on `ChannelCore` should defer users of the
low-level API, since `Channel._channelCore` is marked as for NIO
internal use, and the public `withUnsafeTransportIfAvailable(_:)` will
take care of the runtime checks for channels that have opted into this
API.
## Result
- New opt-in API for channel implementations to expose their underlying
transport
- All socket-based channels from NIOPosix now expose the underlying
socket file descriptor
- New API
`ChannelPipeline.SynchronousOperations.withUnsafeTransportIfAvailable(_:)`
for users
---------
Co-authored-by: Agam Dua <agam_dua@apple.com>
### Motivation:
Additional partial platform support.
### Modifications:
* Create a new CNIOBSD module for OpenBSD, for general cleanliness, and
make use of it throughout. Some of the changes are borrowed directly
from CNIOLinux; some may technically be unnecessary; I am erring
somewhat on expediency to functionality.
* Since `malloc_size` is unavailable on OpenBSD, and some of the helpers
make use of ManagedBufferPointer which makes use of it, mark some of
this as unavailable as well.
* Usual pthread optional typing changes, since pthread types are
pointers on this platform.
* Add conditionals to exclude other functionality not available here,
like IP_RECVPKTINFO or IP_PKTINFO.
* d_ino is a backwards-compatibility macro valued token for dirent, so
instead, just expand with a conditional.
* Use kqueue on OpenBSD. This necessitates adding some conditionals for
type and feature compatibility.
* The vsock API is unavailable on OpenBSD.
### Result:
Tested the NIOTCPEchoClient and NIOTCPEchoServer appears to work and
that swift-nio-ssh (hopefully wlog) builds with a local repository with
these changes.
Because NIOFS/_NIOFileSystem depend on some non-portable components,
such as extended attributes, sendfile, non-portable linkat flags, and
renameat2, this is only just partial OpenBSD support, which means that
`swift build` on the full swift-nio project won't build cleanly, but at
least the portable parts can be used to build servers and clients. This
commit will mark the linked bug as fixed; I will open a new bug for file
system support.
Fixes#3383.
---------
Co-authored-by: Rick Newton-Rogers <rnro@apple.com>
Motivation:
Public static lets can serve a bunch of roles, but one of them is to
store simple constants: integers, and other trivial types, for example.
This is a nice pattern and for internal and private static lets it works
well, but for public ones it produces some inefficient code.
In particular, it has two downsides. First, it allocates storage for
that value. We don't actually need to allocate a few hundred extra
megabytes for the various integers we want to store.
Secondly, it forces calling code to access the address and call the
dispatch_once code in order to get hold of the value. For trivial types
we don't need that cost: they can just know what the value is directly.
Inlinable computed vars avoid all of these costs: they have no size
overhead for storage, and they are visible to all clients so their
values can be directly assembled.
While I'm here, I added a bunch of other inlinable annotations for a few
trivial data types I stumbled onto.
Modifications:
- Added loads of inlinables. Like, loads.
- Swapped many static lets to static vars.
Result:
Better codegen, smaller memory footprints, more attributes.
Motivation:
With our earlier big refactors, NIOCore is now currently strict
concurrency clean. Let's lock in the win by adopting the relevant Swift
settings and fixing up the tests.
Modifications:
- Adopt our strict concurrency settings in NIOCore and NIOCoreTests
- Clean up the tests
Result:
One step closer to a world without warnings.
Add ability to get the amount of buffered outbound data from `Channel`
### Motivation:
Right now, SwiftNIO does not have the API to answer the question "how
much data is buffered in the Channel". Applications focusing on
performance may need to fine-tune the amount of outbound data that will
be sent to optimize data throughput, adjust sending rate to avoid
overflow, and potentially reduce latency.
SwiftNIO currently provides some backpressure mechanism. This new API
will be a good addition. By knowing how much data is buffered directly,
applications can make informed decision to adjust for optimal buffer
sizes and send rates.
### Modifications:
- Expose current buffer size through ChannelOptions so that users can
read the value out. StreamSocketChannel, DatagramSocketChannel,
EmbeddedChannel, and AsyncTestingChannel have the same API interface.
- Various modifications to the existing tests to make sure the new API
is working correctly.
- Add a new `so_sndbuf` socket option so that users can easily adjust
the send buffer size.
### Result:
Users can get the amount of outbound bytes currently buffered in the
`Channel` through the new `BufferedWritableBytesOption` channel option.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Dispatch is not supported on WASI, and only Unix domain sockets are
supported, which means we have to exclude those APIs on this platform.
There's work in progress to enable tests for this on CI, but nothing I
can provide for this PR at the current moment.
---------
Co-authored-by: Franz Busch <f.busch@apple.com>
Motivation:
Get this repo building again for Android with the new overlay
Modifications:
- Import the new module or overlay wherever `Glibc` is used
- Keep this repo building with Swift 5 by duplicating some declarations
Result:
All the same tests keep passing on my Android CI, finagolfin/swift-android-sdk#158
* Apply formatting
* Apply no block comments rule
* Apply OmitExplicitReturns
* Apple OnlyOneTrailingClosureArgument
* Apply NoAssignmentInExpressions
* Fix up DontRepeatTypeInStaticProperties lint errors
* Apply `OrderedImports`
* Apply `ReplaceForEachWithForLoop`
* format file
* Enable the formatting pipeline
* Adopt `AmbiguousTrailingClosureOverload`
* Fix license header
* Fix format check
* Fix `EndOfLineComment`
* Fix CI
* Adapt CI script to check if changes when running formatting
* Separate lint and format into to steps
* Fix format
* Adopt `UseEarlyExits`
* Revert "Adopt `UseEarlyExits`"
This reverts commit d1ac5bbe12.
This change allows users to setup a broadcast socket using a NIO owned
constant instead of directly using a C constant.
```diff
- .channelOption(ChannelOptions.socket(SOL_SOCKET, SO_BROADCAST), value: 1)
+ .channelOption(ChannelOptions.socketOption(.so_broadcast), value: 1)
```
Fixes#2674
Motivation:
Fix build for the latest LTS NDK 26
Modifications:
- Update C declarations
- Add force unwraps where needed
Result:
Everything works on Android with NDK 26b
Motivation
Fix build errors on Android
Modifications
- Fix previous Musl modifications that assumed Glibc wasn't imported on Android
- Add errors for all libc imports, so new platform ports error out early
Result
NIO builds natively on Android again, with all the same tests passing
* Add support for Musl libc
Since Musl is sufficiently different from Glibc (see https://wiki.musl-libc.org/functional-differences-from-glibc.html), it requires a different import, which now should be applied to files that have `import Glibc` in them.
* Fix `msghdr` initialization
* Fix msghdr mutability
* Fix `UnsafeMutableRawPointer` type conversions
Motivation:
Support was added for UDP_SEGMENT in #2372 which allows for large UDP
datagrams to be written to a socket by letting the kernel or NIC segment
the data across multiple datagrams. This reduces traversals across the
network stack which can lead to performance improvements. UDP_GRO is the
receive-side counterpart allowing the kernel/NIC to aggregate datagrams
and reduce network stack traversals.
Modifications:
- Add a function in CNIOLinux to check whether UDP_GRO is supported
- Add the relevant socket and channel options
- Add tests
Result:
- UDP_GRO can be enabled where supported and applications may receive
large buffers.
Motivation:
On Linux, the UDP_SEGMENT socket option allows for large buffers to be
written to the kernel and segmented by the kernel (or in some cases the
NIC) into smaller datagrams. This can substantially decrease the number
of syscalls.
This can be set on a per message basis on a per socket basis. This
change adds per socket configuration.
Modifications:
- Add a CNIOLinux function to check whether UDP_SEGMENT is supported on
that particular Linux.
- Add a helper to `System` to check whether UDP_SEGMENT is supported on
the current platform.
- On Linux only:
- add the udp socket option level
- add the udp_segment socket option
- Add the `DatagramSegmentSize` channel option.
- Get/Set the option in `DatagramChannel`
Results:
UDP GSO is supported on Linux.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation
MPTCP provides multipath capability for TCP connections. This
allows TCP connections to consume multiple independent network
paths, providing devices with a number of capabilities to
improve throughput, latency, or reliability.
MPTCP is not totally transparent, and requires servers to support
the functionality as well as clients. To that end, we should expose
some MPTCP capability.
Importantly, MPTCP uses a number of new socket flags and options.
To enable us to support this when it is available but gracefully fail
when it is not, we've hardcoded a number of Linux kernel constants
instead of relying on libc to expose them. This is safe to do on Linux
because its syscall layer is ABI stable.
Modifications
- Add ClientBootstrap and ServerBootstrap flags for MPTCP
- Plumb MPTCP through the stack
- Add new socket options for MPTCP
Result
MPTCP is supported on Linux
Windows does not provide a definition for `socklen_t`. Create an
internal typealias for it to provide source stability to the internal
interfaces.
Co-authored-by: George Barnett <gbarnett@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
`NIOBSDSocket.inet_pton` and `NIOBSDSocket.inet_ntop` altered the label
for the family parameter but failed to update the Windows path. Adjust
the name accordingly.
* Adopt `Sendable` for types in `BSDSocket` namespace
* Adopt `Sendable` for `CircularBuffer` and `MarkedCircularBuffer`
and also `Endianness`
* move `NIOSendable` into its own file and copy it over to syscall integration tests
Motivation:
An upcoming pull, apple/swift#35707, moves Android to the same single-header
modulemap for Bionic as used on linux, but that doesn't work unless this is
changed. This approach works both with the current release toolchain and
with that new modulemap.
Modifications:
Change how these Bionic constants are imported.
Result:
This repo compiles for Android and the tests pass with both modulemap approaches.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
Unbreak Android build from NIOCore not having these constants defined.
Modifications:
Copy the same constants that Channel was using in NIO over.
Result:
Android compiles again and the same tests pass.
Motivation:
When pulling out the NIOBSDSocket later in #1907 we dropped some of the
socket option fixes for 32 bit platforms, which broke them.
Modifications:
Restore the socket options.
Result:
32-bit build should be working again.
Motivation:
This is the next stage of our move of common interchange objects to
NIOCore and out of the NIO module. This time around we need
SocketAddress, which is part of the Channel API. Sadly, SocketAddress is
not as clean as some of the other ports, because it leaks a number of
POSIX-y concepts. This also forces us to bring along NIOBSDSocket, which
ideally we would not move, but we foolishly exposed as API on
SocketAddress.
Modifications:
- Move NIOBSDSocket API components into NIOCore.
- Split out the internal abstractions for NIOBSDSocket and leave those
in NIO.
- Move SocketAddress into NIOCore.
Result:
SocketAddress will be sitting in NIOCore.
Si BeaumontandAgam Dua
3405691582andRick Newton-Rogers
Fabian FettandCory Benfield
Johannes Weiss
Jeff
John ZhouandCory Benfield
Max DesiatovandFranz Busch
finagolfin
Rauhul Varma
finagolfin
George Barnett
Saleem Abdulrasool
David Nadoba