Fix coreCount on Linux when using cgroup v2 with CFS throttling disabled
### Motivation:
When using `swift-nio` on Linux with cgroup v2 enabled, but with CFS
throttling disabled, it falls back to attempting to read the cpuset file
at the cgroup v1 path. This does not exist, which in turns falls back to
returning `_SC_NPROCESSORS_ONLN`, which will return the total number of
cores available (ignoring cgroup assignments).
This has unexpected effects, including the default behaviour of starting
the `MultiThreadedEventLoopGroup.singleton` with significantly more
event loops than cores available to the workload.
### Modifications:
- Adds `SystemCalls.statfs`, and associated constants, to determine the
cgroup version.
- Adds `Linux.cgroupVersion()` API to expose cgroup version.
- Adds `Linux.cgroupV2MountPoint` variable to determine the cgroup v2
mount point.
- Adds `Linux.cpuSetPathV1` & `Linux.cpuSetPathV2` (and
`Linux.cpuSetPath` convenience) variables to determine the correct cpu
set path.
- Alters `System.coreCount` to use the appropriate logic from above to
ensure that `cpuset.cpus` is parsed from the correct location.
### Result:
`Linux.coreCount` should correctly parse and return the core count on
Linux cgroup v2 enabled systems (when CFS throttling is disabled), while
maintaining correctness for other configurations.
---------
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
### Motivation:
In many situations, for example continuous profiling with sampling
profilers, it's important to distinguish between on- and off-CPU work.
That's most easily done if there are clear function names/prefixes to
grep for that are common places to just wait (off-CPU) and do no real
work. In SwiftNIO those are chiefly three places:
1. The `EventLoops` waiting for work
2. `The NIOThreadPool` threads waiting for work
3. `EventLoopFuture.wait()`
This patch makes sure that each of those will have a function with
prefix `_blockingWaitFor` in the function name which is easily
greppable.
### Modifications:
- Create some non-inlinable functions with `_blockingWaitFor...` that
just wait for `...`.
### Result:
SwiftNIO makes operational excellence easier and your SRE team more
happy.
Dispatch is not supported on WASI, and only Unix domain sockets are
supported, which means we have to exclude those APIs on this platform.
There's work in progress to enable tests for this on CI, but nothing I
can provide for this PR at the current moment.
---------
Co-authored-by: Franz Busch <f.busch@apple.com>
Motivation:
Get this repo building again for Android with the new overlay
Modifications:
- Import the new module or overlay wherever `Glibc` is used
- Keep this repo building with Swift 5 by duplicating some declarations
Result:
All the same tests keep passing on my Android CI, finagolfin/swift-android-sdk#158
* Apply formatting
* Apply no block comments rule
* Apply OmitExplicitReturns
* Apple OnlyOneTrailingClosureArgument
* Apply NoAssignmentInExpressions
* Fix up DontRepeatTypeInStaticProperties lint errors
* Apply `OrderedImports`
* Apply `ReplaceForEachWithForLoop`
* format file
* Enable the formatting pipeline
* Adopt `AmbiguousTrailingClosureOverload`
* Fix license header
* Fix format check
* Fix `EndOfLineComment`
* Fix CI
* Adapt CI script to check if changes when running formatting
* Separate lint and format into to steps
* Fix format
* Adopt `UseEarlyExits`
* Revert "Adopt `UseEarlyExits`"
This reverts commit d1ac5bbe12.
* [GHA] Unacceptable language check
# Motivation
Next up replacing part of our soundness script that checked for unacceptable language.
# Modification
This PR adds a new GH action that checks for unacceptable language.
# Result
One more script replaced
* PR review
* Change homepage link to repo
Motivation:
Fix build for the latest LTS NDK 26
Modifications:
- Update C declarations
- Add force unwraps where needed
Result:
Everything works on Android with NDK 26b
Motivation
Fix build errors on Android
Modifications
- Fix previous Musl modifications that assumed Glibc wasn't imported on Android
- Add errors for all libc imports, so new platform ports error out early
Result
NIO builds natively on Android again, with all the same tests passing
* Add support for Musl libc
Since Musl is sufficiently different from Glibc (see https://wiki.musl-libc.org/functional-differences-from-glibc.html), it requires a different import, which now should be applied to files that have `import Glibc` in them.
* Fix `msghdr` initialization
* Fix msghdr mutability
* Fix `UnsafeMutableRawPointer` type conversions
Windows does not provide a 3 parameter `open` and needs to be passed all
the parameters as `_open` is a variadic function (as per the
specification). This provides a Windows specific path for `open` and
`read` as on Darwin and Linux, `read` uses a non-standard return type
(`ssize_t`). As it happens, the `size` parameter on Windows also uses
`unsigned int` rather than `size_t` which would break on different
bitnesses.
Co-authored-by: Cory Benfield <lukasa@apple.com>
* NIOCore: replace `mode_t` with `CInt`
Windows does not have a `mode_t` type alias, instead using the
de-sugared `CInt` type. De-sugar the instances to permit building on
Windows.
* Update Sources/NIOCore/FileHandle.swift
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
`errno` on Windows cannot be access as a raw name as it goes through a
"complex" macro for the TLS access. Use the `_get_errno` helper instead
of accessing the `errno` through the macro for building on Windows.
Motivation:
MultcastChannel is a general abstraction for expressing multicast
capabilities on a given Channel. This abstraction doesn't have any
particularly tight tie to the POSIX layer, so it belongs in NIOCore.
This is also expressed in terms of NIONetworkDevice, so we need to move
that over. That also encourages us to bring over
System.enumerateDevices, and given that System.coreCount is also fairly
general-purpose we may as well bring it along too.
Modifications:
- Move MulticastChannel to NIOCore
- Move NIONetworkDevice to NIOCore
- Move System to NIOCore
Result:
More general-purpose abstractions in NIOCore.
Motivation:
The most important API surface area in NIO are the Channel abstractions.
These are shared in all NIO programs, and are also used by several
projects to implement their I/O abstraction. There are several moving
parts to this abstraction, all of which are moving:
- Channel itself
- ChannelPipeline
- ChannelHandler
As these all move, they force several other pieces of API to move with
them. Most notably they force us to move NIOAny, which also forces us to
move FileHandle and FileRegion. That also forces us to bring over part
of our syscall abstraction. This duplication is acceptable due to its
minimal surface area, but it is definitely a flaw in our abstraction
design that we had to do that at all.
We also need to move the channel option abstraction, AddressedEnvelope,
and the DeadChannel.
Modifications:
- Moved a bunch of the Channel abstraction over.
- Moved Channel-associated types.
Result:
Channel will be part of NIOCore.
Fabian Fett
Max DesiatovandFranz Busch
finagolfin
finagolfin
Saleem AbdulrasoolandCory Benfield