Motivation
* `check_benchmark_thresholds.sh` was updated to check for exit code 2
from `swift package benchmark thresholds check` to distinguish
threshold regression from build errors.
* It seems SwiftPM's `CommandPlugin` infrastructure always exits with code 1 when
`performCommand` throws, regardless of the error's raw value. So the
exit code is never 2, causing all regressions to fall through to the
`else` branch and be misreported as build errors, with no diff output.
Modifications
* Remove the `rc == 2` check and instead attempt `thresholds update`
for any non-zero `rc` from `thresholds check`.
* Use the result of `thresholds update` to distinguish regression
(success) from build error (failure).
* Remove `--exit-code` from `git diff` to prevent `set -uo pipefail`
from aborting the script before output is fully flushed.
Result
* Benchmark threshold regressions correctly output the `=== BEGIN DIFF
===` section again.
* Actual build errors are still correctly detected and reported.
### Motivation:
Currently if there is an error in the Benchmark targets, that prevents
the Benchmark to build correctly, we still get a CI success.
### Modifications:
- Check for the threshold changed error
### Result:
- More reliable CI
### Motivation:
SwiftPM behaviour has changed in recent Swift nightlies.
In most recent released version of Swift, SwiftPM will use the parent
directory name for the source and test target directory too, even when
it contains a period:
```
% docker run -it swift:6.2 bash -c 'mkdir -p /tmp/foo.bar && cd /tmp/foo.bar && swift package init'
Creating library package: foo.bar
Creating Package.swift
Creating .gitignore
Creating Sources
Creating Sources/foo.bar/foo_bar.swift
Creating Tests/
Creating Tests/foo.barTests/
Creating Tests/foo.barTests/foo_barTests.swift
```
Note in the above how it replaces the `.` with a `_` in the source
files, but the directories retain the `.`.
In the recent nightly-main, SwiftPM has also started replacing `.` with
`_` in the target directory names:
```
% docker run -it swiftlang/swift:nightly-main bash -c 'cat /etc/motd && mkdir -p /tmp/foo.bar && cd /tmp/foo.bar && swift package init'
################################################################
# #
# Swift Nightly Docker Image #
# Tag: swift-DEVELOPMENT-SNAPSHOT-2026-02-06-a #
# #
################################################################
Creating library package: foo.bar
Creating Package.swift
Creating .gitignore
Creating Sources
Creating Sources/foo_bar/foo_bar.swift
Creating Tests/
Creating Tests/foo_barTests/
Creating Tests/foo_barTests/foo_barTests.swift
```
Probably this
https://github.com/swiftlang/swift-package-manager/pull/9252.
This breaks our CI automation, only on nightly-main, because we run
`swift package init` in a directory created by `mktemp -d`, which, by
default, follows the pattern `tmp.XXXXXXX`.
### Modifications:
Create temporary directory for Swift package using explicit pattern
containing no periods.
### Result:
C++ interop CI should now work again on nightly-main Swift, and continue
to work on all supported Swift versions.
This PR moves the nightly-next CI to use the 6.3 images. Since those
checks should be non required on any user of this package and the name
of the check doesn't change with moving images this should be a
non-breaking change.
Generate matrix script finds manifests in subdirs
### Motivation:
The genereate matrix script attempts to derive the minimum Swift version
supported by the package so that it doesn't run CI on versions which are
older than that supported. At the moment it can't cope with repositories
with packages in subdirectories like swift-nio-examples.
### Modifications:
The script looks for manifests in subdirectories if
`FIND_SUBDIRECTORY_MANIFESTS_ENABLED=true`. This is opt-in because by
default CI only uses the top-level manifests.
### Result:
CI should generate more appropriate jobs.
An example of this in action
https://github.com/apple/swift-nio/actions/runs/18968739126/job/54171197381?pr=3431
android build use swift static stdlib
### Motivation:
Android build fails and the guide says to use this flag
https://www.swift.org/documentation/articles/swift-sdk-for-android-getting-started.html
Recent failure - although weirdly it doesn't seem to happen every time:
```
error: link command failed with exit code 1 (use -v to see invocation)
ld.lld: error: cannot open crtbegin_dynamic.o: No such file or directory
ld.lld: error: cannot open crtend_android.o: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[558/601] Linking NIOUDPEchoClient
```
### Modifications:
android build use swift static stdlib
### Result:
hopefully success - making this change locally seemed to work.
This change can be seen in action here
https://github.com/apple/swift-nio/actions/runs/18949279888/job/54110590743?pr=3430
Swift SDK installation script improvements
### Motivation:
The SDK installation scripts had a few rough edges and didn't verify the
artefacts they downloaded.
### Modifications:
* Change naming to the more accurate "Swift SDK" where we can without
breaking downstream adopters.
* Separate Android NDK installation into its own script.
* Avoid using `cd` when setting up Android NDK.
* Verify checksums for SDKs and signatures for toolchains.
* Specify/control the SDK directory.
### Result:
* More accurate naming.
* More maintainable scripts.
* More secure downloads.
An example of this working
https://github.com/apple/swift-nio/actions/runs/18944235613/job/54090944716?pr=3429
### Motivation:
Android SDKs now exist and testing is good.
### Modifications:
Add an Android GitHub Actions workflow using the latest nightly image
and enable it in NIO targetting NIOCore (matching WASM testing).
### Result:
More reusable workflows and more test coverage for NIO
See this in action here (the CI job works but NIOCore fails to build):
https://github.com/apple/swift-nio/actions/runs/18718487605/job/53384086329?pr=3418
WASM SDK GitHub Actions workflows use NIO matrix rather than the
swiftlang one
### Motivation:
Using the NIO one is more in line with the surrounding workflows and
surfaces a simpler UI (possible jobs which aren't run don't appear in
the UI)
### Modifications:
Pull in changes from https://github.com/apple/swift-nio/pull/3159 and
add in the ability to specify environment variables and other build
arguments in line with the static Linux SDK workflow.
### Result:
More regularity, simpler checks UI.
See this in action here:
https://github.com/apple/swift-nio/actions/runs/18717131164/job/53379323803?pr=3417
---------
Co-authored-by: Yuta Saito <kateinoigakukun@gmail.com>
Co-authored-by: Max Desiatov <m_desiatov@apple.com>
Don't generate matrix jobs for Swift versions which aren't supported by
the package.
### Motivation:
Matrix workflows should not generate jobs for Swift versions < tools
version, doing so wastes compute and adds noise.
This means that:
- New repos don’t need to manually opt-out of old Swift versions
- Repos which have dropped an old Swift version (during a migration
window) won’t have failing tests until it’s turned off centrally.
### Modifications:
- The generate script now looks for `Package.swift` and
`Package@swift-*.swift` files and uses the lowest tools version in any
found file to determine the minimum supported Swift version
- Any Swift version lower than the minimum is disregarded and no jobs
are generated for it
- A new parameter is added to the workflows which allows the minimum
Swift version to be set manually (`minimum_swift_version`)
### Result:
By default in this and downstream packages we don't generate matrix jobs
for Swift versions which aren't supported by the package.
An example of this working:
https://github.com/apple/swift-nio/actions/runs/18711362977/job/53360387720?pr=3415
Allow the specification of custom environment variables in our matrix
workflow and those which use it.
### Motivation:
It can be useful to supply custom environment variables to control build
and runtime options.
### Modifications:
Allow users to supply a string of custom environment variable key-value
pairs which are supplied to the docker container running the matrix
commands.
To attempt to avoid issues with this string concatenation the change
also allows the user to specify custom list entry and key-value-pair
separators.
To avoid parameter bloat the env vars are set for all Swift versions,
hopefully that's sufficient for now.
### Result:
Adopters can
Examples of this working on Linux and Windows, note the presence of the
test environment variables I injected:
-
https://github.com/apple/swift-nio/actions/runs/17917619451/job/50943847555?pr=3376
- https://github.com/apple/swift-crypto/pull/413/checks
e.g.
```
Executing Docker command: docker run -v /home/runner/work/swift-nio/swift-nio:/swift-nio -w /swift-nio -e CI=true -e GITHUB_ACTIONS=true -e SWIFT_VERSION=6.0 -e workspace=/swift-nio -e TEST_DEBUG=1 -e TEST_LOG_LEVEL=info swift:6.0-jammy bash -c swift --version && swift test -Xswiftc -warnings-as-errors --explicit-target-dependency-import-check error
```
Teach the script to indicate what is different on failure rather than
keeping quiet. This helps when the script is failing in CI but not
locally.
### Motivation:
The script is reporting that there are missing changes in CI on
swift-crypto, but I am unable to reproduce them locally. This is to get
immediate insight on why the check fails rather than keeping quiet.
### Modifications:
Include the diff in the update-cmake-lists script.
### Result:
update-cmake-lists script outputs the difference between what is in a PR
vs what it expects to see instead of shrugging and leaving it as an
exercise to the user.
Add assembly files for Windows
### Motivation:
These assembly files are now required to build on Windows Arm64.
### Modifications:
* The Windows assembly files are now handled in `update-cmake-lists.sh`.
### Result:
The assembly source files will be properly updated in `swift-crypto`.
### Motivation:
The current method of parsing the ASCII table output is brittle and
because the text output and the stored thresholds in JSON have different
precisions doesn't always work.
### Modifications:
* In the event that a deviation is found re-run the benchmarks in update
mode and print the git diff to the logs.
* The threshold update script scrapes this from the logs and applies it
locally so that it can then be committed.
* Add the functionality in the update script to update thresholds based
on diff in logs
### Result:
More reliable threshold updates.
An example of this working:
https://github.com/apple/swift-nio/actions/runs/15302748893?pr=3257
---------
Co-authored-by: George Barnett <gbarnett@apple.com>
### Motivation:
Nightly builds for 6.2 exist, we should test against them.
### Modifications:
Update the matrix nightly-next to point to 6.2. This will affect
downstream adopters but shouldn't break them because they don't gatet
PRs on this passing.
### Result:
nightly tests will run against Swift 6.2
An example of this working
https://github.com/apple/swift-nio/actions/runs/15185876449/job/42706027686?pr=3250
Update check-matrix-job.sh to default swift commands so that callers
which don't define them can still run workflows. At the moment they see
failures if they don't define any one of the commands.
### Motivation:
Swift 5.9 is no longer supported and CI has been updated to no longer
require it, we should disable it by default.
### Modifications:
* Default Swift 5.9 CI jobs to disabled
### Result:
Swift 5.9 CI jobs will not run unless opted-in
### Motivation:
The matrix job script used by the legacy Swift matrix workflow is still
used directly in some places where they need to run sidecar services, it
hasn't been updated for Swift 6.1
### Modifications:
Add cases for Swift 6.1 and clean up some missed changes for the
nightly-6.1 -> nightly-next naming change.
### Result:
Downstream services should be able to run matrix workflows with Swift
6.1.
Add and enable Swift 6.1 workflows
### Motivation:
Swift 6.1 has been released, we should add it to our CI coverage.
### Modifications:
* Update `appe/swift-nio/scripts/generate_matrix.sh`
* Update reusable adopters of `swift_test_matrix.yml`
* Update end-user adopter workflows of `swift_test_matrix.yml`
* Copy over test flags from Swift 6.0 jobs
### Result:
NIO tests against Swift 6.1 in CI and downstream repositories can
opt-in.
(Successful CI run with the workflows modified to use the changes on
this branch
https://github.com/apple/swift-nio/actions/runs/14400598096?pr=3196)
Static SDK installation script improvements
### Motivation:
Make static SDK installation scripts more modular and make
failures easier to diagnose.
### Modifications:
- Allow specification of binaries from outside the installation script
- Remove script dependency installation from the script
- Separate the Swift dependency installation into its own script
- Fatal error explicitly if the installed Swift can't be found
- Make curl operations for downloading artifacts explicitly fail
### Result:
Scripts which are easier to maintain and operate.
An example CI run with this in operation
https://github.com/apple/swift-nio/actions/runs/14329482180/job/40162003318?pr=3186
### Motivation
A few repositories have custom workflows for checking CMake, we would
like to unify this for maintenance and adoption simplicity.
Further to this at the moment they only check that the scripts are
up-to-date, we'd also like to have them build using CMake.
### Modifications
Add the re-usable workflow and supporting scripts
### Result:
Repositories using CMake can switch to the centralised workflow.
An example of it in action
https://github.com/apple/swift-crypto/actions/runs/13837092514/job/38782284575
### Motivation:
The static SDK workflow currently assumes a local setup script which
isn't true for external adopters.
### Modifications:
* The static SDK workflow now curls for the setup script.
* Run Static SDK checks on PR commits
### Result:
External adopters should be unblocked.
A re-usable static SDK compilation workflow
### Motivation:
We would like to offer the ability to opt-in to CI coverage for
compilation against the Linux static SDK.
### Modifications:
Introduce a workflow which will install a Swift snapshot and Linux SDK
and test the repository builds against it.
The script which sets up the VM will install snapshots and SDKs for
either branch or released-version snapshots. They can be specified as:
- a specific version string
- the 'latest' version, the script attempts to find the most recent
version enumerated on swift.org with both SDK and toolchain available
- the most recent snapshot for the specified branch name enumerated on
swift.org with both SDK and toolchain available
At the moment the script should work on ubuntu and amazonlinux OSes,
this can be expanded in the future.
The current matrix covers testing building on Ubuntu Jammy for the main
branch and latest released Swift version (6.0.3) and enables it on our
periodic CI runs.
See
https://github.com/apple/swift-nio/actions/runs/13722773053/job/38381907947?pr=3019
for a recent example of this workflow in action.
### Result:
Increased CI coverage
Use nightly_next as swift version
see https://github.com/apple/swift-nio/pull/3122
Motivation:
To not have to rename threshold directories when the nightly branch
changes.
Modifications:
* Use nightly_next as swift version in the matrix generation script
which
is picked up by the benchmark script.
* Move nightly-next thresholds and add legacy symlink
Result:
Benchmark thresholds will attempt to find directories named
nightly_next, not nightly_6_1.
Update generate matrix stript to use nightly_next
### Motivation:
Updating the CI parameters for changing nightly job targets with each
new Swift version (6.0, 6.1, ...) is onerous. Whilst this gives us some
value in the non-nightly pipelines by forcing adopters to apply thought
to what is applied to each supported version, the nightlies arguably
deliver less value in this regard as they tend to be non-required.
### Modifications:
Change flags to refer to nightly_next rather than nightly_6_1, implement
a shim so that the old environment variables still work.
### Result:
We will be able to modify adopter workflows to use nightly_next
terminology.
Example of a job where workflows are using the new matrix
https://github.com/apple/swift-nio/actions/runs/13520016728
### Motivation:
We recently added support for nightly 6.1 Swift and old scripts making
use of `check-matrix-job.sh` do not set `COMMAND_OVERRIDE_NIGHTLY_6_1`
which causes the script to fail. We should tolerate this.
### Modifications:
Add a default case for `COMMAND_OVERRIDE_NIGHTLY_6_1`.
### Result:
No failures if a caller doesn't specify `COMMAND_OVERRIDE_NIGHTLY_6_1`.
```
❯ SWIFT_VERSION=nightly-6.1 COMMAND="echo foo" COMMAND_OVERRIDE_5_9="" COMMAND_OVERRIDE_5_10="" COMMAND_OVERRIDE_6_0="" COMMAND_OVERRIDE_NIGHTLY_MAIN="" ./scripts/check-matrix-job.sh
** Running default command
foo
❯ SWIFT_VERSION=nightly-6.1 COMMAND="echo foo" COMMAND_OVERRIDE_5_9="" COMMAND_OVERRIDE_5_10="" COMMAND_OVERRIDE_6_0="" COMMAND_OVERRIDE_NIGHTLY_MAIN="" COMMAND_OVERRIDE_NIGHTLY_6_1="echo bar" ./scripts/check-matrix-job.sh
** Running nightly 6.1 command override
bar
```
### Motivation:
Some changes were missed in #3076, passing through parameters for the
new 6.1 version.
### Modifications:
Pass through more parameters, clean up misleading comments.
### Result:
6.1 nightly runs will be more like 6.0 nightly runs were
### Motivation:
Active development has moved to 6.1 so the Swift 6.0 nightlies no longer
deliver much value.
### Modifications:
* Deprecate 6.0 nightly CI configuration parameters
* Introduce new 6.1 nightly CI configuration parameters
* Update the matrix generation script to handle 6.1
### Result:
* No more nightly 6.0 CI, 6.1 nightlies are introduced.
* Configuration for 6.0 nightlies no longer has any effect.
* All repos are automatically opted-in to 6.1 nightly linux pipelines
**even if they opted out of 6.0**.
### Motivation:
The use of 'swift' here is very arbitrary and adds no value. The object
is only there at all because GitHub Actions seems to require it.
### Modifications:
Rename the top-level object from 'swift' to 'config' to maybe add value
to the intent of the object and hopefully be less cryptic.
### Result:
Everything should just keep working.
Introduce and adopt a new method for defining test matrices,
`swift_test_matrix.yml`.
⚠️ Any external adopters of the unit tests, Cxx interop and benchmarks
workflows are automatically opted in to use the new infrastructure.
### Motivation:
* The current matrix workflow has the limitation that it only supports
pre-defined sets of variables which are explored in the test matrix. At
the moment this is a pre-defined set of Swift versions on Linux and
Windows.
* Adding more means hard-coding them at multiple levels of the workflow
hierarchy.
* Currently skipped Windows matrix jobs show up as successes in the
GitHub UI leading to a misleading impression of good coverage.
### Modifications:
Introduce and adopt a new method for defining test matrices,
`swift_test_matrix.yml`. The new method is based around the approach of
defining the test matrix via a JSON object which may be supplied via an
input string from another workflow or from a file on-disk in a
repository.
Taking this approach means that we have the ability to add new targets
to the matrix simply by adding new elements to the JSON array,
increasing flexibility and the scope for future growth.
The unit tests, Cxx interop and benchmarks workflows are all modified to
use the new approach, this opts-in all downstream adopters. This should
be transparent to all downstream adopters.
In order to unify the Linux and Windows jobs I removed the use of the
`container:` GitHub Actions affordance in the Linux jobs which
transparently means all steps are executed within the nested container.
Instead we must manually call in to the docker container which
complicates scripting a little. I tested to see if doing this slowed
down the jobs (perhaps GitHub was caching the docker images more
intelligently) but it does not.
This approach follows the pattern of @FranzBusch 's open PR
https://github.com/apple/swift-nio/pull/2942
### Result:
* More flexible test matrix definitions
* No more false-passes for disabled Windows targets
Cxx interop CI appends `swiftSettings` to any existing settings rather
than overriding them.
### Motivation:
This better accommodates workflows for example which use Swift 5
language mode on a per-target basis.
### Modifications:
Cxx interop CI appends `swiftSettings` to any existing settings rather
than overriding them.
### Result:
Cxx interop CI jobs don't cause unexpected build configurations.
### Motivation:
At the moment the Cxx interoperability CI workflow doesn't actually test
Cxx interoperability.
### Modifications:
The script now looks for the precise way that the `Package.swift`
currently formats the target entry and appends a `swiftSettings` entry
to it.
This approach is pretty brittle but has the advantage that it's the same
on Linux and Darwin. If this formatting changes too often then we could
give up on this and assume the availability of gnu-sed instead (and make
sure it's present on our CI containers).
### Result:
All Cxx interoperability checks will check what they are named for.
---------
Co-authored-by: George Barnett <gbarnett@apple.com>
Motivation:
netstat -s can identify many system-wide issues that may be useful extra
data. There's no reason not to collect it.
Modifications:
Add netstat -s to the nio-diagnose script.
Result:
Better diagnostics
# Motivation
In the long term we want to add support for Windows in our Swift
packages. To do this we need a CI setup to ensure that our packages are
building and the tests are passing.
# Modification
This PR adds a windows job to our swift matrix that installs Swift on
the runner and then uses Swift PM to build and test the package. Windows
jobs are disabled by default since most of our packages require some
work to add Windows support.
# Result
We have a Windows CI pipeline that allows us to verify that our packages
are working.
# Motivation
Our support strategy includes the last three released Swift versions.
After the recent release of Swift 6 this means we now support 5.9, 5.10
and 6.0.
# Modification
This PR updates our CI configuration and scripts to align with our
supported Swift versions.
# Result
Up to date CI coverage.
# Motivation
Switch to the reusable workflows from the swiftlang organization for
soundness.
# Modification
This PR moves over to the new reusable soundness workflow. Additionally,
it also deletes all files in this repo for the soundness checks since
they are no longer needed.
# Motivation
Some of repos have `.in` and `.cmake` files including header license
comments.
# Modification
This PR adds support to the license check for those type of files.
# Result
Even more license headers.
…and remove docc plugin
# Motivation
Some repositories might want to run their soundness and matrix jobs
checks on different container images e.g. a repo might build against the
nightly toolchains only so their soundness jobs should also use nightly
images. Other repos might want to build against a different linux
distribution since their usual deployment target is not Ubuntu based.
Additionally, we currently require our repos to depend on the docc
plugin for building documentation which adds an unnecessary dependency
to all of our packages.
# Modification
This PR makes the container images for the soundness and matrix jobs
configurable. Furthermore, it modifies the docs check script to add the
docc-plugin dependency and removes the current dependency from our
manifest.
# Result
Our workflows can be adopted by more repositories.
# Motivation
We have quite a lot of shell scripts in our repo and want to make sure that they all pass `shellcheck`.
# Modification
This PR adds a GH action workflow to the soundness script for `shellcheck` and fixes up all errors and warnings.
# Result
No more shell/bash discussions
# Motivation
Some repositories may want to exclude files from being format checked.
# Modification
This PR moves the formatting into a separate script and adds an optional `.swiftformatignore` file that can be placed at the repo root similar to the `.licenseignore` file.
# Result
Repos can now decide what files are ignored when running the formatter.
# Motivation
Another reusable check is to make sure that all library products of a package are successfully building when consumed from a module that has Cxx interoperability enabled. Another check that's missing is running the integration tests.
# Modification
This PR adds two new checks to the reusable workflow. One to check for Cxx interoperability compatibility and another one to run the integration tests. I also fixed a misalgined name for the nightly benchmarks.
# Result
This should be one of the last reusable workflow checks.
* [GHA] Introduce reusable matrix workflow
# Motivation
We have a few checks that we want to run across all supported Swift versions e.g. unit tests and benchmarks. Right now we already added two matrix based jobs to the reusable workflow. This leads to a lot of duplication and harder maintenance.
# Modification
This PR splits off the current reusable workflow to only contain soundness related checks and introduces a new reusable matrix based workflow which can be customized to execute different commands.
# Result
It is now easy to setup multiple matrix based workflows.
* Code review
* Change nightly next
# Motivation
The next two reusable checks that we need to create are for broken symlinks and formatting. The latter is disabled for this repo for now since we need to discuss the formatting rules separately.
# Modification
This PR adds two new checks - broken symlinks and formatting.
# Result
Closer to having everything on GitHub actions.
Rick Newton-Rogers
Fabian Fett
Si Beaumont
Franz Busch
Marc Prud'hommeaux
Evan Wilde
Fabrice de Gans
Cory Benfield
Peter Adams