We have a number of copies of `UnsafeTransfer` and two copies of
`UnsafeMutableTransferBox` in our code base. Before introducing more of
those, lets centralize to just one using a `package` access modifier.
### Motivation:
We have a way to update Benchmarks thresholds from CI logs but not for
the older Integration Tests.
### Modifications:
A new script which pulls integration test logs and uses our existing
parsing script to pull out updated values.
### Result:
Easier threshold updates.
```
❯ ./dev/update-integration-test-thresholds.sh
Usage: ./dev/update-integration-test-thresholds.sh <url>
or: URL=<url> ./dev/update-integration-test-thresholds.sh
Example:
./dev/update-integration-test-thresholds.sh https://github.com/apple/swift-nio/pull/3442
./dev/update-integration-test-thresholds.sh https://github.com/apple/swift-nio/actions/runs/19234929677
URL=https://github.com/apple/swift-nio/pull/3442 ./dev/update-integration-test-thresholds.sh
```
```
❯ ./dev/update-integration-test-thresholds.sh 'https://github.com/apple/swift-nio/actions/runs/19234929677/job/54982236271?pr=3442'
** Processing run in apple/swift-nio
** Fetching integration test checks from workflow run 19234929677
** Pulling logs for Swift nightly-main (job 54982236243)
** Updated IntegrationTests/tests_04_performance/Thresholds/nightly-main.json
** Pulling logs for Swift 6.2 (job 54982236259)
** Updated IntegrationTests/tests_04_performance/Thresholds/6.2.json
** Pulling logs for Swift nightly-next (job 54982236267)
** Updated IntegrationTests/tests_04_performance/Thresholds/nightly-next.json
** Pulling logs for Swift 6.0 (job 54982236271)
** Updated IntegrationTests/tests_04_performance/Thresholds/6.0.json
** Pulling logs for Swift 6.1 (job 54982236345)
** Updated IntegrationTests/tests_04_performance/Thresholds/6.1.json
** Done! Updated 5 threshold file(s)
```
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
When debugging allocations it is frequently helpful to be able to use
bpftrace instead of the more limited heaptrack.
Modifications:
Offer an equivalent of malloc-aggregation.d for bpftrace.
Result:
Allocation recording on Linux is available.
Motivation:
When you diff two sets of alloc stacks there are often natural pairings
which aren't exact matches (e.g. the compiler inlined something in one
but not the other). These are tedious to pair up manually, we can make
that easier.
Modifications:
- Add a 'merge' which allows you to interactively pair up stacks across
the two different sets by using the levenshtein distance between them.
- After interactive merging is complete any remaining alloc stacks.
Result:
Easier to diff stacks
Motivation:
Sometimes it's helpful to view the alloc stacks from a program.
Modifications:
- Add a 'dump' subcommand to stackdiff allowing you to dump the stack
diff from heaptrack/dtrace/bpftrace
- You can also filter the stacks so that only certain stacks are
included
Result:
Can dump filtered alloc stacks to stdout
Motivation:
We often debug allocations on macOS, so being able to parse the output
from malloc-aggregation.d is valuable.
Modifications:
- Add a dtrace parser to stackdiff
Result:
Can diff outputs from dtrace
Motivation:
When debugging allocs its helpful to get a diff between two programs.
Modification:
- Adds a 'diff' subcommand which is roughly equivalent to the existing
'stackdiff-dtrace.py' script.
Result:
Alloc stacks can be diffed
Motivation:
Our alloc regression workflows are somewhat disparate: on macOS we have
dtrace and a script to diff two ouputs. On Linux we have heaptrack and
bpftrace but no way to analyze their outputs.
Diffing output from these tools can be quite tedious if the stacks vary
even a little (because the compiler decided to not inline a function,
for example).
Most of the tedium in this workflow is from pairing up equivalent stacks
across the two inputs. We can make this easier to do by ranking
suggestions based on how similar they are and letting the user decide
whether to accept the match or not.
Modifications:
- Add a subpackage with some internals for parsing heaptrack,
aggregating stacks, and measuring the similarity between them
- The CLI took is currently a no-op, it'll be added in subsequent PRs
Result:
Easier to diagnose alloc regressions
### Motivation:
The current method of parsing the ASCII table output is brittle and
because the text output and the stored thresholds in JSON have different
precisions doesn't always work.
### Modifications:
* In the event that a deviation is found re-run the benchmarks in update
mode and print the git diff to the logs.
* The threshold update script scrapes this from the logs and applies it
locally so that it can then be committed.
* Add the functionality in the update script to update thresholds based
on diff in logs
### Result:
More reliable threshold updates.
An example of this working:
https://github.com/apple/swift-nio/actions/runs/15302748893?pr=3257
---------
Co-authored-by: George Barnett <gbarnett@apple.com>
### Motivation:
A bit confusing to see docker mentioned in Readme file, when in reality
you can't run docker compose anymore.
### Modifications:
Updated README and CONTRIBUTING files, removed some scripts.
### Result:
No mentions of docker so users are not confused.
Co-authored-by: Rick Newton-Rogers <rnro@apple.com>
### Motivation:
The current readMultipleIntegers methods are used to consume multiple
integer values at once, but lack an equivalent nonmutating API. This can
lead to unnecessary complexity when users want to inspect values without
advancing readerIndex. THis patch introduces peekMultipleIntegers which
uses the current readerIndex, improving safety. This extends previous
work on peek methods (issue #2034, issue #2736).
### Modifications:
Updated the generation script (.sh file) to produce peekMultipleIntegers
variants for each arity of readMultipleIntegers.
Added new tests mirroring the existing multi-read/write tests
### Result:
Users can now inspect multiple integer values in a ByteBuffer without
consuming them, enabling safer workflows.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
### Motivation:
We have GitHub Actions CI running Swift Package Benchmark benchmarks on
each commit but when the values don't match updating them can be time
consuming. Particularly because you need to may need to exactly match
the running environment in CI to get matching numbers.
### Modifications:
Added a new dev script which pulls the output for the benchmark GitHub
actions runs on a PR and updates your local threshold files
### Result:
Easier benchmark threshold updates
Example interaction (actually on my branch of NIO SSL where I've messed
with the thresholds so they need an update):
```
❯ FETCH_PR_URL=https://github.com/rnro/swift-nio-ssl/pull/5 OUTPUT_DIRECTORY=Benchmarks/Thresholds/ ../swift-nio/dev/thresholds-from-benchmark-output.sh
** Updating: Benchmarks/Thresholds//nightly-main/NIOSSHBenchmarks.SimpleHandshake.p90.json job:39048331461
** Nothing to update: rnro/swift-nio-ssl 5.10 job:39048331486
** Nothing to update: rnro/swift-nio-ssl 5.9 job:39048331454
** Nothing to update: rnro/swift-nio-ssl 6.0 job:39048331469
** Nothing to update: rnro/swift-nio-ssl nightly-next job:39048331466
### Motivation:
The bootstraps are the next most obvious target for cleaning up
Sendability issues. These issues are mostly just missing annotations,
but there are a few places where we had actual latent threading bugs
that were missed. A lot more of the control flow is made more explicit
in this patch, and in general it should get a lot easier to be confident
that the code is correct.
### Modifications:
- Add necessary `@Sendable` annotations
- Clean up some incorrect `self` captures of bootstraps, which are not
`Sendable`, and which could lead to real threading bugs
- Make a few methods `static` to avoid needing to capture `self` at all.
- Make a few threading assumptions clear by using the isolated views or
the sync operations
- Add some missing `Sendable` constraints on interior generic functions.
### Result:
Better safety, better correctness in the bootstraps.
Motivation:
The alloc-limits-from-test-output doesn't print usage if you don't pass
in a format arg, it just sits and waits for stdin.
Modifications:
Print usage if running interactively
Result:
Easier to get usage
Motivation:
We removed the alloc limits script, but it is still very useful. It just
couldn't produce JSON. So now it can!
Modifications:
Re-adds the alloc limits script.
Makes it produce JSON.
Result:
JSON can be produced from the CI output.
# Motivation
We have quite a lot of shell scripts in our repo and want to make sure that they all pass `shellcheck`.
# Modification
This PR adds a GH action workflow to the soundness script for `shellcheck` and fixes up all errors and warnings.
# Result
No more shell/bash discussions
# Motivation
We added new CI to run both nightly main and nightly next but forgot to update the benchmark thresholds script
# Modification
This PR updates the benchmark thresholds script and adds a nightly next docker compose image.
# Result
We can use the update benchmark thresholds script again.
# Motivation
We currently have an unchecked `Sendable` conformance on `ChannelOptions.Storage` which is not necessary and in fact we are missing a proper `Sendable` requirement on the `ChannelOption.Value` protocol right now.
# Modification
This PR adds the `Sendable` requirement on the `ChannelOption.Value` and replaces the unchecked conformance on the storage with a regular one.
# Result
No more `Sendable` warnings in strict concurrency mode inside `ChannelOptions`.
* Update swift-tools-version to 5.7 in integration tests
* Fix flaky test
* Add missing AtomicCounter dependency in generated Package.swift
* Update swift-tools-version in integration test helper Swift projects
* Make test clearer
* Don't run MTELG integration test on Swift 5.11+
It hasn't been properly tested
# Motivation
We want to migrate our allocation and later on also our performance tests to use the `package-benchmark` plugin. This plugin makes writing benchmarks way easier than our current setup. Furthermore, debugging benchmarks is also possible from within Xcode now.
# Modification
This PR adds the setup for the benchmarking infrastructure and connects it with out
# Result
Allocations tests are more accessible and easier to iterate.
Motivation:
Running the script failed as it looked for a CI build for Swift 5.5,
which is not run by our CI anymore.
Modifications:
* don't look for CI builds for Swift 5.5
* additionally, also check for CI build for Swift 5.9
Motivation:
Channels can read `ChannelOptions.maxMessagesPerRead` times from a
socket in each read cycle. They typically re-use the same buffer for
each read and rely on it CoWing if necessary. If we read more than once
in a cycle then we may CoW the buffer. Instead of reusing one buffer we
can reuse a pool of buffers limited by `maxMessagesPerRead` and cycle
through each, reducing the chance of CoWing the buffers.
Modifications:
- Extend `RecvByteBufferAllocator` to provide the size of the next
buffer with a default implementation returning `nil`.
- Add an recv buffer pool which lazily grows up to a fixed size and
attempts to reuse buffers where possible if doing so avoids CoWing.
Results:
Fewer allocations
Motivation:
Cocoapods appears to have a different idea of the dependency graph to
SPM which has led to a handful of build failures (e.g.
https://github.com/apple/swift-nio/issues/2073).
This appears to have originated when we dropped the explicit dependency
on `CNIOAtomics` from `NIO` (https://github.com/apple/swift-nio/pull/1719).
We can work around this by listing all transitive dependencies as
requirements in the generated podspecs.
Modifications:
- Add a script to list transitive dependencies for a given module.
- Fix a bug in build_podspecs.sh which did a `pod repo update` even if
the pods were not being uploaded.
- Update build_podspecs.sh to use transitive dependencies.
- Use Python3 in 'dev/stackdiff-dtrace.py' and update soundness.sh
Result:
Podspecs should include all transitive dependencies.
We regressed allocs in nightly, probably for a Swift bug. Fixing it is
tracked in #2070, for now we're going to raise the limits so CI isn't
moaning all the time.
Motivation:
Many network protocols (especially for example NFS) have quite a number
of integer values next to each other. In NIO, you'd normally parse/write
them with multiple read/writeInteger calls.
Unfortunately, that's a bit wasteful because we're checking the bounds
as well as the CoW state every time.
Modifications:
- Provide read/writeMultipleIntegers for up to 15 FixedWidthIntegers.
- Benchmarks
Result:
Faster code. For 10 UInt32s, this is a 5x performance win on my machine,
see benchmarks.
Make Swift 5.2 the minimum requirement, dropping support for Swift 5.0 and 5.1.
Motivation:
Whenever we have problems, Swift 5.0 and 5.1 seem to be the culprits. Dropping support for these very old versions will require less maintenance and free up our time to work on new features.
Modifications:
Set the tools version in Package.swift to 5.2
Remove CI configurations for 5.0 and 5.1
Update the various readmes to reflect that this change will be rolled out in NIO 2.30.0
Result:
Swift 5.2 is the minimum version of Swift required to use NIO.
Motivation:
We need license headers but the soundness script couldn't check if the
scripts don't have a `.sh` extension.
Modification:
- Add missing license headers.
- Better license checking (search for shell shebangs too)
Result:
Proper license attribution.
Motivation:
The allocation limit updater script is handy but so far only worked for
the SwiftNIO core repo.
Modifications:
- allow the user to pass other URL prefixes and repos
Result:
Can be used for other repos
Motivation:
We often hit a bunch of problems with LLDB, especially on Linux. So
maybe it would be cool if we could smoke test LLDB using SwiftNIO.
Modifications:
Add script `dev/lldb-smoker` which does the following things:
- takes 2000 random picks amongst all the files in the package
- for each of those files, pick a random line number
- write an LLDB script which sets break points for each of the file/line
pairs (one-shot breakpoints)
- runs the program
- for each breakpoint hit, run `frame variable` to list all the
variables
Result:
LLDB smoke test
Motivation:
5.0 is several years old now, and while we still support it there is
beginning to be some conflict with performance optimizations for newer
Swift versions. We should stop counting the allocation counts on this
version now: we no longer care as much about regressions.
Modifications:
- Stop setting alloc counter limits on 5.0
- Remove the script that tries to add them back.
Result:
No longer measure allocs on 5.0
Motivation:
When doing bigger alloc count changes, it's really tedious to apply them
manually given that the CI has the right numbers already.
Modifications:
Add a script to download and update them automatically.
Result:
Less toil.
Motivation:
The dev/alloc-limits-from-test-output script is handy to get correctly
formatted (and sorted) alloc counter limits from test output.
Modifications:
When running the alloc counter script itself, it outputs suggested
`export MAX_ALLOCS_ALLOWED` lines. We're now also parsing those, so you
can easily go from `export ...*` to the docker-compose form :).
Result:
Even easier alloc counter limits.
Motivation:
Usually, we add a more or less random number of slack allocations to
make sure the tests don't spuriously fail. This makes it quite costly to
support new Swift versions.
Modifications:
Add a script which can spit you out the right allocation limits
including slack.
Result:
Easier to support new Swift versions
Motivation:
Some of the output from 'malloc-aggregation.d' trips up the
'stackdiff-dtrace.py' script -- especially when used with the output
from recent 5.4 builds. The regex for numbers includes hex which trips
us up when we try to convert that match to an int. Since we only use
this pattern for matching allocation counts, where we don't expect hex,
we can just modify the pattern.
Also the input parsing is a little tricky to follow without context so I
tidied it up little.
Modifications:
- relax the number regex (we don't expect counts to be in hex)
- drop the stack regex (it wasn't used)
- aggregate the current stack in a list (instead of a string)
- filter out non-stack lines (which were erroneously used to key the
first stack!)
- a few comments
Result:
stackdiff-dtrace.py doesn't blow up on output from newer builds
Motivation:
The Swift runtime is now using malloc_zone_*, we need to implement
replacements for these too. This is just a first pass, eventually, we
should implement _all_ replacements as `malloc_zone_memalign` which is
powerful enough to implement all others.
Modifications:
Provide new replacements.
Result:
Alloc tests work again on macOS.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Allocation diffing script no longer drops data.
Motivation:
Looking at allocations is hard enough without the script getting the allocation counts wrong.
Modifications:
Where multiple allocations resolved to the same "key" only the last allocation was accounted for. I have changed the script to store all allocations - still keyed in the same way. When diffing the total number of allocations is used - if they are different the total and all contributing stack traces are output.
Added a total for all allocations everywhere at the end together with a difference number.
Output all stack traces before and after contributing to a detected diff.
Update the documentation to reflect changes.
Change the threshold for reporting diffs from > 1000 to >= 1000. This means if allocations go from 1000 to 2000 they are reported as a difference rather than a new allocation.
Result:
It is now easier - if somewhat more verbose to compare allocations.
Motivation:
The make-single-file-spm script used to use a hidden folder in /tmp
which isn't very helpful when tar'ing up the result for a bug report.
Modifications:
Switch to a non-hidden folder.
Result:
Easier bugs.swift.org reports.
Motivation:
We recently grew a script to diff the output of `malloc-aggregation.d`,
we should document how this can be used when debugging allocation
regressions.
Modifications:
- Update the doc to include an example of using the script to debug an
allocation regression
- Fix a bug in the script where before/after were mixed up
- Fix a bug in the script where usage would print '()\n' instead of '\n'
Result:
- Better info on debugging allocations
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Motivation:
NIO has allocation counter tests, NIO also has dev/malloc-aggregation.d
which can give you aggregated stacks that allocated alongside how many
times they allocated. What's missing is a script that can diff two
aggregated stack traces.
Modifications:
Add dev/stacktrace-dtrace which allows aggregated stack trace diffing.
Result:
Easier to track performance regressions.
Motivation:
make-single-file-spm has a weird bug where it would chop off trailing
`n` characters in all lines due to a badly set `IFS` bash variable.
Modifications:
Set `IFS=""` which is correct.
Result:
make-single-file-spm will not chop off trailing `n`s.
Motivation:
make-single-file-spm is a simple bash script that can turn a single file
into a SwiftPM project with multiple modules (all defined in a single
file with special markers such as `// MODULE: Foo`. This is mostly
useful for performance evaluation of Swift becuase in almost all cases,
you'll need multiple modules which is quite a bit of work to set up.
Modifications:
Add make-single-file-spm
Result:
More useful scripts in the NIO repo.
Motivation:
When counting the number of allocations it's important not to forget an
allocation function that Swift uses.
Modifications:
add posix_memalign to dev/malloc-aggregation.d
Result:
malloc aggregations accurate with recent Swift versions
Fabian Fett
Rick Newton-Rogers
Cory Benfield
George Barnett
Jaleel Akbashev
Raghav Roy
Peter Adams
Franz Busch
Si Beaumont
Gustavo Cairo
Felix Schlegel
Johannes Weiss
David Evans