Files
coturn/filc/docker-entrypoint.sh
Pavel Punsky 24f474878e Filc harness and pointer typedefs (#1896)
## Summary

- Add a self-contained Fil-C build/test harness under `filc/` that
mirrors the existing `fuzzing/` pattern: one host script
(`filc/run-local.sh`) builds an Ubuntu 24.04 image with the
[Fil-C](https://github.com/pizlonator/fil-c) optfil 0.678 toolchain,
builds turnserver with `CC=filcc`/`CXX=fil++`, runs unit tests + system
tests, and drops a per-run timestamped log directory with `SUMMARY.txt`
+ `ISSUES.txt`.
- Fix the two real Fil-C compatibility bugs the harness surfaces by
changing `ur_map_value_type` and `ur_addr_map_value_type` from
`uintptr_t` to `void *` in `src/server/ns_turn_maps.h`.

## Why

[Fil-C](https://fil-c.org) is a memory-safe C/C++ compiler (Clang 20
fork) that pairs every pointer with an "InvisiCap" capability and turns
UB into deterministic panics with no `unsafe` escape hatch. Putting
coturn through it answers two questions: (a) does it compile unmodified,
and (b) does it run correctly under capability-enforced memory safety.
After this PR, the answer is **yes** for both — turnserver,
turnutils_peer, and turnutils_uclient relay TCP/TLS/UDP/DTLS traffic
with full Fil-C enforcement, all unit tests pass, and
`examples/run_tests_conf.sh` runs end-to-end.

## What's in the PR

### `filc/` harness (commit 1)

| File | Purpose |
|---|---|
| `filc/Dockerfile` | Ubuntu 24.04 + Fil-C optfil 0.678 (extracts the
nested `fil.tar.xz` to `/opt/fil`); `--platform linux/amd64` so it works
on Apple Silicon under emulation. |
| `filc/run-local.sh` | Host-side: build image, create
`filc/logs/<UTC-ts>/`, run container with source mounted read-only and
log dir mounted r/w. |
| `filc/docker-entrypoint.sh` | In-container orchestrator. Phases: env /
source-copy / build / unit-tests / system-cli / system-conf. Runs every
phase even when a prior one fails (no aborting mid-pass). Captures
per-phase logs + a combined `all.log` + JUnit XML for ctest. Greps
panics/errors into `ISSUES.txt`. Downgrades `system-*` phases to FAIL
when `examples/run_tests*.sh` prints `FAIL` despite exiting 0 (existing
fragility in those scripts). |
| `filc/build.sh` | `cmake … -DBUILD_TESTING=ON -DCMAKE_C_COMPILER=filcc
-DCMAKE_CXX_COMPILER=fil++ -DCMAKE_BUILD_TYPE=RelWithDebInfo`, then
build. |
| `filc/.gitignore` | Ignore the on-host `logs/` dir. |

The harness also bumps the post-launch sleep in `examples/run_tests.sh`
from 2s to 6s **only inside the container** (sed-in-place on the copied
source; upstream is untouched). Under linux/amd64 emulation the
Fil-C-built turnserver isn't accepting TCP at 2s, so the first sub-test
races and prints `FAIL`. Matches the 5s sleep already used by
`run_tests_conf.sh`.

### Pointer-typedef fixes (commit 2)

`src/server/ns_turn_maps.h`:

```diff
-typedef uintptr_t ur_map_value_type;
+typedef void *ur_map_value_type;
...
-typedef uintptr_t ur_addr_map_value_type;
+typedef void *ur_addr_map_value_type;
```

**Why this is necessary.** Both maps store pointers, but their value
slot is integer-typed. Every existing `_put` site casts a pointer
through `(ur_*_value_type)` to store, and every `_get` site casts back.
Under standard C this is a well-defined no-op. Under Fil-C, casting a
pointer to `uintptr_t` discards its InvisiCap; casting back yields a
pointer with a non-null address but a NULL Fil-C object — the next
dereference panics with `cannot read pointer with null object`.

The harness caught two such panics, both in the auth-resume /
relay-allocate flow:

1. `src/server/ns_turn_server.c:3248` — `ss->client_socket`, where `ss`
came from `sessions_map` (a `ur_map`).
2. `src/apps/relay/turn_ports.c:225` — `tp->mutex`, where `tp` came from
`ip_to_turnports_*` (a `ur_addr_map`) via `turnipports_add`.

**Why this is also a correctness improvement on a normal build.** The
new typedef makes the API strictly more type-safe — the compiler now
enforces "you put a pointer in." It eliminates a class of accidental
misuse (storing a non-pointer integer where a pointer was expected) that
the integer typedef silently allowed. Same generated code on a normal
build; different (correct) Fil-C semantics.

**Audit.** Verified before changing:
- All `ur_map_put` / `lm_map_put` / `ur_addr_map_put` callers store
pointer-typed values exclusively (no callers store raw integers).
- No internal arithmetic on the value type anywhere in `ns_turn_maps.c`.
- `ur_map_del_func` / `ur_addr_map_func` implementations either don't
exist (all `_del` callers pass `NULL`) or immediately cast their
parameter to a real pointer type — no source change needed.
- `KHASH_MAP_INIT_INT64(3, ur_map_value_type)` works identically with
`void *`.
- `ur_addr_map`'s `addr_elem.value` is assigned, read, compared for
truthiness, and cleared with `= 0` — all valid for `void *`.

## Test plan

- [ ] `filc/run-local.sh` reports all six phases PASS (env / source-copy
/ build / unit-tests / system-cli / system-conf), `ISSUES.txt` carries
no Fil-C panic / safety / sanitizer entries.
- [ ] Local `cmake -S . -B build -DBUILD_TESTING=ON && cmake --build
build && ctest --test-dir build --output-on-failure` is green (no
regression on the regular build).
- [ ] `examples/run_tests.sh` and `examples/run_tests_conf.sh` are green
on Linux per `CLAUDE.md`.
- [ ] Existing `fuzzing/run-local.sh ASan 0 -runs=1` still passes (the
new `filc/` directory is independent and shouldn't perturb anything).
2026-05-04 18:49:18 -07:00

213 lines
6.5 KiB
Bash
Executable File

#!/usr/bin/env bash
# In-container orchestrator. Runs each phase, captures stdout+stderr to its
# own log file under /out, records exit codes in /out/SUMMARY.txt, and never
# aborts on a single phase failure — so one run surfaces every issue.
# Deliberately not setting `set -e` at the top level: each phase manages its
# own failure handling so subsequent phases still run.
set -uo pipefail
readonly mounted_src=/src
readonly work_src=/work/coturn
readonly log_dir=/out
mkdir -p "${log_dir}"
: > "${log_dir}/all.log"
: > "${log_dir}/SUMMARY.txt"
printf '%-18s %-6s %-5s %s\n' phase status exit log >> "${log_dir}/SUMMARY.txt"
overall_status=0
ts() { date -u +'%Y-%m-%dT%H:%M:%SZ'; }
# run_phase <name> <log-file> <cmd...>
# Streams output to both the per-phase log and /out/all.log, with timestamps.
# Records the result in SUMMARY.txt. Updates overall_status on failure.
run_phase() {
local name=$1
local log_file=$2
shift 2
echo
echo "===== [$(ts)] PHASE: ${name} ====="
{
echo "===== [$(ts)] PHASE: ${name} ====="
echo "Command: $*"
} | tee -a "${log_dir}/${log_file}" "${log_dir}/all.log" >/dev/null
local start_epoch end_epoch dur status
start_epoch=$(date -u +%s)
set +e
( "$@" ) 2>&1 \
| gawk '{ print strftime("[%Y-%m-%dT%H:%M:%SZ] ", systime(), 1) $0; fflush(); }' \
| tee -a "${log_dir}/${log_file}" "${log_dir}/all.log"
status=${PIPESTATUS[0]}
set -e
end_epoch=$(date -u +%s)
dur=$((end_epoch - start_epoch))
local label=PASS
if [ "${status}" -ne 0 ]; then
label=FAIL
overall_status=1
fi
printf '%-18s %-6s %-5s %s\n' "${name}" "${label}" "${status}" "${log_file}" \
>> "${log_dir}/SUMMARY.txt"
echo "===== [$(ts)] PHASE ${name}: ${label} (exit=${status}, ${dur}s) ====="
return 0
}
# ----- phase: env -------------------------------------------------------------
phase_env() {
echo "## uname"
uname -a
echo
echo "## /etc/os-release"
cat /etc/os-release
echo
echo "## CPU model"
grep -m1 'model name' /proc/cpuinfo || true
echo
echo "## Apt-installed build deps"
dpkg -l | grep -E 'libssl|libevent|sqlite|cmake|pkg-config|^ii curl' || true
echo
echo "## filcc --version"
filcc --version || true
echo
echo "## /opt/fil contents"
ls -la /opt/fil/bin 2>/dev/null | head -50 || true
echo
echo "## Fil-C-built libs available in /opt/fil"
ls /opt/fil/lib*/libssl* 2>/dev/null || true
ls /opt/fil/lib*/libcrypto* 2>/dev/null || true
ls /opt/fil/lib*/libevent* 2>/dev/null || true
ls /opt/fil/lib*/libsqlite* 2>/dev/null || true
echo
echo "## pkg-config sees"
for pkg in openssl libevent sqlite3; do
printf '%-12s ' "${pkg}"
pkg-config --modversion "${pkg}" 2>&1 || echo "(missing)"
done
echo
echo "## env"
env | sort | grep -E '^(CC|CXX|PATH|CMAKE_|PKG_CONFIG|LD_)' || true
}
# ----- phase: source-copy -----------------------------------------------------
phase_source_copy() {
rm -rf "${work_src}"
mkdir -p "${work_src}"
echo "Copying ${mounted_src} -> ${work_src} (excluding .git, build*, .DS_Store)"
tar \
--exclude='.git' \
--exclude='.DS_Store' \
--exclude='build' \
--exclude='build-*' \
--exclude='filc/logs' \
-C "${mounted_src}" \
-cf - . | tar -C "${work_src}" -xf -
echo "Copy complete."
du -sh "${work_src}"
ls "${work_src}"
}
# ----- phase: build -----------------------------------------------------------
phase_build() {
LOG_DIR="${log_dir}" coturn-filc-build
}
# ----- phase: unit-tests ------------------------------------------------------
phase_unit_tests() {
ctest \
--test-dir "${work_src}/build" \
--output-on-failure \
--output-junit "${log_dir}/unit-tests.junit.xml"
}
# ----- phase: system-tests-cli ------------------------------------------------
phase_system_cli() {
cd "${work_src}/examples"
# Bump the post-launch sleep from 2s to 6s. Under linux/amd64 emulation on
# Apple Silicon, the Fil-C-built turnserver is not yet accepting TCP at 2s,
# so the very first sub-test races and prints FAIL. Matches run_tests_conf.sh.
sed -i 's/^sleep 2$/sleep 6/' run_tests.sh
./run_tests.sh
}
# ----- phase: system-tests-conf -----------------------------------------------
phase_system_conf() {
cd "${work_src}/examples"
./run_tests_conf.sh
}
# Run every phase, but skip later ones if a prerequisite already failed.
run_phase env env.log phase_env
run_phase source-copy source-copy.log phase_source_copy
if grep -E '^source-copy +PASS' "${log_dir}/SUMMARY.txt" >/dev/null; then
run_phase build build.log phase_build
fi
if grep -E '^build +PASS' "${log_dir}/SUMMARY.txt" >/dev/null; then
run_phase unit-tests unit-tests.log phase_unit_tests
run_phase system-cli run_tests.log phase_system_cli
run_phase system-conf run_tests_conf.log phase_system_conf
else
echo "Build failed; skipping test phases."
for skipped in unit-tests system-cli system-conf; do
printf '%-18s %-6s %-5s %s\n' "${skipped}" SKIP -- skipped \
>> "${log_dir}/SUMMARY.txt"
done
fi
# examples/run_tests.sh prints "FAIL" on failed test cases but exits 0.
# Catch that explicitly.
for phase_log in run_tests.log run_tests_conf.log; do
if [ -f "${log_dir}/${phase_log}" ] && \
grep -Eq '(^|\] )FAIL$' "${log_dir}/${phase_log}"; then
echo "Detected FAIL marker in ${phase_log}; downgrading phase status."
case "${phase_log}" in
run_tests.log) target=system-cli ;;
run_tests_conf.log) target=system-conf ;;
esac
# Mark failure and update overall status.
awk -v t="${target}" '
$1==t { printf "%-18s %-6s %-5s %s\n", $1, "FAIL", "1", $4; next }
{ print }
' "${log_dir}/SUMMARY.txt" > "${log_dir}/SUMMARY.txt.new"
mv "${log_dir}/SUMMARY.txt.new" "${log_dir}/SUMMARY.txt"
overall_status=1
fi
done
# ----- issues extraction ------------------------------------------------------
{
echo "# Issues extracted from /out/*.log"
echo "# Generated: $(ts)"
echo
for f in "${log_dir}"/*.log; do
[ -f "${f}" ] || continue
base=$(basename "${f}")
[ "${base}" = "all.log" ] && continue
matches=$(grep -nE \
'error:|undefined reference|warning:|FAIL$|SEGV|Segmentation fault|panic|Filc panic|filc panic|AddressSanitizer|LeakSanitizer|UndefinedBehaviorSanitizer|MemorySanitizer|assertion|Assertion .* failed|cannot find' \
"${f}" || true)
if [ -n "${matches}" ]; then
echo "## ${base}"
echo "${matches}"
echo
fi
done
} > "${log_dir}/ISSUES.txt"
echo
echo "===== [$(ts)] FINAL SUMMARY ====="
cat "${log_dir}/SUMMARY.txt"
echo "===== overall exit: ${overall_status} ====="
exit "${overall_status}"