PHPStan narrowed $dsnScheme to literal 'redis' after the other arms
were ruled out, making the default arm dead code.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stage-5 re-review C-Re2 outcome: the C8 decision was that Redis stays
on logs/console only, NOT on documentsdb/vectorsdb/database. However,
.github/workflows/ci.yml currently runs a Redis matrix entry with
_APP_DB_ADAPTER=redis that flows through the database pool.
Removing 'redis' from the database pool schemes here would break that
CI matrix entry. Per the fixup spec, do not remove it until the matrix
is reframed (point Redis at logs/console only) or dropped (rely on the
utopia-php/database adapter trait suite for coverage).
This commit replaces the soft NOTE with a loud WARNING that:
- states unambiguously that Redis is logs/console only;
- explains why 'redis' is still listed on the database pool today;
- documents the two paths to remove it (CI reframe or matrix drop);
- reaffirms that documentsdb/vectorsdb must NEVER allow 'redis'.
documentsdb and vectorsdb schemes were already redis-free; verified.
CONFLICT FLAGGED: this is the deviation called out in the fixup spec.
The C-Re2 fix cannot be completed in this pass without breaking CI.
Follow-up work: decide between reframing the CI matrix or dropping the
Redis matrix entry, then drop 'redis' from the 'database' and 'console'
pool schemes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
C8: Add a comment block above the connection pool definitions warning
that the `redis` scheme on the `database` and `logs` pools is for the
test matrix only — operators who set `_APP_DB_ADAPTER=redis` in
production will lose data on cache eviction or process restart.
Redis stays OFF the `documentsdb` and `vectorsdb` pools, which have
stricter durability and indexing requirements that the in-memory
Utopia Redis adapter does not provide.
M12: Restore the matrix-generator logic that picks the per-PR matrix
based on whether `composer.lock` changed `utopia-php/database`. PRs
that bump the database package run the full Mariadb+Postgres+Mongo+
Redis matrix in both dedicated and shared modes; everything else
runs the SQLite+Redis short matrix in shared mode. The previous
"TEMP — revert before merge" early `return` short-circuited that
logic and forced every PR through the short matrix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the new Redis database adapter through the Appwrite stack:
- registers.php: add 'redis' to schemes for console/database/logs pools,
add an explicit redis resource arm honouring the optional db segment,
and dispatch to Utopia\Database\Adapter\Redis in the database match
- composer.json: switch utopia-php/database constraint to
dev-feat-redis-adapter to pick up the new adapter (lockfile bump
follows after the database PR is pushed to origin)
- docker-compose.yml: add a dedicated redis-mirror service with
noeviction policy and AOF persistence so the database backend stays
isolated from the cache instance
- ci.yml: add Redis to default and full database matrices and export
the matching env vars (_APP_DB_HOST=redis-mirror, port 6379)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously every connection pool (console, database, documentsdb,
vectorsdb, logs) opened the same SQLite file, so they all serialised
through one writer lock — even though they're conceptually independent
databases. Treat _APP_DB_SQLITE_PATH as a directory + stem template and
replace the stem with the pool key, so each gets its own file and
unrelated writes no longer queue against each other.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tune SQLite for high-concurrency test workloads:
- synchronous=OFF skips fsync (safe for ephemeral test data only).
- 256 MB page cache + 2 GB mmap window cuts read I/O dramatically.
- temp_store=MEMORY keeps temporary tables off disk.
- busy_timeout to 60s and wal_autocheckpoint to every 10k pages so
long write bursts don't fight the checkpointer.
- 64 MB WAL size cap.
Mount the SQLite docker volume as tmpfs so writes never reach disk.
The test DB is wiped between CI runs so durability is irrelevant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
E2E test backend was hitting SQLITE_BUSY ("database is locked") under
Swoole's concurrent worker fanout, since SQLite serialises writes through
a single file lock. The previous 5s timeout was too tight for parallel
test load. NORMAL synchronous halves write fsync cost on WAL — fine for
ephemeral test databases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls latest utopia-php/database (e44768f) with the emulateMySQL flag and
opts SQLite into MariaDB-shape behaviour so the existing test suite, which
assumes MariaDB semantics, continues to pass against SQLite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The original fulltext filter only covered the createDatabase() helper, but
http.php has three more inline createCollection paths — two for default
buckets and one for project metadata collections — each of which builds
indexes via array_map without consulting adapter capability flags. SQLite
doesn't support fulltext, so the project-collection path tripped during
EVENT_START and crashed every worker.
Extract buildSupportedIndexes() and use it from all four sites so the
filter is applied consistently. The 1-line ?? defaults on lengths/orders
match the dbForProject site's tolerance for raw config without those keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
utopia-php/audit's Adapter\Database::setup() calls $db->exists($db->getDatabase())
and throws Utopia\Exception on false. Two issues:
1. SQLite adapter's exists($db, null) always returns false (it has no concept
of named databases — once the file is open, the DB exists), so the throw
path always fires.
2. The audit lib's `use Utopia\Exception` references a class that doesn't
exist anywhere in the dependency tree, so the throw fatals with
`Class "Utopia\Exception" not found`.
Both are upstream bugs, but for this PR I just sidestep the broken setup()
and create the audit collection directly through the regular Database API
using the adapter's schema documents. Same end result, no exists() probe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes for the SQLite test wiring:
1. .env had emptied _APP_DB_USER/_APP_DB_PASS/_APP_DB_ROOT_PASS, but the
mongodb/mariadb/postgresql containers still consume those during init
regardless of which adapter Appwrite uses. Empty values left the
side-running mongodb (needed for documentsdb) refusing to start, which
blocked docker compose up --wait for every CI matrix row, including
MongoDB. Restoring the original test credentials.
2. Bootstrap was unconditionally creating fulltext indexes from collection
configs. SQLite adapter explicitly returns false for
getSupportForFulltextIndex(), so it threw on the first such index. Now
the bootstrap consults the adapter and skips fulltext indexes when not
supported.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SQLite gives e2e tests a single shared file across all containers without
needing a heavyweight MariaDB/MongoDB/PostgreSQL service. WAL + busy_timeout
keep multi-process writers tolerable. Memory adapter was considered but its
per-process PHP arrays don't survive Swoole worker fan-out or cross-container
queue handoffs, so file-backed SQLite is the practical choice.
- registers: add 'sqlite' scheme to pool factory and worker db register,
configured with PRAGMA journal_mode=WAL, busy_timeout=5000, foreign_keys=ON
- compose: pass _APP_DB_SQLITE_PATH and mount appwrite-sqlite:/storage/sqlite
on every service that touches the main DB
- .env: switch _APP_DB_ADAPTER to sqlite, drop mariadb from COMPOSE_PROFILES
- ci.yml: add SQLite to the e2e matrix alongside MongoDB so each service
is timed against both adapters in the same workflow run
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Jake BarnbyandClaude Opus 4.7
Shmuel Fogel
frubio
Torsten Dittmann
Matej Bačo
Matej Bačo
ArnabChatterjee20k
premtsd-code
Harsh Mahajan
harsh mahajan