- Updated span logging keys to use camelCase for uniformity across connection and message events.
- Added checks to ensure project and user IDs are only logged if they are not empty, enhancing data integrity.
- Improved error handling and logging structure to maintain consistency in telemetry data.
- Introduced new arrays to capture subscribed channels and passed queries during connection and message events.
- Enhanced span logging to include details about channels and queries for better monitoring and analysis.
- Updated telemetry data structure to reflect the new metrics, improving traceability of realtime interactions.
- Introduced span logging for connection open and close events, capturing metrics such as inbound and outbound bytes, subscription counts, and response codes.
- Enhanced error handling with logging of exceptions during connection lifecycle.
- Updated the structure of the telemetry data to include project and user IDs for better traceability.
Realtime was ignoring _APP_WORKERS_NUM and always computing workers as
CPU × _APP_WORKER_PER_CORE, making it impossible to cap the worker count
without also changing the per-core multiplier. Prefer _APP_WORKERS_NUM
when set, falling back to the CPU × per-core calculation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root causes are fixed: Realtime now purges both projects:$projectId
and users:$userId from the shared cache before the respective reads,
eliminating the cross-process negative-cache races that surfaced as
intermittent 'Session is not valid.' failures on
appwrite-labs/cloud#3214's Realtime (dedicated) E2E.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-process negative-cache race in Realtime's onMessage handler: when
the WebSocket opens against a just-created project, an earlier router
probe or subscription lookup may have cached projects:\$projectId as
empty before the project actually existed. Although the HTTP worker
purges on createDocument, a narrow window lets Realtime re-populate the
stale empty entry in its own process cache, after which getProjectDB()
sees an empty Document and falls back to getConsoleDB(), which looks
for users in the console namespace and returns nothing — sessionVerify
then fails with 'Session is not valid.'
Surfaces on cloud#3214 as a recurring Realtime (dedicated) E2E failure.
Diagnosed via [realtime-project-diag] + [realtime-auth-diag]
instrumentation — the logs show consoleDb=appwrite consoleNs=console15x
(correct), project lookup empty, and subsequent user lookup landing in
the console namespace.
Purge is bounded: one cache DEL per WS message (not per read), runs
only when a projectId is present, and only forces the single
projects:\$projectId key to be re-read from adapter.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Temporary instrumentation to diagnose the 'Session is not valid.' failure
on appwrite-labs/cloud#3214's Realtime (dedicated) E2E. Emits:
- project ID in scope
- user ID from the session payload
- whether the user document was found in the project DB
- session count on the returned user document
- session-secret prefix being verified
- sessionVerify result
This will reveal whether the race is (a) user doc missing in project DB
(routing/provisioning issue) or (b) user doc found but sessions array
empty or mismatched (session write path issue). Revert once the root
cause is identified.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-process read-after-write race: the HTTP worker writes a new session
into the user document on /account login endpoints, then the client sends
an authentication frame over a different Swoole process (Realtime). Cache
propagation between processes is not guaranteed to be observed on the
very next read, so sessionVerify() occasionally fails with a stale user
document whose sessions array does not yet contain the just-created one.
Purge the cached user locally before the read so sessionVerify sees the
freshly-written session deterministically. Overhead is bounded: one DEL
on cache + one extra primary-key read, executed once per WebSocket
authentication frame (not per message).
Surfaces on PR appwrite-labs/cloud#3214 as an intermittent
'Session is not valid.' failure in RealtimeConsoleClientTest +
RealtimeCustomClientTest manual-authentication cases in dedicated mode.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ArnabChatterjee20k
Jake Barnby
loks0nandClaude Opus 4.7