Commit Graph

270 Commits

Author SHA1 Message Date
Jake Barnby b747c97561 fix(advisor): address review comments on scopes and schema
- Register dedicated reports.write scope and switch deleteReport to it
  so cloud can issue narrowly-scoped delete keys without granting
  insights.write.
- Make insights.parentResourceInternalId optional with null default to
  match its companion parentResourceType/parentResourceId fields and
  unblock insights with no parent (e.g. database-level performance
  insights).
- Tighten Insight.reportId model description: insights always belong to
  a report, ad-hoc insights are not supported.
- Add reports.write to default test API key and admin role so existing
  e2e tests using serverHeaders() can hit the delete endpoint.
- Bump APP_CACHE_BUSTER for the schema change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 12:18:39 +12:00
copilot-swe-agent[bot] 6d0eab2583 refactor(advisor): make insights API read-only in CE
Agent-Logs-Url: https://github.com/appwrite/appwrite/sessions/8d7897b5-ac68-487d-954a-be717380bf66

Co-authored-by: abnegate <5857008+abnegate@users.noreply.github.com>
2026-05-08 06:07:23 +00:00
Jake Barnby 6c6782dd28 Merge remote-tracking branch 'origin/1.9.x' into pr-12194-feat-insights-module
# Conflicts:
#	app/config/scopes/project.php
2026-05-06 20:33:40 +12:00
Jake Barnby 4fc3e9c386 refactor(insights): manager-only Create endpoint + native categories array
Insights are produced by internal Appwrite services (edge, executor,
background analyzers) — never by user clients. Move the ingestion
endpoint accordingly.

- Move Http/Insights/Create.php → Http/Manager/Insights/Create.php.
- Path: /v1/insights → /v1/manager/insights. SDK Method marked
  `hide: true` and namespaced under `manager` so generated SDKs don't
  expose it. Auth narrowed from [ADMIN, KEY] to [KEY] only.
- New scope `insights.manager`. Not granted by any user role
  (app/config/roles.php) — Cloud/edge teams configure their internal
  key issuance to grant it. `insights.write` description trimmed to
  the user-facing surface (update/dismiss/delete) since create is now
  manager-only.
- Reports, ListInsights, GetInsight, UpdateInsight, DeleteInsight
  remain at /v1/insights/*. Existing scopes unchanged.
- Reports `categories` switched from JSON-encoded string to a native
  array<string> column (size 64 per entry, up to 32 entries via the
  endpoint validator). MySQL JSON-array indexes are weak and we never
  query individual entries — read+rewrite only.
- E2E test API key in tests/e2e/Scopes/ProjectCustom.php gains
  insights.read/write/manager + reports.read/write so the manager
  endpoint is reachable from the test harness.
- E2E InsightsBase.createInsight() helper now POSTs /manager/insights.
- New testCreateRequiresManagerScope verifies a key with
  insights.read/write but no insights.manager is rejected with 401.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 17:20:49 +12:00
Matej Bačo b0220292a7 Rename policies to prevent double scope usease 2026-05-05 12:58:06 +02:00
Matej Bačo aaf91f3816 Improve scopes quality 2026-04-29 13:52:13 +02:00
Matej Bačo 93f7a0d902 GitHub oauth endpoint 2026-04-24 11:17:18 +02:00
Matej Bačo bdbc5b92df Fix after code review 2026-04-23 13:47:31 +02:00
Matej Bačo 9065d9ada4 Add mocks scopes 2026-04-22 12:13:10 +02:00
Matej Bačo 0d27c59cb8 Merge branch '1.9.x' into feat-public-project-policies 2026-04-22 09:57:48 +02:00
Matej Bačo 5f9dc0fcd8 Req & res filters, review fixes 2026-04-21 13:58:36 +02:00
Matej Bačo 489b2c4e21 Add new scopes 2026-04-17 16:45:04 +02:00
Matej Bačo 27fc8058b9 Fix failing tests 2026-04-11 14:19:05 +02:00
Matej Bačo eef2a7abdf Fix scopes 2026-04-08 10:01:52 +02:00
Jake Barnby 8666bf1325 Merge branch '1.9.x' into revert-11585-revert-11402-sync-mongodb 2026-03-24 03:03:02 +00:00
Matej Bačo 01142bba2c Merge branch '1.8.x' into feat-public-project-variables-api 2026-03-23 12:21:06 +01:00
ArnabChatterjee20k c7907932e4 Revert "Revert "Documentsdb + vectordb (latest)"" 2026-03-19 20:30:42 +05:30
ArnabChatterjee20k 9917f95dfd Revert "Documentsdb + vectordb (latest)" 2026-03-19 19:18:27 +05:30
Matej Bačo 8af0efafd4 Merge branch '1.8.x' into feat-public-project-variables-api 2026-03-19 14:17:26 +01:00
Matej Bačo 7e7cac017c Merge pull request #11582 from appwrite/fix-webhooks-duplication
Fix: webhook endpoints duplication
2026-03-19 14:16:30 +01:00
Matej Bačo 37a7c70c2b Fix webhook endpoints duplication 2026-03-19 11:27:13 +01:00
ArnabChatterjee20k 7071e6f080 Merge remote-tracking branch 'origin/1.8.x' into sync-mongodb 2026-03-19 15:55:28 +05:30
Matej Bačo ba94bff8d4 Public project variables API 2026-03-18 14:48:31 +01:00
Matej Bačo 677bb048cc Introduce new webhooks API 2026-03-17 11:03:18 +01:00
ArnabChatterjee20k fddaebf254 Merge branch 'sync-mongodb' into vectorsdb 2026-03-09 15:51:30 +05:30
ArnabChatterjee20k d6bc9f120e Merge remote-tracking branch 'origin/1.8.x' into sync-mongodb 2026-03-09 15:50:48 +05:30
ArnabChatterjee20k 681d930da3 renamed vectordb to vectordb 2026-03-09 13:51:48 +05:30
Jake Barnby 65780d75f9 fix: test flakes — correct index length fallback, add retries
- Fix getMaxIndexLength() fallback from 768 to 767 to match MariaDB's
  actual InnoDB index key limit
- Add retry logic to getConsoleVariables() to handle intermittent 401s
- Add retry logic to API key creation in ProjectCustom to prevent
  cascading 401 failures in test methods

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 03:01:23 +13:00
Jake Barnby ab85942276 fix: add retry and session verification to getRoot() for parallel tests
getRoot() now retries up to 5 times with session verification to handle
race conditions when multiple paratest workers initialize simultaneously.
Previously, if account creation or session creation failed under load,
all subsequent test requests would fail with 401.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 23:19:54 +13:00
ArnabChatterjee20k a76a5cda38 Refactor code structure for improved readability and maintainability 2026-02-27 17:15:00 +05:30
ArnabChatterjee20k 905763e755 added tests 2026-02-25 16:08:40 +05:30
Jake Barnby c91252d94c Switch types 2026-02-25 21:42:28 +13:00
Jake Barnby e1791e17c3 fix: increase attribute/index polling timeouts to 240s for shared mode CI
Shared tables mode experiences significant worker queue contention during
parallel test execution. 120s was insufficient for attribute processing
in Shared V1 mode and occasionally for PostgreSQL under load.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 17:50:46 +13:00
Jake Barnby 7df86a3838 fix: cast attribute defaults after queue deserialization and improve test robustness
- Fix float/int/bool attribute creation failures caused by type loss
  during queue serialization in the Databases worker
- Rework session limit test to retry session creation for cache
  propagation in shared mode
- Increase GraphQL attribute polling timeouts from 30s to 60s
- Increase SchemaPolling waitForAllAttributes timeout to 180s
- Increase Realtime WebSocket client timeout from 30s to 45s

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 16:02:23 +13:00
Jake Barnby 07dc088e4e fix: improve E2E test probes and enable function webhook events
- Fix testPasswordRecoveryUrlParams: use URL-based probes instead of
  userId-based probes to distinguish between test cases that share the
  same user
- Enable functions.* webhook events in ProjectCustom for function
  webhook tests that were silently passing by matching stale events
- Fix setupTeamMembership: add email address probe to getLastEmail to
  prevent picking up wrong invitation email
- Fix getLastEmail multi-email race: assert count inside assertEventually
  when requesting multiple emails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 15:15:59 +13:00
Jake Barnby 5cb36a0a3a fix: resolve E2E test race conditions in parallel execution
- Add probe callback support to getLastEmail(), getLastEmailByAddress(),
  and getLastRequest() to filter results by content before accepting
- Fix variable name typo in TeamsCustomClientTest ($email vs $lastEmail)
- Add event probes to all 56 webhook test getLastRequest() calls to
  filter by specific event pattern (resource ID + action)
- Add email probes to Account OTP/recovery/magic-url tests to wait for
  the correct email (security phrase, Password Reset subject, etc.)
- Add email probes to Projects tests for recovery email URL matching
- Increase FunctionsSchedule future time from 1min to 2min to avoid
  timing issues when seconds are zeroed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 14:43:15 +13:00
Jake Barnby bada1a7c05 Timing updates 2026-02-24 14:00:07 +13:00
Jake Barnby f2759cb65a fix: remove file caching, disable --functional for Databases/Functions/Realtime
Services that depend on shared static state between test methods
(Databases, Functions, Realtime) now run without --functional flag,
so test methods execute sequentially within each class while classes
still run in parallel. All other services keep --functional mode.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 03:44:28 +13:00
Jake Barnby a0b8437f08 fix: scope file caching to Database tests only, revert for non-Database tests
The previous file caching approach cached getRoot(), getUser(), getProject(),
and getConsoleVariables() globally. This caused all test methods in a class
to share the same project, breaking non-Database tests that expect isolated
state (Account 401s, Storage 500s, Users 404s, etc.).

Now file caching is only applied in Database/Transaction test setup chains:
- ensureSharedProject() in DatabasesBase, TransactionsBase, TransactionPermissionsBase
  creates and file-caches both the project AND user so all methods share
  consistent project + user state (needed for collection permissions)
- Non-Database tests (Account, Storage, Users, etc.) create their own
  isolated projects per-process as before

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 02:46:56 +13:00
Jake Barnby 3ea1e8e59f fix: add file-based cross-process caching for ParaTest --functional mode
With --functional mode, each test method runs in its own process so
static caches are empty. This causes every method to recreate projects,
databases, collections, attributes, and indexes - flooding the serial
database worker queue and causing attribute polling timeouts.

File-based caching with file locks ensures resources are created once
per test class, then shared across all method processes. This restores
--functional mode and dramatically reduces worker queue load.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 01:47:20 +13:00
Jake Barnby b976454416 fix: add early failure detection and increase polling timeouts
Throw Critical exception when attribute/index/deployment status is
'failed' to fail tests immediately instead of waiting 15-20 minutes.
Increase default schema polling timeouts from 15 to 20 minutes for
CI stability under parallel load.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 19:51:59 +13:00
Jake Barnby 89f0f227d6 fix: resolve CI failures in database, functions, and realtime tests
- Increase SchemaPolling default timeouts from 8min to 15min for
  waitForAttribute, waitForAttributes, waitForAttributeCount, and
  waitForIndex to match waitForAllAttributes/waitForAllIndexes. Under
  parallel test load the database worker gets backlogged, causing
  attributes to stay in 'processing' state longer than 8 minutes.
- Add isEmpty check in Functions Delete before updating schedules to
  prevent 500 errors when the schedule document is missing (consistent
  with other modules like Executions/Delete).
- Add retry logic to cleanupFunction in FunctionsBase for transient
  MongoDB transaction abort errors during function deletion.
- Increase Realtime attribute/index polling timeouts from 2min to 15min
  to handle worker backlog under CI load.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 17:22:51 +13:00
Jake Barnby c1cb7bdc8e fix: increase schema polling timeouts to 15 minutes
Spatial attribute creation under parallel CI load consistently exceeds
10 minutes. Increased both waitForAllAttributes and waitForAllIndexes
from 10/8 minutes to 15 minutes to accommodate heavy parallel load.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 05:12:09 +13:00
Jake Barnby 54505c0600 fix: increase polling timeouts for CI stability
Spatial attribute creation under parallel MariaDB load and function
deployment builds under MongoDB can exceed previous limits. Increased
SchemaPolling timeout from 8 to 10 minutes and GraphQL function build
timeout from 2 to 4 minutes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 04:21:49 +13:00
Jake Barnby ddb392caf7 fix: increase SchemaPolling default timeouts from 4min to 8min
MariaDB attribute/index processing consistently times out under parallel
test load in CI with different tests failing each run. Increase all default
polling timeouts from 240s to 480s to handle worst-case MariaDB load.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 23:18:56 +13:00
Jake Barnby 3e0febca92 Revert "fix: increase SchemaPolling default timeouts from 4min to 6min"
This reverts commit 2a4c33be11.
2026-02-18 22:49:31 +13:00
Jake Barnby 2a4c33be11 fix: increase SchemaPolling default timeouts from 4min to 6min
MariaDB attribute/index processing consistently times out under parallel
test load in CI. Increase all default polling timeouts from 240s to 360s.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 22:39:07 +13:00
Jake Barnby ca01ec7b6b fix: increase SchemaPolling default timeout to 240s for CI stability
120s was insufficient for attribute processing under parallel CI load,
causing spurious timeout failures in spatial, transaction, and attribute tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 16:28:24 +13:00
Jake Barnby 84c40cea41 fix: resolve testCreateAttributes, testUpsertDocument, and SchemaPolling timeouts
- Fix testCreateAttributes: use $actorsAttr instead of undefined $actors variable,
  and use $this->getSchemaResource() instead of hardcoded 'attributes' for TablesDB compat
- Fix testUpsertDocument: add missing 'data' => $upsertData in PUT request body
- Increase SchemaPolling default timeout from 60s to 120s for CI stability with
  spatial attributes under parallel load

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 15:25:45 +13:00
Jake Barnby acad17f3b8 fix: resolve test failures and reduce excessive timeouts
- DatabasesBase: fix 5 merge corruption issues (missing integers2 attr,
  undefined vars, wrong assertion, wrong setup method)
- WebhooksBase: increase deployment build timeout from 10s to 120s
- RealtimeBase: reduce WebSocket timeout 120s→30s, fix deprecated nullable
- RealtimeConsoleClientTest: reduce assertEventually 120s→30s
- SchemaPolling: reduce all default timeouts 600s→60s
- SitesBase: reduce build timeout 300s→120s, activation 200s→60s

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 14:23:42 +13:00