Commit Graph

812 Commits

Author SHA1 Message Date
Jake Barnby a4d1043684 Merge pull request #12302 from appwrite/fix-transaction-deletes
fix: match transactions collection name in deletes worker
2026-05-14 17:48:18 +12:00
Jake Barnby 10eae0ab87 fix: match transactions collection name in deletes worker
The DELETE_TYPE_TRANSACTION constant was 'transaction' (singular), so
the worker's inner switch on $document->getCollection() (which returns
'transactions') never matched and every transaction deletion fell to
the default branch, logging "No lazy delete operation available for
document of type: transactions". Renamed to DELETE_TYPE_TRANSACTIONS
to align with every other constant used in that switch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 13:58:40 +12:00
Chirag Aggarwal 85e2cf7d8b (refactor): Migrate queueForFunctions to FunctionPublisher and FunctionMessage 2026-05-13 18:28:05 +05:30
loks0n a0c4b7695e chore: normalize Span keys and centralize span lifecycle
Span attribute keys are now snake_case with dots only for child
relationships. Worker span lifecycle moved to app/worker.php; selective
trace filtering moved to the exporter sampler in app/init/span.php so
handlers only call Span::add.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 10:29:42 +01:00
Jake Barnby be2bb984c5 Merge pull request #12194 from appwrite/feat-insights-module
feat(insights): add insights module with CTA framework
2026-05-13 17:34:56 +12:00
Jake Barnby b058eb31f4 (refactor): inline CTAs as JSON on insights collection 2026-05-13 17:10:33 +12:00
Luke B. Silver 35941ddc34 Merge pull request #12220 from appwrite/feat/utopia-http-resources-context
Migrate to utopia-php/http resources/context API
2026-05-12 23:01:29 +01:00
Jake Barnby 304b0dab35 (fix): address greptile P1 review comments
- Add skipFilters to Reports/Get.php (was the only endpoint still
  triggering the full N+1 subquery cascade)
- Scale CTA batch limit dynamically (insightCount * MAX_CTA_COUNT)
  instead of fixed APP_LIMIT_SUBQUERY to prevent silent truncation
- Revert deleteReport to callback-based pagination so CTAs are not
  orphaned when a report has more than APP_LIMIT_SUBQUERY insights
- Add explicit prefix lengths (700) to _key_project_resource and
  _key_project_parent_resource indexes to stay under InnoDB 3072-byte limit
- Validate CTA service/method against ADVISOR_CTA_SERVICES and
  ADVISOR_CTA_METHODS enums in the CTAs validator

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-13 01:42:59 +12:00
Jake Barnby 9da4a3260d (refactor): improve advisor module perf, security, and maintainability
- Fix N+1 in Reports/XList (51→4 queries) via skipFilters + batch fetch
- Add skipFilters to Reports/Delete and cursor fetch (avoid loading all
  nested insights/CTAs just for ownership check)
- Fix N+1 in deleteReport worker (flat CTA deletion instead of per-insight)
- Add advisor entity cleanup on project deletion (reports, insights, CTAs)
- Remove resourceInternalId, parentResourceInternalId, $permissions from
  Insight response model (internal IDs leak DB internals, permissions unused)
- Remove dead subQueryInsightCTAs filter registration
- Remove stale enum-value comments from platform schema
- Fix _key_dismissedAt index to include projectInternalId
- Fix scope category from 'Other' to 'Advisor'
- Switch action base class from Utopia\Platform\Action to Appwrite\Platform\Action

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-13 01:13:37 +12:00
Jake Barnby f9ff1166b9 (perf): batch-fetch CTAs across an insight set to collapse N+1 2026-05-12 23:24:26 +12:00
Jake Barnby 4cd35e4d93 Merge branch '1.9.x' into feat-insights-module 2026-05-12 22:59:34 +12:00
Matej Bačo 5e1439f64e Merge pull request #12276 from appwrite/feat-rework-project-response-model
Feat: Rework project response model
2026-05-12 12:57:16 +02:00
loks0n 4bdcdb6f96 Migrate to utopia-php/http resources/context API
Adopts the new split DI containers in utopia-php/http: `resources()` for
boot-time wiring (shared across requests) and `context()` for per-request
state. Replaces the removed `getResource()`/`setResource()`/`getContainer()`
helpers throughout the HTTP entry point, controllers, GraphQL layer, and
installer.

Bumps the dependency chain accordingly: utopia-php/http to the dev branch
(aliased to 0.34.25 to satisfy platform's exact pin), servers 0.4.*,
queue 0.18.*, and pulled-along cli/platform/database upgrades.

Also tightens app/init/resources/request.php by collapsing single-return
factories to arrow functions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 11:41:42 +01:00
copilot-swe-agent[bot] 63df500b6f refactor(constants): remove Advisor alias prefixes for insight/report enums
Agent-Logs-Url: https://github.com/appwrite/appwrite/sessions/55c56d91-4db4-4501-b5cb-42159dbc9778

Co-authored-by: abnegate <5857008+abnegate@users.noreply.github.com>
2026-05-12 08:09:18 +00:00
Jake Barnby 53d18a0d08 refactor(advisor): rename module to Advisor (service-named, top-level)
The module's namespace and directory now match the top-level service
name (`advisor`) instead of one of its resource names (`insights`):

- src/Appwrite/Platform/Modules/Insights -> .../Modules/Advisor
- src/Appwrite/Insights -> src/Appwrite/Advisor
- tests/unit/Insights -> tests/unit/Advisor
- Route group label flipped from `'insights'` to `'advisor'`
- Section-header comments aligned

Resource names (`insights`, `reports`, `insightCTAs`) and the
`Insight*`/`Report` response models stay — those are the resources the
service exposes, not the service itself.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 23:40:06 +12:00
Matej Bačo 2da400d5b4 Bump version to introduce response filter (for project get) 2026-05-11 12:53:01 +02:00
Matej Bačo 443ebac71c Finish project response model 2026-05-11 11:56:22 +02:00
Jake Barnby db4cb5361d fix(databases): fall back to project DSN when database attr is empty
`getDatabasesDB` used `??` to fall back from a database doc's `database`
attribute to the project DSN, but `??` only triggers on null. Migration
destinations end up with an empty-string `database` (the value is copied
from the source DB but isn't a valid DSN on the destination's pool),
which slipped past the fallback and surfaced as a 500 with
`new DSN('mysql://')` in the catch block. Use elvis so empty strings
fall back too.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 19:12:48 +12:00
copilot-swe-agent[bot] 49bf1198b2 refactor(advisor): polish constants aliases and insight lookup helper
Agent-Logs-Url: https://github.com/appwrite/appwrite/sessions/517a4586-d3e8-40b3-a3a9-f2d2ca82b0a0

Co-authored-by: abnegate <5857008+abnegate@users.noreply.github.com>
2026-05-11 02:08:29 +00:00
copilot-swe-agent[bot] bfd6cebbb0 fix(advisor): address review comments on tests, naming, docs, and get insight
Agent-Logs-Url: https://github.com/appwrite/appwrite/sessions/517a4586-d3e8-40b3-a3a9-f2d2ca82b0a0

Co-authored-by: abnegate <5857008+abnegate@users.noreply.github.com>
2026-05-11 01:59:31 +00:00
Jake Barnby 8fa9ead279 chore: merge 1.9.x into feat-insights-module
Resolves conflict in app/init/constants.php (kept 4327 cache buster, took 1.9.4 stable version).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 17:59:01 +12:00
Jake Barnby 83c0a32494 fix(advisor): make reports.target a TEXT column
target stores free-form URLs or resource IDs. URLs in the wild can
exceed the prior 2048-char cap, so switch the column to VAR_TEXT
(65535). The _key_project_target index already declares an explicit
700-char prefix length, so indexing still works on both MariaDB and
MongoDB.

Bump APP_CACHE_BUSTER for the schema change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 12:42:29 +12:00
Jake Barnby b830c08257 fix(advisor): use VAR_TEXT for free-form fields
Switch markdown summaries, JSON CTA params, and the embedded
insights/ctas subQuery payloads from sized VAR_STRING to VAR_TEXT.
None of these fields are indexed and they all hold free-form data, so
removing the artificial size cap means we never have to revisit the
limit when an analyzer surfaces a longer report.

- reports.summary, insights.summary: markdown, no upper bound enforced
- reports.insights, insights.ctas: subQuery virtuals carrying full
  child rows
- insightCTAs.params: JSON parameter blob for arbitrary CTA actions

Bump APP_CACHE_BUSTER for the schema change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 12:30:45 +12:00
Jake Barnby b747c97561 fix(advisor): address review comments on scopes and schema
- Register dedicated reports.write scope and switch deleteReport to it
  so cloud can issue narrowly-scoped delete keys without granting
  insights.write.
- Make insights.parentResourceInternalId optional with null default to
  match its companion parentResourceType/parentResourceId fields and
  unblock insights with no parent (e.g. database-level performance
  insights).
- Tighten Insight.reportId model description: insights always belong to
  a report, ad-hoc insights are not supported.
- Add reports.write to default test API key and admin role so existing
  e2e tests using serverHeaders() can hit the delete endpoint.
- Bump APP_CACHE_BUSTER for the schema change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 12:18:39 +12:00
Chirag Aggarwal 34075322d7 Migrate mails and messaging queues to publishers 2026-05-08 14:32:11 +05:30
copilot-swe-agent[bot] 56b9de097f fix(advisor): restore report delete with async nested cleanup
Agent-Logs-Url: https://github.com/appwrite/appwrite/sessions/6496395d-5cbf-42ba-a3e0-d3c9e6ca901c

Co-authored-by: abnegate <5857008+abnegate@users.noreply.github.com>
2026-05-08 06:54:49 +00:00
copilot-swe-agent[bot] 6d0eab2583 refactor(advisor): make insights API read-only in CE
Agent-Logs-Url: https://github.com/appwrite/appwrite/sessions/8d7897b5-ac68-487d-954a-be717380bf66

Co-authored-by: abnegate <5857008+abnegate@users.noreply.github.com>
2026-05-08 06:07:23 +00:00
Chirag Aggarwal 8ebf4fee46 Use storage cache callback max age 2026-05-08 11:18:55 +05:30
Chirag Aggarwal 9337daf7ac Preserve default storage cache hit max age 2026-05-08 11:15:17 +05:30
Chirag Aggarwal bf76736971 Split storage cache control context 2026-05-08 09:57:33 +05:30
Chirag Aggarwal 6abd88b8f1 Type storage cache control config 2026-05-08 09:44:00 +05:30
Chirag Aggarwal 7b5a5b8d19 Rename storage cache control hook 2026-05-08 08:44:35 +05:30
Chirag Aggarwal 20ab5b026e Add cached response cache-control callback 2026-05-08 08:33:37 +05:30
loks0n 7d0843cc7f Migrate queueForBuilds to publisherForBuilds pattern
Replaces the stateful Appwrite\Event\Build queue class with a stateless
BuildPublisher and BuildMessage DTO, matching the publisher pattern used
by audits, certificates, executions, migrations, screenshots, stats, and
usage. Call sites now enqueue messages directly instead of mutating a
shared event object and relying on the API shutdown hook.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 13:23:21 +01:00
Matej Bačo ec927104fe Merge pull request #12221 from appwrite/fix-variables-api-feature-parity
Feat: Variables API feature parity
2026-05-06 11:27:32 +02:00
Jake Barnby 6c6782dd28 Merge remote-tracking branch 'origin/1.9.x' into pr-12194-feat-insights-module
# Conflicts:
#	app/config/scopes/project.php
2026-05-06 20:33:40 +12:00
Matej Bačo 1ccf058c14 Fix failing tests 2026-05-06 10:19:30 +02:00
Jake Barnby c5dfc42a60 refactor(insights): rename ctas collection to insightCTAs
Disambiguate the platform-level collection name. Field/request-param
remains `ctas` (the embedded array on the insight response).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 18:12:17 +12:00
Jake Barnby 5404bfec75 refactor(insights): promote CTAs to own collection with backref
Embedding CTAs as a 16384-byte JSON blob on `insights` was the wrong
shape — they're real documents with their own lifecycle. Move them out.

Schema:
- New platform `ctas` collection. Each row carries `projectInternalId`,
  `projectId`, `insightInternalId`, `insightId` (backref), plus the
  CTA fields: `key`, `label`, `service`, `method`, `params`.
- Indexes: `(projectInternalId, insightInternalId)` for the subquery
  lookup and a UNIQUE `(insightInternalId, key)` so the per-insight
  uniqueness invariant lives at the DB layer (not just in PHP).
- The `ctas` field on `insights` becomes a virtual attribute backed by
  a new `subQueryInsightCTAs` filter that joins child docs at read
  time. Consumers still get CTAs embedded on the insight response —
  one round-trip from their perspective.
- The CTA descriptor's within-insight identifier renamed `id` → `key`
  (clashed with the document `$id`). Validator updated.

Endpoints:
- Manager Create now persists CTAs as separate `ctas` documents after
  the parent insight, then re-fetches the insight so the response
  carries the freshly-joined CTA list.
- User Update trimmed to user-controlled state only (`severity`,
  `status`). `title`, `summary`, `payload`, `ctas`, and `analyzedAt`
  are analyzer-controlled — analyzers re-ingest by deleting and
  POSTing again to the manager endpoint.
- Insight Delete cascades to CTAs.
- Report Delete cascades through Insights → CTAs.

Response model:
- InsightCTA gains the standard document headers (`$id`,
  `$createdAt`, `$updatedAt`) and an `insightId` backref. The
  caller-supplied identifier is now `key`.

Tests:
- E2E sampleCTA factory uses `key` everywhere; testCreate asserts the
  freshly-created CTA carries `$id`, `$createdAt`, `insightId`, and
  the right shape.
- Dropped the testUpdate*CTA* tests — user Update no longer accepts
  CTAs. testDismissViaUpdate now depends on testUpdate directly.
- Unit tests rewritten to validate `key` instead of `id`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 18:07:49 +12:00
Jake Barnby a1f64c6f71 refactor(insights): split CTA action into service + method
- InsightCTA model now exposes `service` (SDK namespace) and `method`
  (function name) instead of a single `action` string. Drops the
  string-splitting burden on every consumer and lets the console reach
  the right SDK method directly.

- Validator requires both `service` and `method` non-empty; same
  16-entry max still enforced.

- Endpoint normalization (Create + Update) splits the new shape into
  the persisted CTA descriptor.

- Constants split: INSIGHT_CTA_SERVICE_* (databases / tablesDB /
  documentsDB / vectorsDB) and INSIGHT_CTA_METHOD_* (createIndex).

- Insight model + InsightCTA model docs updated with the new field
  semantics and per-engine examples.

- E2E factory `sampleCTA($id, $engine)` emits the correct service and
  engine-appropriate params keys (tableId/columns for tablesDB;
  collectionId/attributes everywhere else). Engine matrix asserts
  `service` and `method` independently.

- Added e2e + unit coverage for the new failure modes (missing
  service, missing method, empty service, empty method).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 16:35:33 +12:00
Jake Barnby 8f79379b6e test(insights): full e2e + per-engine CTA action mapping
- Restructure InsightsBase trait with explicit helper methods
  (createInsight/getInsight/listInsights/updateInsight/deleteInsight,
  createReport/getReport/listReports/updateReport/deleteReport, plus
  sampleInsight/sampleCTA factories) — same shape ProxyBase uses.

- Add coverage for: report CRUD + duplicate-id rejection, invalid type
  rejection, list filtering by all allowed attributes, cursor
  pagination + missing-cursor, update preserving untouched fields,
  CTA validation edge cases (duplicate ids, empty fields, count > 16),
  dismissal round-trip + status filter, report cascade delete,
  unauthorized access (no server key), empty-result list.

- Engine-specific insight types (tablesDBIndex, documentsDBIndex,
  vectorsDBIndex, plus the legacy databaseIndex) so the CTA's `action`
  can map to the matching public API: databases.createIndex,
  tablesDB.createIndex, documentsDB.createIndex,
  vectorsDB.createIndex. dataProvider drives the engine matrix and
  asserts the right action lands in the persisted CTA. Constants for
  each action name live in app/init/constants.php.

- InsightCTA model docs spell out which action belongs to which engine
  and that the params keys differ between APIs (tableId/columns for
  tablesDB vs collectionId/attributes for the legacy / DocumentsDB /
  VectorsDB APIs).

- Insight model `type` description now lists every engine variant.

- CTAsTest gains coverage for object-shaped params, empty-action and
  empty-label rejection, and the default 16-entry cap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 16:25:49 +12:00
Jake Barnby 00565ea471 refactor(insights): metadata-only CTAs, platform DB, reports parent
Address review feedback on PR #12194:

- Pivot CTAs to pure descriptors (id/label/action/params). Drop the
  server-side execution layer: Action interface, registry, the
  databases.indexes.create CTA action, the params validator, the
  /v1/insights/:id/ctas/:id/executions endpoint, the InsightCTAExecution
  model, the INSIGHT_CTA_* errors, and the corresponding events. The
  console invokes the existing public API directly with the descriptor's
  action + params.

- Restore Databases\Indexes\Action.php to its pre-CTA shape and inline
  the index-create body back into Create.php (the createIndex helper
  was added solely for CTA reuse).

- Move insights collection from project DB to platform DB and add a
  parent reports collection alongside it. Insights carry projectId /
  projectInternalId for tenant scoping and an optional reportId for
  grouping. List endpoints filter by projectInternalId; Get/Update/
  Delete also enforce project ownership before touching the document.

- New Reports module with full CRUD (Create/Get/XList/Update/Delete),
  Report response model, Reports query validator, REPORT_NOT_FOUND /
  REPORT_ALREADY_EXISTS errors, reports.read / reports.write scopes,
  and reports.* event tree. Delete cascades to child insights.

- Update.php now mutates the loaded document via setAttribute (instead
  of passing a partial new Document), reuses CTAsValidator (instead of
  the looser ArrayList<JSON> + isset check), and rejects duplicate CTA
  ids.

- Create.php enforces unique CTA ids during normalization.

- CTAsValidator gained a configurable maxCount (default 16) so the
  Create path matches the Update path and the DB column size, and
  oversized payloads return a clean 400.

- Validator\Queries\Insights adds status and reportId to
  ALLOWED_ATTRIBUTES so dismissal / report workflows are filterable.

- Realtime channel parser guards $parts[1] for both insights and
  reports event names.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 13:46:07 +12:00
Matej Bačo 305e8e1ec7 Add request filter 2026-05-05 16:24:41 +02:00
ArnabChatterjee20k bca876acaf Merge remote-tracking branch 'origin/1.9.x' into big-int 2026-05-05 18:55:11 +05:30
Jake Barnby 69c637c72d style(insights): use aliased import for DatabasesIndexesCreate registry entry
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 15:01:31 +12:00
Jake Barnby 242c7f75ad refactor(insights): nest databases create-index CTA under resource path
Move the action class from Action/DatabasesCreateIndex.php to
Action/Databases/Indexes/Create.php so the directory mirrors the
underlying resource hierarchy. Action name follows: databases.createIndex
becomes databases.indexes.create, with the constant renamed to
INSIGHT_CTA_ACTION_DATABASES_INDEXES_CREATE.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 14:57:28 +12:00
Jake Barnby f779c7aa3b refactor(insights): collapse dismissal into a status field on update
Dismissal was a sub-resource (POST /v1/insights/:id/dismissals) but a
dismissal is just a state transition, not a thing the client creates.
Drop the dedicated endpoint and add a `status` enum (`active` |
`dismissed`) to the insights schema, settable via the existing PATCH
update route. The server still derives `dismissedAt` and `dismissedBy`
on transition for audit/sorting, but the client-facing API is just a
single status toggle.

- Schema: add `status` attribute (default `active`)
- Constants: add `INSIGHT_STATUSES`
- Update endpoint: accept `status` param, derive dismissedAt/By on
  active <-> dismissed transitions
- Response model: add `status` rule
- Drop Insights/Dismissal/Create.php, the createInsightDismissal SDK
  method, the `insights.[id].dismissals.create` event, and the
  `insight.dismissal.create` audit
- E2E: replace testCreateDismissal with testDismissViaUpdate covering
  both directions of the toggle

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 14:36:31 +12:00
Jake Barnby cd539d972a refactor(insights): capitalise CTA acronym in identifiers
Project-specific override of the default camelCase-acronyms convention:
namespaces, class names, file paths, and SDK method names use `CTA` in
all caps. Touches all insights surfaces — directories, response models,
validators, container resource keys, and SDK method names like
`createInsightCTAExecution`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 14:27:45 +12:00
Jake Barnby d188bd7a2e refactor(insights): register CTA registry on the global $register
Persist the insight CTA registry across requests by attaching it to the
boot-time global $register, mirroring the pattern used for `geodb`,
`passwordsDictionary`, `hooks`, etc. The container resource now looks
the registry up from `$register` instead of rebuilding it on every
request.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 14:22:10 +12:00
Jake Barnby da5a137b98 refactor(insights): nest CTA execution and dismissal as sub-resources
Move POST /v1/insights/:id/dismiss to /v1/insights/:id/dismissals
and POST /v1/insights/:id/ctas/:ctaId/trigger to
/v1/insights/:id/ctas/:ctaId/executions, with the corresponding
class moves into Http/Insights/Dismissal/Create.php and
Http/CTA/Execution/Create.php. Rename the response model to
InsightCtaExecution and update events.php to surface dismissal
and execution as resource events with create verbs. The reshape
matches the rest of the API where verbs hang off plural sub-resources.
2026-05-01 14:04:31 +12:00