Compare commits

..
Author SHA1 Message Date
Chirag Aggarwal 601e5a819f strip 500 error html tags in tests 2026-03-16 11:52:41 +05:30
Chirag Aggarwal b65549ac29 chore: update database lib 2026-03-16 11:42:14 +05:30
Luke B. SilverandGitHub 5a56a2fc35 Merge pull request #11551 from appwrite/ci/better-checks
ci: migrate checks to use setup-php and setup-node
2026-03-15 18:59:35 +00:00
loks0nandClaude Sonnet 4.6 5e3f698b57 chore: remove HeadersBench entries from phpstan baseline
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:51:07 +00:00
loks0nandClaude Sonnet 4.6 a86217bf4b chore: remove phpbench dependency and benchmark files
phpbench pulled in doctrine/annotations which has a security advisory.
The bench script and its only benchmark file are unused in CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:48:24 +00:00
loks0nandClaude Sonnet 4.6 4863b29754 ci: install dependencies before composer audit
composer audit requires packages to be installed to check them.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:43:51 +00:00
loks0nandClaude Sonnet 4.6 e0a2b5c1ef ci: unset COMPOSER_NO_AUDIT for audit step
setup-php sets COMPOSER_NO_AUDIT=1 by default, which causes composer
audit to skip. Override it to 0 for the audit step.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:42:36 +00:00
loks0nandClaude Sonnet 4.6 bc2a0d573c ci: resolve merge conflict with 1.8.x
Use composer analyze (renamed from composer check in 1.8.x).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:40:38 +00:00
loks0nandClaude Sonnet 4.6 e1bb4fc2cc ci: migrate checks to use setup-php and setup-node instead of Docker
Replace docker-based composer/node invocations in the format and analyze
jobs with shivammathur/setup-php@v2 and actions/setup-node@v4. Split
locale check and composer validate/audit into dedicated jobs for clarity.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:38:50 +00:00
Luke B. SilverandGitHub 79f907ff55 Merge pull request #11550 from appwrite/ci/phpstan-baseline
ci: upgrade PHPStan to v2 with baseline and full codebase coverage
2026-03-15 18:12:52 +00:00
loks0nandClaude Sonnet 4.6 937d94972f ci: upgrade PHPStan to v2 with baseline and full codebase coverage
- Upgrade phpstan/phpstan from 1.12.* to ^2.0
- Raise analysis level to max
- Expand paths from 3 specific dirs to src/, app/, bin/, tests/
- Generate baseline capturing 95,365 existing errors for incremental adoption
- Rename composer script from `check` to `analyze`
- Add --memory-limit=1G to handle larger scan scope

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 17:49:49 +00:00
Luke B. SilverandGitHub aefa51f261 Merge pull request #11539 from appwrite/chore-speedup-tests
Chore: Speed-up deployments in tests
2026-03-15 17:16:26 +00:00
loks0nandClaude Sonnet 4.6 f9ecd28dd0 Revert composer.lock to 1.8.x
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 16:57:39 +00:00
Matej Bačoandloks0n 7e6db9a99b Fix tests missing lock files 2026-03-15 16:39:12 +00:00
Matej Bačoandloks0n c618372d37 linter fix 2026-03-15 16:39:12 +00:00
Matej Bačoandloks0n 975a38bf43 Speed-up deployments in tests 2026-03-15 16:39:12 +00:00
Luke B. SilverandGitHub 6b21968413 Merge pull request #11549 from appwrite/test/split-databases
Split TablesDB e2e tests into a separate CI job
2026-03-15 16:36:14 +00:00
loks0nandClaude Opus 4.6 286e0e5802 Fix import ordering in TablesDB test files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 16:25:05 +00:00
loks0nandClaude Opus 4.6 ac05c9e9ce Split TablesDB e2e tests into a separate CI job
Move all TablesDB* test files from tests/e2e/Services/Databases/ to a
new tests/e2e/Services/TablesDB/ directory, updating namespaces and
adding explicit imports for shared base traits. Add TablesDB as a
separate service in the CI matrix so /v1/databases and /v1/tables tests
run as independent parallel jobs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-15 16:22:01 +00:00
Luke B. SilverandGitHub 153e660274 Merge pull request #11543 from appwrite/fix/ci-before-main-2
Fix devalue vulnerability in astro test sites
2026-03-14 20:13:46 +00:00
loks0nandClaude Opus 4.6 ad97feaf6e Optimize CI matrix job to fetch composer.lock directly
Replace paginated listFiles API call with targeted getContent calls
to avoid timeouts on large PRs with thousands of changed files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 19:25:40 +00:00
loks0nandClaude Opus 4.6 ef618b831d Fix devalue vulnerability by updating from 5.6.3 to 5.6.4
Resolves GHSA-cfw5-2vxh-hr84 (CVSS 6.3) and GHSA-mwv9-gp5h-frr4
(CVSS 2.7) in astro test resource sites.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 19:12:10 +00:00
Luke B. SilverandGitHub 021c5fcad6 Merge pull request #11449 from appwrite/refactor/usage-publishing
Refactor usage metrics to stateless publisher pattern
2026-03-13 16:36:04 +00:00
Luke B. SilverandGitHub 4309bd075e Merge pull request #11538 from appwrite/chore/pin-servers
Pin utopia-php/servers to 0.2.5
2026-03-13 14:00:43 +00:00
loks0nandClaude Opus 4.6 236dceae2c Pin utopia-php/servers to 0.2.5
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:59:06 +00:00
Matej BačoandGitHub 884b8f10bb Merge pull request #11536 from appwrite/fix-staging-project-creation
Fix: staging env
2026-03-13 12:57:36 +01:00
Matej Bačo 530d681c48 Fix staging env 2026-03-13 11:25:17 +01:00
loks0nandClaude Opus 4.6 0295e27b5d Fix build usage metrics to use actual memory/cpus and fix escapeshellarg double-quoting
Use $memory (which includes minMemory floor) and $cpus instead of raw
spec values in MB-seconds metrics, fixing underreporting for sites and
frameworks bumped to the minimum memory. Also remove redundant double
quotes around escapeshellarg() calls in mv command.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:42:41 +00:00
loks0nandClaude Opus 4.6 a804cba999 Refactor usage metrics to stateless publisher pattern
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 15:36:16 +00:00
Luke B. SilverandGitHub d8f733787f Merge pull request #11531 from appwrite/ci/single-pr-workflow
Consolidate PR workflows into single CI workflow
2026-03-12 14:17:23 +00:00
loks0nandClaude Opus 4.6 1abbca9318 Split SARIF uploads with unique categories to fix codeql-action error
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:42:28 +00:00
loks0nandClaude Opus 4.6 e99f682cd6 Update trivy-action to v0.35.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:27:38 +00:00
loks0nandClaude Opus 4.6 26326d05e9 Guard SARIF upload against missing files from failed Trivy scans
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:04:33 +00:00
loks0nandClaude Opus 4.6 e67ed2660a Add actions: read permission for osv-scanner reusable workflow
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 13:00:44 +00:00
loks0nandClaude Opus 4.6 8d0a4d7f92 Consolidate remaining PR workflows and simplify Trivy scan
- Move check-dependencies into ci.yml as Checks / Dependencies
  (upgrade to osv-scanner-reusable-pr.yml@v2.3.3, drop merge_group)
- Move pr-scan into ci.yml as Checks / Image
  (upgrade Trivy to 0.33.1, use SARIF + upload-sarif instead of
  custom PR comment logic)
- Rename Setup job to Build
- Fix format job git checkout HEAD^2 to only run on pull_request
- Rename PHPStan step correctly (was mislabeled CodeQL)
- Add Docker Hub login to benchmark job
- Remove no-op pull_request trigger from ai-moderator

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 12:56:41 +00:00
loks0nandClaude Opus 4.6 aecca2f503 Consolidate PR workflows into single CI workflow
Merge linter, static-analysis, tests, and benchmark workflows into ci.yml
with structured job naming (Checks / Format, Tests / E2E / ..., etc.).
Shared Docker image build between tests and benchmark. Update actions to
latest versions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 12:39:14 +00:00
Luke B. SilverandGitHub 7e4ec9eae0 Merge pull request #11529 from appwrite/ci/focused-testing
ci: focused testing
2026-03-12 11:38:27 +00:00
loks0nandClaude Opus 4.6 edd948557e Refactor matrix job to use GitHub API and clean up test config
Replace shell-based database change detection with github-script using
the GitHub API, eliminating the need for a full checkout. Restructure
matrix generation with guard clauses and no mutation. Remove ciIgnore
exclude group from test command.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 11:01:36 +00:00
loks0nandClaude Opus 4.6 09317f290a Clean up database env setup and improve matrix naming
Hardcode lowercase env vars per database branch instead of using tr.
Use proper casing for database matrix values (MongoDB, MariaDB, PostgreSQL).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 08:56:47 +00:00
loks0nandClaude Opus 4.6 e5f0c2df12 Consolidate CI test matrix with dynamic database and mode dimensions
Merge 6 E2E jobs into 3 by combining dedicated/shared mode variants
into a single matrix dimension. Database adapters and table modes
expand dynamically based on whether utopia-php/database changed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 08:56:47 +00:00
Eldad A. FuxandGitHub 2afec80a25 Merge pull request #11530 from appwrite/fix-allow-null-branch
Allow null branch
2026-03-12 09:29:25 +01:00
eldadfux 20c65bac37 Allow null branch 2026-03-12 09:02:16 +01:00
Luke B. SilverandGitHub 2c6f37ecad Merge pull request #11526 from appwrite/perf/test-image-pulls
perf: various ci fixes and optimisations
2026-03-12 00:22:06 +00:00
loks0nandClaude Opus 4.6 f09b9a994e Revert .env defaults back to MongoDB
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 23:59:31 +00:00
loks0nandClaude Opus 4.6 9f4ba3a4a2 Reduce php-retry wait time from 300s to 60s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 23:47:03 +00:00
loks0nandClaude Opus 4.6 8cb36835cc Restore php-retry action for flaky test resilience
Re-add itznotabug/php-retry@v3 wrapping all test steps with
max_attempts: 2 and retry_wait_seconds: 300. Also restore
pull-requests: write permission needed by the action.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 23:41:26 +00:00
loks0nandClaude Opus 4.6 89419344c2 Restore MariaDB defaults, fix remaining timeouts, filter listener span logs
- Change .env defaults back from MongoDB to MariaDB
- Bump all remaining "Load and Start Appwrite" timeouts from 3 to 5 minutes
- Filter listener.* span logs to only export on error

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 23:20:51 +00:00
loks0nandClaude Opus 4.6 7dfe44cb36 Run abuse-enabled tests across entire test suite, not just Projects
The abuseEnabled jobs previously only ran tests in Services/Projects,
missing the Account abuse test and any future abuseEnabled tests in
other services. Also rename jobs to "Abuse" for consistency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:44:42 +00:00
loks0nandClaude Opus 4.6 4cbe50193a Increase Load and Start Appwrite timeout to 5 minutes
The docker compose pull step alone can take over 2.5 minutes on CI
(e.g. openruntimes-executor, traefik), leaving no time for
docker compose up --wait within 3 minutes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:24:52 +00:00
loks0nandClaude Opus 4.6 16929bc420 Remove remaining --debug flags from test commands
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 22:00:00 +00:00
loks0nandClaude Opus 4.6 fd28ad8a66 Remove --debug flag from test commands for quieter CI output
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:59:33 +00:00
loks0nandClaude Opus 4.6 38798c8993 Add healthchecks for MariaDB, PostgreSQL, and Redis
Add proper healthchecks to infrastructure services and use
condition: service_healthy for redis in appwrite's depends_on
so it waits for Redis to be ready before starting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:58:30 +00:00
loks0nandClaude Opus 4.6 33ce469ab0 Add Docker healthcheck and use --wait instead of polling loop
Replace the manual shell polling loop (until doctor > /dev/null) with a
proper Docker healthcheck on the appwrite service and `docker compose up
--wait`, which blocks until healthchecks pass. Also reverts the timeout
back to 3 minutes now that image pulls are cached.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:44:15 +00:00
loks0nandClaude Opus 4.6 0bbcc3f570 Move dev tools to docker-compose.override.yml
Move adminer, redis-insight, mongo-express, and graphql-explorer to an
override file. CI sets COMPOSE_FILE=docker-compose.yml explicitly so
these are excluded from test runs, reducing the number of images to
pull from 14 to 10. Local docker compose auto-loads both files so dev
workflow is unchanged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:06:44 +00:00
loks0nandClaude Opus 4.6 ecca0d8036 Increase Load and Start Appwrite timeout from 3 to 10 minutes
The 3-minute timeout was too tight with the added docker compose pull
step for downloading third-party images.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:48:39 +00:00
loks0nandClaude Opus 4.6 534dc55f17 Remove unnecessary abuse enabled from screenshot tests
The screenshot tests have no abuse-related code. Abuse was only enabled
on these jobs as a side effect of the original sed command applying to
all jobs below e2e_service_test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:35:42 +00:00
loks0nandClaude Opus 4.6 d30df31b82 Move compose-time env vars to docker compose up step
_APP_DATABASE_SHARED_TABLES and _APP_DATABASE_SHARED_TABLES_V1 are
read by the server at boot time via System::getEnv(), not by the test
runner. Passing them via docker compose exec -e had no effect on the
already-running Swoole server. Move them to the Load and Start
Appwrite step so they're set at docker compose up time.

Keep _APP_E2E_RESPONSE_FORMAT on exec since it's read by the test
runner process (tests/e2e/Scopes/Scope.php).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:25:47 +00:00
loks0nandClaude Opus 4.6 dc9a1c03d1 Replace shell exports with GitHub Actions env fields
Use step-level env: fields and GHA expressions for conditional values
instead of shell export statements and if/elif blocks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:20:23 +00:00
loks0nandClaude Opus 4.6 3613a645d0 Remove php-retry action and run test commands directly
Replace itznotabug/php-retry with native run steps and timeout-minutes.
Also remove pull-requests: write permission that was only needed by
php-retry to post PR comments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:19:04 +00:00
loks0nandClaude Opus 4.6 6aac7ea6ad Update GitHub Actions to latest versions
- actions/cache: v4 → v5
- docker/setup-buildx-action: v3 → v4

Fixes Node.js 20 deprecation warnings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:17:03 +00:00
loks0nandClaude Opus 4.6 d22642590f Add build directive to all appwrite-dev services for --ignore-buildable
Add x-build YAML anchor and apply it to all services using the
appwrite-dev image so docker compose pull --ignore-buildable correctly
skips them instead of trying to pull from Docker Hub.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:11:59 +00:00
loks0nandClaude Opus 4.6 0b416e4475 Skip locally-built images during docker compose pull
Add --ignore-buildable flag so docker compose pull skips images with
build directives (appwrite-dev) instead of trying to pull them from
Docker Hub.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:01:25 +00:00
loks0nandClaude Opus 4.6 4e1b710503 Fix Docker Hub login credentials to match repo config
Use `vars.DOCKERHUB_USERNAME` and `secrets.DOCKERHUB_TOKEN` to match
the existing publish and release workflows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:59:23 +00:00
loks0nandClaude Opus 4.6 e557a5fc6e Add Docker Hub login and parallel image pulls to CI
Login to Docker Hub in all test jobs to avoid rate limits, add
`docker compose pull --quiet` to parallelize image downloads before
`docker compose up`, and replace sed-based .env overrides with native
GitHub Actions env fields.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:41:49 +00:00
HemachandarandGitHub 7d15d7e1f5 Rename time-travel task (#11523)
* Rename time-travel task

* rename
2026-03-11 16:15:50 +05:30
Luke B. SilverandGitHub ab1d88a44a Merge pull request #11496 from appwrite/telemetry/worker-mails
Enable SMTP keep-alive in mail worker
2026-03-11 09:43:07 +00:00
Jake BarnbyandGitHub af54ff83dc Merge pull request #11495 from appwrite/feat-message-migration 2026-03-11 21:28:00 +13:00
premtsd-codeandGitHub 4b1c6fdf83 Merge branch '1.8.x' into feat-message-migration 2026-03-11 07:10:21 +00:00
Jake BarnbyandGitHub e42656d51a Merge pull request #11521 from appwrite/fix-duplicated-empty-phones 2026-03-11 19:49:46 +13:00
Prem Palanisamy a56c2c11d3 Merge remote-tracking branch 'origin/1.8.x' into feat-message-migration 2026-03-11 06:00:26 +00:00
Prem Palanisamy 0ebda89adf update utopia-php/database to released 5.3.8 2026-03-11 05:56:44 +00:00
eldadfux 4bf3c72196 fix: allow users to update phone number to empty without causing duplicate errors 2026-03-11 06:25:08 +01:00
Damodar LohaniandGitHub c23945ebc6 Merge pull request #11519 from appwrite/copilot/fix-received-at-timestamp
Update usage to use timestamp of event rather than current timestamp
2026-03-11 07:21:42 +05:45
Steven NguyenandGitHub 0984daf37e Merge branch '1.8.x' into copilot/fix-received-at-timestamp 2026-03-10 15:39:22 -07:00
copilot-swe-agent[bot]andstnguyen90 1113bffb78 fix: use message timestamp for receivedAt in StatsUsage worker
Co-authored-by: stnguyen90 <1477010+stnguyen90@users.noreply.github.com>
2026-03-10 22:19:06 +00:00
copilot-swe-agent[bot] 33cafdfc0e Initial plan 2026-03-10 22:17:26 +00:00
Prem Palanisamy 1be142e4be test: use database fix branch for preserveDates datetime format 2026-03-10 12:33:30 +00:00
loks0nandClaude Opus 4.6 f4b8992cde Enable SMTP keep-alive to reuse connections across mail jobs
Reduces job processing time by avoiding repeated TCP connect, TLS
handshake, and SMTP AUTH on every email sent.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 12:19:18 +00:00
Prem Palanisamy ee2e41fa45 feat: add messaging resource migration support 2026-03-10 09:48:45 +00:00
Jake BarnbyandGitHub 94c6f6e07d Merge pull request #11438 from appwrite/realtime-stats 2026-03-10 21:58:57 +13:00
Luke B. SilverandGitHub f23411f549 Merge pull request #11490 from appwrite/fix-empty-comment
Fix for when vcs comment is empty
2026-03-10 08:51:14 +00:00
Luke B. SilverandGitHub 299778cd50 Merge pull request #11489 from appwrite/feat-namespace-fix
Fix SDK namespace call
2026-03-10 08:33:32 +00:00
ArnabChatterjee20kandGitHub b677586925 Merge branch '1.8.x' into realtime-stats 2026-03-10 13:12:04 +05:30
ArnabChatterjee20k 7f4cba276e updated tests 2026-03-10 12:19:46 +05:30
ArnabChatterjee20k eccc39a466 refactor: remove realtime metrics from project usage endpoints and related tests 2026-03-10 12:15:25 +05:30
Jake BarnbyandGitHub a10595f501 Merge pull request #11488 from appwrite/feat-project-realtime-check 2026-03-10 15:10:11 +13:00
Luke B. SilverandGitHub e42812283b Merge pull request #11492 from appwrite/debug/investigate_bucket_2
fix: health storage errors
2026-03-09 23:07:55 +00:00
loks0nandClaude Opus 4.6 4d7aa6d8ab style: format storage health check braces
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 20:47:07 +00:00
loks0n 95b3db0228 fix: storage health error swallowing 2026-03-09 20:40:34 +00:00
eldadfux a0167d6c6c Fix for when vcs comment is empty 2026-03-09 20:17:32 +01:00
eldadfux 39f3bc7b9d Fix SDK namespace call 2026-03-09 20:08:41 +01:00
eldadfux ab0fa70bca Enhance project context validation in realtime message handling. Ensure that projectId is checked for emptiness before processing messages, and enforce project context requirement for non-ping messages. 2026-03-09 19:31:44 +01:00
Matej BačoandGitHub 74ab41f2b5 Merge pull request #11448 from appwrite/revert-11447-revert-10862-feat-1.8.x-new-schema
Revert "Revert "Features with schema changes""
2026-03-09 17:19:08 +01:00
Matej Bačo d8954f424c Fix specs tests 2026-03-09 15:09:57 +01:00
Matej Bačo 38493094db Fix leftover 2026-03-09 13:31:01 +01:00
Matej Bačo 58e5e69fec Vuln fixes 2026-03-09 13:01:00 +01:00
Matej Bačo 895ec7fd20 security fix 2026-03-09 12:56:51 +01:00
Matej Bačo d3442d86c1 Rework time travel to CLI task 2026-03-09 12:54:12 +01:00
Matej Bačo c20dfc7063 PR simplification 2026-03-09 11:51:32 +01:00
Matej Bačo 591cc70e76 Merge branch '1.8.x' into revert-11447-revert-10862-feat-1.8.x-new-schema 2026-03-09 11:42:09 +01:00
Jake BarnbyandGitHub 2de28cfcb7 Merge pull request #11483 from appwrite/count-buckets 2026-03-09 17:34:07 +13:00
fogelito c6d476b978 catch error 2026-03-08 16:52:22 +02:00
Luke B. SilverandGitHub 5dca3ad875 Merge pull request #11469 from appwrite/fix/blocks-response-code
fix: blocks response code
2026-03-06 20:46:33 +00:00
loks0nandClaude Sonnet 4.5 b6793dc0b5 Fix remaining blocked account test assertions
Update testBlockedAccount and testSelfBlockedAccount to expect 403
instead of 401 for blocked user responses. These were missed in the
previous test assertion update.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-06 17:43:36 +00:00
loks0nandClaude Sonnet 4.5 a2ad25a00a Update test assertions for blocked user status code
Change expected status code from 401 to 403 for USER_BLOCKED errors
to match the semantic change in error codes.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-06 17:30:51 +00:00
loks0nandClaude Sonnet 4.5 16ad05792d Change blocked user/resource errors from 401 to 403
- Update USER_BLOCKED from 401 to 403 and simplify description
- Update GENERAL_RESOURCE_BLOCKED from 401 to 403

Rationale: 403 Forbidden is the correct HTTP status for authorization
failures where the user is authenticated but not permitted access.
401 Unauthorized is for authentication failures.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-03-06 17:30:51 +00:00
Luke B. SilverandGitHub e5f0bc2df6 Merge pull request #11457 from appwrite/fix-migration-errors
Fix migration errors
2026-03-06 17:20:39 +00:00
Chirag AggarwalandGitHub 0d3d14d9e1 Merge pull request #11465 from appwrite/optimize-update-document-calls
perf: optimize updateDocument() calls to use sparse documents
2026-03-06 21:17:31 +05:30
Chirag Aggarwal f2826189c6 fix: remove asserting z does not exist in truncated logs 2026-03-06 18:24:04 +05:30
Chirag Aggarwal 8b026d3459 perf: optimize updateDocument() calls to use sparse documents
Optimize updateDocument() calls across the codebase to pass only changed
attributes as sparse Document objects rather than full documents. This is
more efficient because updateDocument() internally performs array_merge().

Changes:
- Updated 58 files to use sparse Document objects
- Added Performance Patterns section to AGENTS.md with optimization guidelines
- Applied pattern to Workers, Functions, Sites, Teams, VCS modules
- Updated app/controllers/api files (account, users, messaging)
- Updated app infrastructure files (realtime, general, init/resources, shared/api)

Exceptions maintained:
- Migration files (need full document updates by design)
- Cases with 6+ attributes (marginal benefit)
- Complex nested relationship logic
2026-03-06 17:05:19 +05:30
ArnabChatterjee20k 0ea196d21c updated inbound raw size to the request size 2026-03-06 15:37:30 +05:30
ArnabChatterjee20k ece5b81732 updated test 2026-03-06 15:27:30 +05:30
ArnabChatterjee20kandGitHub 2d92e56795 Merge branch '1.8.x' into realtime-stats 2026-03-06 15:16:26 +05:30
Jake Barnby 2306072dda Merge branch 'fix-ce-test-flakes' into 1.8.x 2026-03-06 14:18:37 +13:00
Jake BarnbyandClaude Opus 4.6 91edf82060 fix: database index length assertion and email race in password recovery
- Remove specific index length number from testPatchAttribute assertion
  since the value differs between shared/non-shared table modes (767 vs
  768) and the console API returns the console project's value, not the
  user project's
- Use getLastEmailByAddress in testPasswordRecoveryUrlParams to avoid
  retrieving emails from parallel test classes sharing the same maildev

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 03:41:35 +13:00
Chirag AggarwalandGitHub 3fc50e4d8d Merge pull request #11464 from appwrite/fix/schedule-base-project-access-update
fix: Update in-memory project document after accessedAt update
2026-03-05 20:07:49 +05:30
Chirag Aggarwal 71dff441ed fix: Update in-memory project document after accessedAt update
Previously, the updateProjectAccess method updated the database with the new

accessedAt timestamp but did not update the in-memory project document. This

caused the if statement to constantly evaluate to true on subsequent calls,

triggering unnecessary database updates.
2026-03-05 20:03:42 +05:30
Jake BarnbyandClaude Opus 4.6 65780d75f9 fix: test flakes — correct index length fallback, add retries
- Fix getMaxIndexLength() fallback from 768 to 767 to match MariaDB's
  actual InnoDB index key limit
- Add retry logic to getConsoleVariables() to handle intermittent 401s
- Add retry logic to API key creation in ProjectCustom to prevent
  cascading 401 failures in test methods

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 03:01:23 +13:00
ArnabChatterjee20kandGitHub 12ae61b915 Merge branch '1.8.x' into realtime-stats 2026-03-05 18:40:15 +05:30
ArnabChatterjee20k 03d7bccde2 updated tests 2026-03-05 18:14:16 +05:30
ArnabChatterjee20k b5c2cc9716 Add triggerStats function for realtime usage metrics tracking 2026-03-05 18:10:09 +05:30
Jake BarnbyandClaude Opus 4.6 ee107d30b3 fix: make Projects tests parallel-safe
- Use getLastEmailByAddress for SMTP tests instead of getLastEmail(2) to
  avoid shared mail server state issues under parallel execution
- Add retry logic to setupProject, setupProjectData, and
  setupScheduleProjectData for intermittent 401 errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 01:20:09 +13:00
Shmuel FogelandGitHub 499ca71c28 Merge pull request #11459 from appwrite/stop-on-error
Phpunit stop on error
2026-03-05 14:06:04 +02:00
fogelito cfe60dd2a2 Merge branch '1.8.x' of https://github.com/appwrite/appwrite into stop-on-error 2026-03-05 12:32:33 +02:00
Jake BarnbyandClaude Opus 4.6 ab85942276 fix: add retry and session verification to getRoot() for parallel tests
getRoot() now retries up to 5 times with session verification to handle
race conditions when multiple paratest workers initialize simultaneously.
Previously, if account creation or session creation failed under load,
all subsequent test requests would fail with 401.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 23:19:54 +13:00
Chirag AggarwalandGitHub 0214f22d20 Merge pull request #11461 from appwrite/fix/stale-project-overwrites-oauth
fix: prevent stale project documents from overwriting OAuth providers
2026-03-05 15:46:51 +05:30
Chirag Aggarwal e0269e268f fix: re-read project from DB before updating accessedAt to prevent stale writes
Stale in-memory project documents in ScheduleBase (and request-scoped
copies in api.php/general.php) were overwriting current DB state when
updateProjectAccess triggered. Because Database::updateDocument uses
array_merge with the passed document taking priority, cached projects
missing recent OAuth provider changes would silently disable them.

Now fetches a fresh project document from the DB before writing, so only
accessedAt is updated without clobbering other fields.
2026-03-05 15:44:21 +05:30
fogelito a4844599c6 Merge branch '1.8.x' of https://github.com/appwrite/appwrite into stop-on-error 2026-03-05 12:00:07 +02:00
Shmuel FogelandGitHub ce876adb3a Merge pull request #11460 from appwrite/stats-count-buckets
Stats count image transformations errors
2026-03-05 11:58:40 +02:00
fogelito 59edcf1934 Merge branch '1.8.x' of https://github.com/appwrite/appwrite into stop-on-error 2026-03-05 11:42:16 +02:00
fogelito 5821832da6 count_for_image_transformations_bucket_ 2026-03-05 11:08:45 +02:00
Chirag AggarwalandGitHub d4df2c51de Merge pull request #11442 from appwrite/fix/sdk-beta-versioning
refactor: improve SDK generation task reliability and maintainability
2026-03-05 14:17:25 +05:30
fogelito dbbc57bc37 Merge branch '1.8.x' of https://github.com/appwrite/appwrite into stop-on-error 2026-03-05 10:31:32 +02:00
fogelito d772220414 stopOnError 2026-03-05 10:28:42 +02:00
ArnabChatterjee20k 9fdd7c1c6e added try catch connection close metric 2026-03-05 11:30:15 +05:30
Chirag AggarwalandGitHub 5510231b8e Merge branch '1.8.x' into fix/sdk-beta-versioning 2026-03-05 09:52:47 +05:30
Chirag Aggarwal 152de6c584 remove goto 2026-03-05 09:32:07 +05:30
Chirag Aggarwal 161c5af66f fixes 2026-03-04 19:35:30 +05:30
Chirag Aggarwal c497a68613 fixes 2026-03-04 19:33:41 +05:30
Chirag Aggarwal 50f9c67862 fixes 2026-03-04 19:10:39 +05:30
Chirag Aggarwal 198f9a64a3 refactoring 2026-03-04 19:03:21 +05:30
Matej BačoandGitHub 248b3c8b32 Revert "Revert "Features with schema changes"" 2026-03-04 14:31:27 +01:00
Chirag Aggarwal 300aaeb251 fix: escape shell arguments in SDK git and gh CLI commands
Replace raw string interpolation with escapeshellarg() for all
arguments passed to exec/shell_exec calls that build git commit,
gh pr create, gh api, and gh release commands. This prevents
shell injection from AI-generated changelog text or any other
dynamically constructed values.
2026-03-04 18:46:49 +05:30
Chirag Aggarwal f989ccde57 fix: add post-parse guard for beta SDK version bumps
Add a programmatic guard after parsing the AI response that rejects
major bumps or versions >= 1.0.0 for beta SDKs. When triggered, the
SDK is skipped with a warning instead of proceeding with an invalid
version.
2026-03-04 18:44:29 +05:30
Chirag Aggarwal fc88d4b4ab feat: auto-generate commit messages for SDK pushes
Remove the manual commit message prompt. When AI is available and
produces a changelog, use it as the commit message. Otherwise fall
back to a descriptive message based on SDK name and version. A
manually provided --message flag still takes priority.
2026-03-04 13:19:48 +05:30
Chirag Aggarwal df604cb2ef refactor: derive supported SDKs from config instead of hardcoded array
Replace the static supportedSDKS array with a getSupportedSDKs() method
that reads SDK keys from the sdks.php config file. This eliminates the
need to maintain the list in two places.
2026-03-04 13:16:55 +05:30
Chirag Aggarwal a6d58a847d fix: handle beta SDK versioning in DiffCheck prompt
Beta SDKs (version < 1.0.0) should not be bumped to 1.0.0. The prompt
now checks the beta flag from sdk config and instructs the AI to use
minor bumps for both breaking changes and new features, and patch for
bug fixes only. Also adds a rule to wrap code identifiers in backticks
for better changelog rendering.
2026-03-04 13:15:26 +05:30
ArnabChatterjee20k 82db411517 updated 2026-03-03 19:36:49 +05:30
ArnabChatterjee20k 7644e0fe48 Add realtime metrics for connections, messages, and bandwidth in project usage 2026-03-03 18:48:37 +05:30
187 changed files with 105141 additions and 3467 deletions
-2
View File
@@ -5,8 +5,6 @@ on:
types: [opened, edited]
issue_comment:
types: [created, edited]
pull_request:
types: [opened, edited]
pull_request_review:
types: [submitted, edited]
pull_request_review_comment:
-123
View File
@@ -1,123 +0,0 @@
name: Benchmark
concurrency:
group: '${{ github.workflow }}-${{ github.ref }}'
cancel-in-progress: true
env:
COMPOSE_FILE: docker-compose.yml
IMAGE: appwrite-dev
CACHE_KEY: 'appwrite-dev-${{ github.event.pull_request.head.sha }}'
'on':
- pull_request
jobs:
setup:
name: Setup & Build Appwrite Image
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
submodules: recursive
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build Appwrite
uses: docker/build-push-action@v6
with:
context: .
push: false
tags: '${{ env.IMAGE }}'
load: true
cache-from: type=gha
cache-to: 'type=gha,mode=max'
outputs: 'type=docker,dest=/tmp/${{ env.IMAGE }}.tar'
target: development
build-args: |
DEBUG=false
TESTING=true
VERSION=dev
- name: Cache Docker Image
uses: actions/cache@v4
with:
key: '${{ env.CACHE_KEY }}'
path: '/tmp/${{ env.IMAGE }}.tar'
benchmarking:
name: Benchmark
runs-on: ubuntu-latest
needs: setup
permissions:
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: '${{ env.CACHE_KEY }}'
path: '/tmp/${{ env.IMAGE }}.tar'
fail-on-cache-miss: true
- name: Load and Start Appwrite
run: |
sed -i 's/traefik/localhost/g' .env
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
sleep 10
- name: Install Oha
run: |
echo "deb [signed-by=/usr/share/keyrings/azlux-archive-keyring.gpg] http://packages.azlux.fr/debian/ stable main" | sudo tee /etc/apt/sources.list.d/azlux.list
sudo wget -O /usr/share/keyrings/azlux-archive-keyring.gpg https://azlux.fr/repo.gpg
sudo apt update
sudo apt install oha
oha --version
- name: Benchmark PR
run: 'oha -z 180s http://localhost/v1/health/version --output-format json > benchmark.json'
- name: Cleaning
run: docker compose down -v
- name: Installing latest version
run: |
rm docker-compose.yml
rm .env
curl https://appwrite.io/install/compose -o docker-compose.yml
curl https://appwrite.io/install/env -o .env
sed -i 's/_APP_OPTIONS_ABUSE=enabled/_APP_OPTIONS_ABUSE=disabled/g' .env
docker compose up -d
sleep 10
- name: Benchmark Latest
run: oha -z 180s http://localhost/v1/health/version --output-format json > benchmark-latest.json
- name: Prepare comment
run: |
echo '## :sparkles: Benchmark results' > benchmark.txt
echo ' ' >> benchmark.txt
echo "- Requests per second: $(jq -r '.summary.requestsPerSec|tonumber?|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json)" >> benchmark.txt
echo "- Requests with 200 status code: $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json)" >> benchmark.txt
echo "- P99 latency: $(jq -r '.latencyPercentiles.p99' benchmark.json )" >> benchmark.txt
echo " " >> benchmark.txt
echo " " >> benchmark.txt
echo "## :zap: Benchmark Comparison" >> benchmark.txt
echo " " >> benchmark.txt
echo "| Metric | This PR | Latest version | " >> benchmark.txt
echo "| --- | --- | --- | " >> benchmark.txt
echo "| RPS | $(jq -r '.summary.requestsPerSec|tonumber?|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json) | $(jq -r '.summary.requestsPerSec|tonumber|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark-latest.json) | " >> benchmark.txt
echo "| 200 | $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json) | $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark-latest.json) | " >> benchmark.txt
echo "| P99 | $(jq -r '.latencyPercentiles.p99' benchmark.json ) | $(jq -r '.latencyPercentiles.p99' benchmark-latest.json ) | " >> benchmark.txt
- name: Save results
uses: actions/upload-artifact@v6
if: '${{ !cancelled() }}'
with:
name: benchmark.json
path: benchmark.json
retention-days: 7
- name: Find Comment
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: '${{ github.event.pull_request.number }}'
comment-author: 'github-actions[bot]'
body-includes: Benchmark results
- name: Comment on PR
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: '${{ steps.fc.outputs.comment-id }}'
issue-number: '${{ github.event.pull_request.number }}'
body-path: benchmark.txt
edit-mode: replace
-19
View File
@@ -1,19 +0,0 @@
name: Check dependencies
# Adapted from https://google.github.io/osv-scanner/github-action/#scan-on-pull-request
on:
pull_request:
branches: [main, 1.*.x]
merge_group:
branches: [main, 1.*.x]
permissions:
# Require writing security events to upload SARIF file to security tab
security-events: write
# Only need to read contents
contents: read
jobs:
scan-pr:
uses: "google/osv-scanner-action/.github/workflows/osv-scanner-reusable.yml@v1.7.1"
+706
View File
@@ -0,0 +1,706 @@
name: CI
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
env:
COMPOSE_FILE: docker-compose.yml
IMAGE: appwrite-dev
CACHE_KEY: appwrite-dev-${{ github.event.pull_request.head.sha }}
on:
pull_request:
workflow_dispatch:
inputs:
response_format:
description: 'Response format version to test (e.g., 1.5.0, 1.4.0)'
required: false
type: string
default: ''
jobs:
dependencies:
name: Checks / Dependencies
if: github.event_name == 'pull_request'
permissions:
actions: read
security-events: write
contents: read
uses: "google/osv-scanner-action/.github/workflows/osv-scanner-reusable-pr.yml@v2.3.3"
security:
name: Checks / Image
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
steps:
- name: Check out code
uses: actions/checkout@v6
with:
fetch-depth: 0
submodules: 'recursive'
- name: Build the Docker image
uses: docker/build-push-action@v6
with:
context: .
push: false
load: true
tags: pr_image:${{ github.sha }}
target: production
- name: Run Trivy vulnerability scanner on image
uses: aquasecurity/trivy-action@0.35.0
with:
image-ref: 'pr_image:${{ github.sha }}'
format: 'sarif'
output: 'trivy-image-results.sarif'
severity: 'CRITICAL,HIGH'
- name: Run Trivy vulnerability scanner on source code
uses: aquasecurity/trivy-action@0.35.0
with:
scan-type: 'fs'
scan-ref: '.'
format: 'sarif'
output: 'trivy-fs-results.sarif'
severity: 'CRITICAL,HIGH'
skip-setup-trivy: true
- name: Upload image scan results
uses: github/codeql-action/upload-sarif@v4
if: always() && hashFiles('trivy-image-results.sarif') != ''
with:
sarif_file: 'trivy-image-results.sarif'
category: 'trivy-image'
- name: Upload source code scan results
uses: github/codeql-action/upload-sarif@v4
if: always() && hashFiles('trivy-fs-results.sarif') != ''
with:
sarif_file: 'trivy-fs-results.sarif'
category: 'trivy-source'
composer:
name: Checks / Composer
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.3'
tools: composer:v2
coverage: none
- name: Validate
run: composer validate
- name: Install dependencies
run: composer install --prefer-dist --no-progress --ignore-platform-reqs
- name: Audit
env:
COMPOSER_NO_AUDIT: 0
run: composer audit
format:
name: Checks / Format
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 2
- run: git checkout HEAD^2
if: github.event_name == 'pull_request'
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.3'
tools: composer:v2
coverage: none
- name: Install dependencies
run: composer install --prefer-dist --no-progress --ignore-platform-reqs
- name: Run Linter
run: composer lint
analyze:
name: Checks / Analyze
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v6
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.3'
tools: composer:v2
coverage: none
- name: Install dependencies
run: composer install --prefer-dist --no-progress --ignore-platform-reqs
- name: Run PHPStan
run: composer analyze
locale:
name: Checks / Locale
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v6
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: '24'
- name: Run Locale check
run: node .github/workflows/static-analysis/locale/index.js
matrix:
name: Tests / Matrix
runs-on: ubuntu-latest
outputs:
databases: ${{ steps.generate.outputs.databases }}
modes: ${{ steps.generate.outputs.modes }}
steps:
- name: Generate matrix
id: generate
uses: actions/github-script@v8
with:
script: |
const allDatabases = ['MariaDB', 'PostgreSQL', 'MongoDB'];
const allModes = ['dedicated', 'shared_v1', 'shared_v2'];
const defaultDatabases = ['MongoDB'];
const defaultModes = ['dedicated'];
const pr = context.payload.pull_request;
if (!pr) {
core.setOutput('databases', JSON.stringify(allDatabases));
core.setOutput('modes', JSON.stringify(allModes));
return;
}
const getContent = (ref) => github.rest.repos.getContent({
owner: context.repo.owner,
repo: context.repo.repo,
path: 'composer.lock',
ref,
});
const getDbVersion = (lock) => lock.packages?.find(p => p.name === 'utopia-php/database')?.version;
const [{ data: base }, { data: head }] = await Promise.all([
getContent(pr.base.sha),
getContent(pr.head.sha),
]);
const decode = (content) => JSON.parse(Buffer.from(content, 'base64').toString());
const databaseChanged = getDbVersion(decode(base.content)) !== getDbVersion(decode(head.content));
core.setOutput('databases', JSON.stringify(databaseChanged ? allDatabases : defaultDatabases));
core.setOutput('modes', JSON.stringify(databaseChanged ? allModes : defaultModes));
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
submodules: recursive
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build Appwrite
uses: docker/build-push-action@v6
with:
context: .
push: false
tags: ${{ env.IMAGE }}
load: true
cache-from: type=gha
cache-to: type=gha,mode=max
outputs: type=docker,dest=/tmp/${{ env.IMAGE }}.tar
target: development
build-args: |
DEBUG=false
TESTING=true
VERSION=dev
- name: Cache Docker Image
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
unit:
name: Tests / Unit
runs-on: ubuntu-latest
needs: build
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Environment Variables
run: docker compose exec -T appwrite vars
- name: Run Unit Tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/unit
command: >-
docker compose exec
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/unit
e2e_general:
name: Tests / E2E / General
runs-on: ubuntu-latest
needs: build
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run General Tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/General
command: >-
docker compose exec -T
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/e2e/General
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_service:
name: Tests / E2E / ${{ matrix.database }} (${{ matrix.mode }}) / ${{ matrix.service }}
runs-on: ubuntu-latest
needs: [build, matrix]
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
database: ${{ fromJSON(needs.matrix.outputs.databases) }}
mode: ${{ fromJSON(needs.matrix.outputs.modes) }}
service: [
Account,
Avatars,
Console,
Databases,
TablesDB,
Functions,
FunctionsSchedule,
GraphQL,
Health,
Locale,
Projects,
Realtime,
Sites,
Proxy,
Storage,
Tokens,
Teams,
Users,
Webhooks,
VCS,
Messaging,
Migrations
]
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Set database environment
run: |
if [ "${{ matrix.database }}" = "MariaDB" ]; then
echo "COMPOSE_PROFILES=mariadb" >> $GITHUB_ENV
echo "_APP_DB_ADAPTER=mariadb" >> $GITHUB_ENV
echo "_APP_DB_HOST=mariadb" >> $GITHUB_ENV
echo "_APP_DB_PORT=3306" >> $GITHUB_ENV
elif [ "${{ matrix.database }}" = "MongoDB" ]; then
echo "COMPOSE_PROFILES=mongodb" >> $GITHUB_ENV
echo "_APP_DB_ADAPTER=mongodb" >> $GITHUB_ENV
echo "_APP_DB_HOST=mongodb" >> $GITHUB_ENV
echo "_APP_DB_PORT=27017" >> $GITHUB_ENV
elif [ "${{ matrix.database }}" = "PostgreSQL" ]; then
echo "COMPOSE_PROFILES=postgresql" >> $GITHUB_ENV
echo "_APP_DB_ADAPTER=postgresql" >> $GITHUB_ENV
echo "_APP_DB_HOST=postgresql" >> $GITHUB_ENV
echo "_APP_DB_PORT=5432" >> $GITHUB_ENV
fi
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
env:
_APP_BROWSER_HOST: http://invalid-browser/v1
_APP_DATABASE_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'database_db_main' || '' }}
_APP_DATABASE_SHARED_TABLES_V1: ${{ matrix.mode == 'shared_v1' && 'database_db_main' || '' }}
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 20
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/${{ matrix.service }}
command: |
SERVICE_PATH="/usr/src/code/tests/e2e/Services/${{ matrix.service }}"
# Services that rely on sequential test method execution (shared static state)
FUNCTIONAL_FLAG="--functional"
case "${{ matrix.service }}" in
Databases|TablesDB|Functions|Realtime) FUNCTIONAL_FLAG="" ;;
esac
docker compose exec -T \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite vendor/bin/paratest --processes $(nproc) $FUNCTIONAL_FLAG "$SERVICE_PATH" --exclude-group abuseEnabled --exclude-group screenshots --log-junit tests/e2e/Services/${{ matrix.service }}/junit.xml
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_abuse:
name: Tests / E2E / Abuse (${{ matrix.mode }})
runs-on: ubuntu-latest
needs: [build, matrix]
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
mode: ${{ fromJSON(needs.matrix.outputs.modes) }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
env:
_APP_OPTIONS_ABUSE: enabled
_APP_DATABASE_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'database_db_main' || '' }}
_APP_DATABASE_SHARED_TABLES_V1: ${{ matrix.mode == 'shared_v1' && 'database_db_main' || '' }}
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Run tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e
command: >-
docker compose exec -T
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/e2e --group=abuseEnabled
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_screenshots:
name: Tests / E2E / Screenshots (${{ matrix.mode }})
runs-on: ubuntu-latest
needs: [build, matrix]
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
mode: ${{ fromJSON(needs.matrix.outputs.modes) }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
timeout-minutes: 5
env:
_APP_DATABASE_SHARED_TABLES: ${{ matrix.mode != 'dedicated' && 'database_db_main' || '' }}
_APP_DATABASE_SHARED_TABLES_V1: ${{ matrix.mode == 'shared_v1' && 'database_db_main' || '' }}
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose pull --quiet --ignore-buildable
docker compose up -d --quiet-pull --wait
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 60
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Sites
command: >-
docker compose exec -T
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/e2e/Services/Sites --group=screenshots
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
benchmark:
name: Benchmark
runs-on: ubuntu-latest
needs: build
permissions:
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v5
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Login to Docker Hub
uses: docker/login-action@v4
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Load and Start Appwrite
run: |
sed -i 's/traefik/localhost/g' .env
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
sleep 10
- name: Install Oha
run: |
echo "deb [signed-by=/usr/share/keyrings/azlux-archive-keyring.gpg] http://packages.azlux.fr/debian/ stable main" | sudo tee /etc/apt/sources.list.d/azlux.list
sudo wget -O /usr/share/keyrings/azlux-archive-keyring.gpg https://azlux.fr/repo.gpg
sudo apt update
sudo apt install oha
oha --version
- name: Benchmark PR
run: 'oha -z 180s http://localhost/v1/health/version --output-format json > benchmark.json'
- name: Cleaning
run: docker compose down -v
- name: Installing latest version
run: |
rm docker-compose.yml
rm .env
curl https://appwrite.io/install/compose -o docker-compose.yml
curl https://appwrite.io/install/env -o .env
sed -i 's/_APP_OPTIONS_ABUSE=enabled/_APP_OPTIONS_ABUSE=disabled/g' .env
docker compose up -d
sleep 10
- name: Benchmark Latest
run: oha -z 180s http://localhost/v1/health/version --output-format json > benchmark-latest.json
- name: Prepare comment
run: |
echo '## :sparkles: Benchmark results' > benchmark.txt
echo ' ' >> benchmark.txt
echo "- Requests per second: $(jq -r '.summary.requestsPerSec|tonumber?|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json)" >> benchmark.txt
echo "- Requests with 200 status code: $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json)" >> benchmark.txt
echo "- P99 latency: $(jq -r '.latencyPercentiles.p99' benchmark.json )" >> benchmark.txt
echo " " >> benchmark.txt
echo " " >> benchmark.txt
echo "## :zap: Benchmark Comparison" >> benchmark.txt
echo " " >> benchmark.txt
echo "| Metric | This PR | Latest version | " >> benchmark.txt
echo "| --- | --- | --- | " >> benchmark.txt
echo "| RPS | $(jq -r '.summary.requestsPerSec|tonumber?|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json) | $(jq -r '.summary.requestsPerSec|tonumber|floor|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark-latest.json) | " >> benchmark.txt
echo "| 200 | $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark.json) | $(jq -r '.statusCodeDistribution."200"|tostring|[while(length>0;.[:-3])|.[-3:]]|reverse|join(",")' benchmark-latest.json) | " >> benchmark.txt
echo "| P99 | $(jq -r '.latencyPercentiles.p99' benchmark.json ) | $(jq -r '.latencyPercentiles.p99' benchmark-latest.json ) | " >> benchmark.txt
- name: Save results
uses: actions/upload-artifact@v7
if: ${{ !cancelled() }}
with:
name: benchmark.json
path: benchmark.json
retention-days: 7
- name: Find Comment
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: Benchmark results
- name: Comment on PR
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body-path: benchmark.txt
edit-mode: replace
-28
View File
@@ -1,28 +0,0 @@
name: "Linter"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on: [pull_request]
jobs:
lint:
name: Linter
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 2
- run: git checkout HEAD^2
- name: Validate composer.json and composer.lock
run: |
docker run --rm -v $PWD:/app composer:2.8 sh -c \
"composer validate"
- name: Run Linter
run: |
docker run --rm -v $PWD:/app composer:2.8 sh -c \
"composer install --profile --ignore-platform-reqs && composer lint"
-106
View File
@@ -1,106 +0,0 @@
name: PR Security Scan
on:
pull_request_target:
types: [opened, synchronize, reopened]
jobs:
scan:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- name: Check out code
uses: actions/checkout@v6
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
submodules: 'recursive'
- name: Build the Docker image
uses: docker/build-push-action@v6
with:
context: .
push: false
load: true
tags: pr_image:${{ github.sha }}
target: production
- name: Run Trivy vulnerability scanner on image
uses: aquasecurity/trivy-action@0.20.0
with:
image-ref: 'pr_image:${{ github.sha }}'
format: 'json'
output: 'trivy-image-results.json'
severity: 'CRITICAL,HIGH'
- name: Run Trivy vulnerability scanner on source code
uses: aquasecurity/trivy-action@0.20.0
with:
scan-type: 'fs'
scan-ref: '.'
format: 'json'
output: 'trivy-fs-results.json'
severity: 'CRITICAL,HIGH'
- name: Process Trivy scan results
id: process-results
uses: actions/github-script@v8
with:
script: |
const fs = require('fs');
let commentBody = '## Security Scan Results for PR\n\n';
function processResults(results, title) {
let sectionBody = `### ${title}\n\n`;
if (results.Results && results.Results.some(result => result.Vulnerabilities && result.Vulnerabilities.length > 0)) {
sectionBody += '| Package | Version | Vulnerability | Severity |\n';
sectionBody += '|---------|---------|----------------|----------|\n';
const uniqueVulns = new Set();
results.Results.forEach(result => {
if (result.Vulnerabilities) {
result.Vulnerabilities.forEach(vuln => {
const vulnKey = `${vuln.PkgName}-${vuln.InstalledVersion}-${vuln.VulnerabilityID}`;
if (!uniqueVulns.has(vulnKey)) {
uniqueVulns.add(vulnKey);
sectionBody += `| ${vuln.PkgName} | ${vuln.InstalledVersion} | [${vuln.VulnerabilityID}](https://nvd.nist.gov/vuln/detail/${vuln.VulnerabilityID}) | ${vuln.Severity} |\n`;
}
});
}
});
} else {
sectionBody += '🎉 No vulnerabilities found!\n';
}
return sectionBody;
}
try {
const imageResults = JSON.parse(fs.readFileSync('trivy-image-results.json', 'utf8'));
const fsResults = JSON.parse(fs.readFileSync('trivy-fs-results.json', 'utf8'));
commentBody += processResults(imageResults, "Docker Image Scan Results");
commentBody += '\n';
commentBody += processResults(fsResults, "Source Code Scan Results");
} catch (error) {
commentBody += `There was an error while running the security scan: ${error.message}\n`;
commentBody += 'Please contact the core team for assistance.';
}
core.setOutput('comment-body', commentBody);
- name: Find Comment
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: Security Scan Results for PR
- name: Create or update comment
uses: peter-evans/create-or-update-comment@v3
with:
issue-number: ${{ github.event.pull_request.number }}
comment-id: ${{ steps.fc.outputs.comment-id }}
body: ${{ steps.process-results.outputs.comment-body }}
edit-mode: replace
+1 -1
View File
@@ -9,7 +9,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v9
- uses: actions/stale@v10
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: "This issue has been labeled as a 'question', indicating that it requires additional information from the requestor. It has been inactive for 7 days. If no further activity occurs, this issue will be closed in 14 days."
-21
View File
@@ -1,21 +0,0 @@
name: "Static code analysis"
on: [pull_request]
jobs:
lint:
name: CodeQL
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v6
- name: Run CodeQL
run: |
docker run --rm -v $PWD:/app composer:2.8 sh -c \
"composer install --profile --ignore-platform-reqs && composer check"
- name: Run Locale check
run: |
docker run --rm -v $PWD:/app node:24-alpine sh -c \
"cd /app/.github/workflows/static-analysis/locale && node index.js"
-691
View File
@@ -1,691 +0,0 @@
name: "Tests"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
COMPOSE_FILE: docker-compose.yml
IMAGE: appwrite-dev
CACHE_KEY: appwrite-dev-${{ github.event.pull_request.head.sha }}
on:
pull_request:
workflow_dispatch:
inputs:
response_format:
description: 'Response format version to test (e.g., 1.5.0, 1.4.0)'
required: false
type: string
default: ''
jobs:
check_database_changes:
name: Check if utopia-php/database changed
runs-on: ubuntu-latest
outputs:
database_changed: ${{ steps.check.outputs.database_changed }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Fetch base branch
run: git fetch origin ${{ github.event.pull_request.base.ref }}
- name: Check for utopia-php/database changes
id: check
run: |
if git diff origin/${{ github.event.pull_request.base.ref }} HEAD -- composer.lock | grep -q '"name": "utopia-php/database"'; then
echo "Database version changed, going to run all mode tests."
echo "database_changed=true" >> "$GITHUB_ENV"
echo "database_changed=true" >> "$GITHUB_OUTPUT"
else
echo "database_changed=false" >> "$GITHUB_ENV"
echo "database_changed=false" >> "$GITHUB_OUTPUT"
fi
setup:
name: Setup & Build Appwrite Image
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
submodules: recursive
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build Appwrite
uses: docker/build-push-action@v6
with:
context: .
push: false
tags: ${{ env.IMAGE }}
load: true
cache-from: type=gha
cache-to: type=gha,mode=max
outputs: type=docker,dest=/tmp/${{ env.IMAGE }}.tar
target: development
build-args: |
DEBUG=false
TESTING=true
VERSION=dev
- name: Cache Docker Image
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
unit_test:
name: Unit Test
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Environment Variables
run: docker compose exec -T appwrite vars
- name: Run Unit Tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/unit
command: >-
docker compose exec
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/unit
e2e_general_test:
name: E2E General Test
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run General Tests
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/General
command: >-
docker compose exec -T
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}"
appwrite test /usr/src/code/tests/e2e/General --debug
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_service_test:
name: E2E Service Test
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
db_adapter: [
MARIADB,
POSTGRESQL,
MONGODB
]
service: [
Account,
Avatars,
Console,
Databases,
Functions,
FunctionsSchedule,
GraphQL,
Health,
Locale,
Projects,
Realtime,
Sites,
Proxy,
Storage,
Tokens,
Teams,
Users,
Webhooks,
VCS,
Messaging,
Migrations
]
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Set DB Adapter environment
id: set-db-env
run: |
DB_ADAPTER_LOWER=$(echo "${{ matrix.db_adapter }}" | tr 'A-Z' 'a-z')
echo "COMPOSE_PROFILES=${DB_ADAPTER_LOWER}" >> $GITHUB_ENV
if [ "${{ matrix.db_adapter }}" = "MARIADB" ]; then
echo "_APP_DB_ADAPTER=mariadb" >> $GITHUB_ENV
echo "_APP_DB_HOST=mariadb" >> $GITHUB_ENV
echo "_APP_DB_PORT=3306" >> $GITHUB_ENV
elif [ "${{ matrix.db_adapter }}" = "MONGODB" ]; then
echo "_APP_DB_ADAPTER=mongodb" >> $GITHUB_ENV
echo "_APP_DB_HOST=mongodb" >> $GITHUB_ENV
echo "_APP_DB_PORT=27017" >> $GITHUB_ENV
elif [ "${{ matrix.db_adapter }}" = "POSTGRESQL" ]; then
echo "_APP_DB_ADAPTER=postgresql" >> $GITHUB_ENV
echo "_APP_DB_HOST=postgresql" >> $GITHUB_ENV
echo "_APP_DB_PORT=5432" >> $GITHUB_ENV
fi
- name: Load and Start Appwrite
timeout-minutes: 3
env:
_APP_BROWSER_HOST: http://invalid-browser/v1
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run ${{ matrix.service }} tests with Project table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 20
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/${{ matrix.service }}
command: |
echo "Using project tables"
SERVICE_PATH="/usr/src/code/tests/e2e/Services/${{ matrix.service }}"
# Services that rely on sequential test method execution (shared static state)
FUNCTIONAL_FLAG="--functional"
case "${{ matrix.service }}" in
Databases|Functions|Realtime) FUNCTIONAL_FLAG="" ;;
esac
echo "Running with paratest (parallel) for: ${{ matrix.service }} ${FUNCTIONAL_FLAG:+(functional)}"
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES="" \
-e _APP_DATABASE_SHARED_TABLES_V1="" \
-e _APP_DB_ADAPTER="${{ env._APP_DB_ADAPTER }}" \
-e _APP_DB_HOST="${{ env._APP_DB_HOST }}" \
-e _APP_DB_PORT="${{ env._APP_DB_PORT }}" \
-e _APP_DB_SCHEMA=appwrite \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite vendor/bin/paratest --processes $(nproc) $FUNCTIONAL_FLAG "$SERVICE_PATH" --exclude-group abuseEnabled --exclude-group screenshots --exclude-group ciIgnore --log-junit tests/e2e/Services/${{ matrix.service }}/junit.xml
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Logs ==="
docker compose logs
e2e_shared_mode_test:
name: E2E Shared Mode Service Test
runs-on: ubuntu-latest
needs: [ setup, check_database_changes ]
if: needs.check_database_changes.outputs.database_changed == 'true'
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
service:
[
Account,
Avatars,
Console,
Databases,
Functions,
FunctionsSchedule,
GraphQL,
Health,
Locale,
Projects,
Realtime,
Sites,
Proxy,
Storage,
Teams,
Users,
Webhooks,
VCS,
Messaging,
Migrations,
Tokens
]
tables-mode: [
'Shared V1',
'Shared V2',
]
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run ${{ matrix.service }} tests with ${{ matrix.tables-mode }} table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 20
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/${{ matrix.service }}
command: |
if [ "${{ matrix.tables-mode }}" == "Shared V1" ]; then
echo "Using shared tables V1"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=database_db_main
elif [ "${{ matrix.tables-mode }}" == "Shared V2" ]; then
echo "Using shared tables V2"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=
fi
SERVICE_PATH="/usr/src/code/tests/e2e/Services/${{ matrix.service }}"
# Services that rely on sequential test method execution (shared static state)
FUNCTIONAL_FLAG="--functional"
case "${{ matrix.service }}" in
Databases|Functions|Realtime) FUNCTIONAL_FLAG="" ;;
esac
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite vendor/bin/paratest --processes $(nproc) $FUNCTIONAL_FLAG "$SERVICE_PATH" --exclude-group abuseEnabled --exclude-group screenshots --exclude-group ciIgnore --log-junit tests/e2e/Services/${{ matrix.service }}/junit.xml
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
e2e_abuse_enabled:
name: E2E Service Test (Abuse enabled)
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
sed -i 's/_APP_OPTIONS_ABUSE=disabled/_APP_OPTIONS_ABUSE=enabled/' .env
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Run Projects tests in dedicated table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Projects
command: |
echo "Using project tables"
export _APP_DATABASE_SHARED_TABLES=
export _APP_DATABASE_SHARED_TABLES_V1=
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite test /usr/src/code/tests/e2e/Services/Projects --debug --group=abuseEnabled
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
e2e_abuse_enabled_shared_mode:
name: E2E Shared Mode Service Test (Abuse enabled)
runs-on: ubuntu-latest
needs: [ setup, check_database_changes ]
if: needs.check_database_changes.outputs.database_changed == 'true'
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
tables-mode: [
'Shared V1',
'Shared V2',
]
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
sed -i 's/_APP_OPTIONS_ABUSE=disabled/_APP_OPTIONS_ABUSE=enabled/' .env
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Run Projects tests in ${{ matrix.tables-mode }} table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Projects
command: |
if [ "${{ matrix.tables-mode }}" == "Shared V1" ]; then
echo "Using shared tables V1"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=database_db_main
elif [ "${{ matrix.tables-mode }}" == "Shared V2" ]; then
echo "Using shared tables V2"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=
fi
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite test /usr/src/code/tests/e2e/Services/Projects --debug --group=abuseEnabled
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
e2e_screenshots:
name: E2E Service Test (Site Screenshots)
runs-on: ubuntu-latest
needs: setup
permissions:
contents: read
pull-requests: write
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
sed -i 's/_APP_OPTIONS_ABUSE=disabled/_APP_OPTIONS_ABUSE=enabled/' .env
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run Site tests with browser connected in dedicated table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Sites
command: |
echo "Keeping original value of _APP_BROWSER_HOST"
echo "Using project tables"
export _APP_DATABASE_SHARED_TABLES=
export _APP_DATABASE_SHARED_TABLES_V1=
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite test /usr/src/code/tests/e2e/Services/Sites --debug --group=screenshots
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
e2e_screenshots_shared_mode:
name: E2E Shared Mode Service Test (Site Screenshots)
runs-on: ubuntu-latest
needs: [ setup, check_database_changes ]
if: needs.check_database_changes.outputs.database_changed == 'true'
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
tables-mode: [
'Shared V1',
'Shared V2',
]
steps:
- name: checkout
uses: actions/checkout@v6
- name: Load Cache
uses: actions/cache@v4
with:
key: ${{ env.CACHE_KEY }}
path: /tmp/${{ env.IMAGE }}.tar
fail-on-cache-miss: true
- name: Load and Start Appwrite
timeout-minutes: 3
run: |
docker load --input /tmp/${{ env.IMAGE }}.tar
sed -i 's/_APP_OPTIONS_ABUSE=disabled/_APP_OPTIONS_ABUSE=enabled/' .env
docker compose up -d
until docker compose exec -T appwrite doctor > /dev/null 2>&1; do
echo "Waiting for Appwrite to be ready..."
sleep 2
done
- name: Wait for Open Runtimes
timeout-minutes: 3
run: |
while ! docker compose logs openruntimes-executor | grep -q "Executor is ready."; do
echo "Waiting for Executor to come online"
sleep 1
done
- name: Run Site tests with browser connected in ${{ matrix.tables-mode }} table mode
uses: itznotabug/php-retry@v3
with:
max_attempts: 2
retry_wait_seconds: 300
timeout_minutes: 15
job_id: ${{ job.check_run_id }}
github_token: ${{ secrets.GITHUB_TOKEN }}
test_dir: tests/e2e/Services/Sites
command: |
echo "Keeping original value of _APP_BROWSER_HOST"
if [ "${{ matrix.tables-mode }}" == "Shared V1" ]; then
echo "Using shared tables V1"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=database_db_main
elif [ "${{ matrix.tables-mode }}" == "Shared V2" ]; then
echo "Using shared tables V2"
export _APP_DATABASE_SHARED_TABLES=database_db_main
export _APP_DATABASE_SHARED_TABLES_V1=
fi
docker compose exec -T \
-e _APP_DATABASE_SHARED_TABLES \
-e _APP_DATABASE_SHARED_TABLES_V1 \
-e _APP_E2E_RESPONSE_FORMAT="${{ github.event.inputs.response_format }}" \
appwrite test /usr/src/code/tests/e2e/Services/Sites --debug --group=screenshots
- name: Failure Logs
if: failure()
run: |
echo "=== Appwrite Worker Builds Logs ==="
docker compose logs appwrite-worker-builds
echo "=== OpenRuntimes Executor Logs ==="
docker compose logs openruntimes-executor
+30
View File
@@ -47,6 +47,36 @@ Examples:
'resourceType' => 'deployments'
```
## Performance Patterns
### Document Update Optimization
When updating documents, always pass only the changed attributes as a sparse `Document` rather than the full document. This is more efficient because `updateDocument()` internally performs `array_merge($old, $new)`.
**Correct Pattern:**
```php
// Good: Pass only changed attributes directly
$user = $dbForProject->updateDocument('users', $user->getId(), new Document([
'name' => $name,
'email' => $email,
]));
```
**Incorrect Pattern:**
```php
$user->setAttribute('name', $name);
$user->setAttribute('email', $email);
// Bad: Passing full document is inefficient
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
```
**Exceptions:**
- Migration files (need full document updates by design)
- Cases already using `array_merge()` with `getArrayCopy()`
- Updates where almost all attributes of the document change at once (sparse update provides little benefit compared to passing the full document)
- Complex nested relationship logic where full document state is required
## Security Considerations
### Critical Security Practices
+1
View File
@@ -0,0 +1 @@
@AGENTS.md
+36 -23
View File
@@ -111,7 +111,7 @@ $ git push origin [name_of_your_new_branch]
## Setup From Source
To set up a working **development environment**, just fork the project git repository and install the backend and frontend dependencies using the proper package manager and create run the docker-compose stack.
To set up a working **development environment**, just fork the project git repository and install the backend and frontend dependencies using the proper package manager and run the docker-compose stack.
> If you just want to install Appwrite for day-to-day use and not as a contributor, you can reference the [installation guide](https://github.com/appwrite/appwrite#installation), the [getting started guide](https://appwrite.io/docs/quick-starts), or the main [README](README.md) file.
@@ -173,12 +173,12 @@ Learn more at our [Technology Stack](#technology-stack) section.
- [MVVM](https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93viewmodel) - Appwrite console architecture
##### Container Namespace Conventions
To keep our services easy to understand within Docker we follow a naming convention for all our containers depending on it's intended use.
To keep our services easy to understand within Docker we follow a naming convention for all our containers depending on its intended use.
`appwrite-worker-X` - Workers (`src/Appwrite/Platform/Workers/*`)
`appwrite-task-X` - Tasks (`src/Appwrite/Platform/Tasks/*`)
Other containes should be named the same as their service, for example `redis` should just be called `redis`.
Other containers should be named the same as their service, for example `redis` should just be called `redis`.
##### Security
@@ -189,7 +189,7 @@ Other containes should be named the same as their service, for example `redis` s
## Modules
As Appwrite grows, we noticed approach of having all service endpoints in `app/controllers/api/[service].php` is not maintainable. Not only it creates massive files, it also doesnt contain all product's features such as workers or tasks. While there might still be some occurances of those controller files, we avoid it in all new development, and gradually migrate existing controllers to **HTTP modules**.
As Appwrite grows, we noticed approach of having all service endpoints in `app/controllers/api/[service].php` is not maintainable. Not only it creates massive files, it also doesn't contain all product's features such as workers or tasks. While there might still be some occurrences of those controller files, we avoid it in all new development, and gradually migrate existing controllers to **HTTP modules**.
### HTTP Endpoints
@@ -204,7 +204,7 @@ Tips and tricks:
1. If endpoint doesn't have resource, use service name as resource name too
> Example: `Modules/Sites/Http/Sites/Get.php`
2. If there are multiple resources, use then all in folder structure
2. If there are multiple resources, use them all in folder structure
> Example: `Modules/Sites/Http/Deployments/Builds/Create.php`
3. Action can only be `Get`, `Create`, `Update`, `Delete` or `XList`
@@ -395,7 +395,7 @@ These are the current metrics we collect usage stats for:
> Note: The curly brackets in the metric name represents a template and is replaced with a value when the metric is processed.
Metrics are collected within 3 scopes Daily, monthly, an infinity. Adding new usage metric in order to aggregate usage stats is very simple, but very much dependent on where do you want to collect
Metrics are collected within 3 scopes Daily, monthly, and infinity. Adding new usage metric in order to aggregate usage stats is very simple, but very much dependent on where do you want to collect
statistics ,via API or via background worker. For both cases you will need to add a `const` variable in `app/init.php` under the usage metrics list using the naming convention `METRIC_<RESOURCE_NAME>` as shown below.
```php
@@ -409,14 +409,16 @@ Next follow the appropriate steps below depending on whether you're adding the m
**API**
In file `app/controllers/shared/api.php` On the database listener, add to an existing or create a new switch case. Add a call to the usage worker with your new metric const like so:
In file `app/controllers/shared/api.php` On the database listener, add to an existing or create a new switch case. Accumulate metrics in the usage context like so:
```php
case $document->getCollection() === 'teams':
$queueForStatsUsage
->addMetric(METRIC_TEAMS, $value); // per project
$usage->addMetric(METRIC_TEAMS, $value); // per project
break;
```
The metrics will be automatically published by the shutdown hook at the end of the request. There is no need to manually trigger or publish.
There are cases when you need to handle metric that has a parent entity, like buckets.
Files are linked to a parent bucket, you should verify you remove the files stats when you delete a bucket.
@@ -425,14 +427,13 @@ In that case you need also to handle children removal using addReduce() method c
```php
case $document->getCollection() === 'buckets': //buckets
$queueForStatsUsage
->addMetric(METRIC_BUCKETS, $value); // per project
$usage->addMetric(METRIC_BUCKETS, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage
$usage
->addReduce($document);
}
break;
```
In addition, you will also need to add some logic to the `reduce()` method of the Usage worker located in `/src/Appwrite/Platform/Workers/Usage.php`, like so:
@@ -460,8 +461,12 @@ case $document->getCollection() === 'buckets':
**Background worker**
You need to inject the usage queue in the desired worker on the constructor method
You need to inject the usage context and publisher in the desired worker on the constructor method
```php
use Appwrite\Usage\Context;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Message\Usage as UsageMessage;
/**
* @throws Exception
*/
@@ -474,24 +479,32 @@ public function __construct()
->inject('dbForProject')
->inject('queueForFunctions')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('publisherForUsage')
->inject('log')
->callback(fn (Message $message, Database $dbForProject, Func $queueForFunctions, Event $queueForEvents, StatsUsage $queueForStatsUsage, Log $log) => $this->action($message, $dbForProject, $queueForFunctions, $queueForEvents, $queueForStatsUsage, $log));
->callback(fn (Message $message, Database $dbForProject, Func $queueForFunctions, Event $queueForEvents, Context $usage, UsagePublisher $publisherForUsage, Log $log) => $this->action($message, $dbForProject, $queueForFunctions, $queueForEvents, $usage, $publisherForUsage, $log));
}
```
and then trigger the queue with the new metric like so:
and then accumulate metrics, create a message, and publish like so:
```php
$queueForStatsUsage
$usage
->addMetric(METRIC_BUILDS, 1)
->addMetric(METRIC_BUILDS_STORAGE, $build->getAttribute('size', 0))
->addMetric(METRIC_BUILDS_COMPUTE, (int)$build->getAttribute('duration', 0) * 1000)
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS), 1)
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS), 1)
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS_STORAGE), $build->getAttribute('size', 0))
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS_COMPUTE), (int)$build->getAttribute('duration', 0) * 1000)
->setProject($project)
->trigger();
->addMetric(str_replace('{functionInternalId}', $function->getSequence(), METRIC_FUNCTION_ID_BUILDS_COMPUTE), (int)$build->getAttribute('duration', 0) * 1000);
// Publish the accumulated metrics (workers don't have shutdown hooks)
$message = new UsageMessage(
project: $project,
metrics: $usage->getMetrics(),
reduce: $usage->getReduce()
);
$publisherForUsage->enqueue($message);
$usage->reset();
```
@@ -661,7 +674,7 @@ docker compose exec redis redis-cli FLUSHALL
## Using preview domains locally
Appwrite Functions are automatically given a domain you can visit to execute the function. This domain has format `[SOMETHING].functions.localhost` unless you changed `_APP_DOMAIN_FUNCTIONS` environment variable. This default value works great when running Appwrite locally, but it can be impossible to use preview domains with Cloud woekspaces such as Gitpod or GitHub Codespaces.
Appwrite Functions are automatically given a domain you can visit to execute the function. This domain has format `[SOMETHING].functions.localhost` unless you changed `_APP_DOMAIN_FUNCTIONS` environment variable. This default value works great when running Appwrite locally, but it can be impossible to use preview domains with Cloud workspaces such as Gitpod or GitHub Codespaces.
To use preview domains on Cloud workspaces, you can visit hostname provided by them, and supply function's preview domain as URL parameter:
+1
View File
@@ -70,6 +70,7 @@ RUN chmod +x /usr/local/bin/doctor && \
chmod +x /usr/local/bin/sdks && \
chmod +x /usr/local/bin/specs && \
chmod +x /usr/local/bin/ssl && \
chmod +x /usr/local/bin/task-time-travel && \
chmod +x /usr/local/bin/screenshot && \
chmod +x /usr/local/bin/test && \
chmod +x /usr/local/bin/upgrade && \
+24 -18
View File
@@ -4,11 +4,13 @@ require_once __DIR__ . '/init.php';
use Appwrite\Event\Certificate;
use Appwrite\Event\Delete;
use Appwrite\Event\Event;
use Appwrite\Event\Func;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\StatsResources;
use Appwrite\Event\StatsUsage;
use Appwrite\Platform\Appwrite;
use Appwrite\Runtimes\Runtimes;
use Appwrite\Usage\Context as UsageContext;
use Appwrite\Utopia\Database\Documents\User;
use Executor\Executor;
use Swoole\Runtime;
@@ -29,6 +31,7 @@ use Utopia\Platform\Service;
use Utopia\Pools\Group;
use Utopia\Queue\Broker\Pool as BrokerPool;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
use Utopia\Registry\Registry;
use Utopia\System\System;
use Utopia\Telemetry\Adapter\None as NoTelemetry;
@@ -47,7 +50,7 @@ $platform = new Appwrite();
$args = $platform->getEnv('argv');
\array_shift($args);
if (!isset($args[0])) {
if (! isset($args[0])) {
Console::error('Missing task name');
Console::exit(1);
}
@@ -85,6 +88,7 @@ $setResource('pools', function (Registry $register) {
$setResource('authorization', function () {
$authorization = new Authorization();
$authorization->disable();
return $authorization;
}, []);
@@ -113,7 +117,7 @@ $setResource('dbForPlatform', function ($pools, $cache, $authorization) {
$collections = Config::getParam('collections', [])['console'];
$last = \array_key_last($collections);
if (!($dbForPlatform->exists($dbForPlatform->getDatabase(), $last))) { /** TODO cache ready variable using registry */
if (! ($dbForPlatform->exists($dbForPlatform->getDatabase(), $last))) { /** TODO cache ready variable using registry */
throw new Exception('Tables not ready yet.');
}
@@ -122,10 +126,10 @@ $setResource('dbForPlatform', function ($pools, $cache, $authorization) {
Console::warning($err->getMessage());
sleep($sleep);
}
} while ($attempts < $maxAttempts && !$ready);
} while ($attempts < $maxAttempts && ! $ready);
if (!$ready) {
throw new Exception("Console is not ready yet. Please try again later.");
if (! $ready) {
throw new Exception('Console is not ready yet. Please try again later.');
}
return $dbForPlatform;
@@ -163,7 +167,7 @@ $setResource('getProjectDB', function (Group $pools, Database $dbForPlatform, $c
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -184,7 +188,7 @@ $setResource('getProjectDB', function (Group $pools, Database $dbForPlatform, $c
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -207,8 +211,9 @@ $setResource('getLogsDB', function (Group $pools, Cache $cache, Authorization $a
$database = null;
return function (?Document $project = null) use ($pools, $cache, $database, $authorization) {
if ($database !== null && $project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int)$project->getSequence());
if ($database !== null && $project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
return $database;
}
@@ -224,8 +229,8 @@ $setResource('getLogsDB', function (Group $pools, Cache $cache, Authorization $a
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES);
// set tenant
if ($project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int)$project->getSequence());
if ($project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
}
return $database;
@@ -243,15 +248,16 @@ $setResource('publisherFunctions', function (BrokerPool $publisher) {
$setResource('publisherMigrations', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
$setResource('publisherStatsUsage', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
$setResource('publisherMessaging', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
$setResource('queueForStatsUsage', function (Publisher $publisher) {
return new StatsUsage($publisher);
}, ['publisher']);
$setResource('usage', function () {
return new UsageContext();
}, []);
$setResource('publisherForUsage', fn (Publisher $publisher) => new UsagePublisher(
$publisher,
new Queue(System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME))
), ['publisher']);
$setResource('queueForStatsResources', function (Publisher $publisher) {
return new StatsResources($publisher);
}, ['publisher']);
+1
View File
@@ -777,6 +777,7 @@ return [
'filters' => [],
],
[
// At the moment, always empty (no runtime supports it yet)
'array' => false,
'$id' => ID::custom('startCommand'),
'type' => Database::VAR_STRING,
+3 -3
View File
@@ -27,7 +27,7 @@ return [
Exception::GENERAL_RESOURCE_BLOCKED => [
'name' => Exception::GENERAL_RESOURCE_BLOCKED,
'description' => 'Access to this resource is blocked.',
'code' => 401,
'code' => 403,
],
Exception::GENERAL_UNKNOWN_ORIGIN => [
'name' => Exception::GENERAL_UNKNOWN_ORIGIN,
@@ -168,8 +168,8 @@ return [
],
Exception::USER_BLOCKED => [
'name' => Exception::USER_BLOCKED,
'description' => 'The current user has been blocked. You can unblock the user by making a request to the User API\'s "Update User Status" endpoint or in the Appwrite Console\'s Auth section.',
'code' => 401,
'description' => 'The current user has been blocked.',
'code' => 403,
],
Exception::USER_INVALID_TOKEN => [
'name' => Exception::USER_INVALID_TOKEN,
+1 -1
View File
@@ -231,7 +231,7 @@ return [
'url' => 'https://github.com/appwrite/sdk-for-cli',
'package' => 'https://www.npmjs.com/package/appwrite-cli',
'enabled' => true,
'beta' => true,
'beta' => false,
'dev' => false,
'hidden' => false,
'family' => APP_SDK_PLATFORM_CONSOLE,
+46 -32
View File
@@ -14,7 +14,6 @@ use Appwrite\Event\Delete;
use Appwrite\Event\Event;
use Appwrite\Event\Mail;
use Appwrite\Event\Messaging;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Hooks\Hooks;
use Appwrite\Network\Validator\Email as EmailValidator;
@@ -28,6 +27,7 @@ use Appwrite\SDK\MethodType;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Template\Template;
use Appwrite\URL\URL as URLParser;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Database\Validator\CustomId;
use Appwrite\Utopia\Database\Validator\Queries\Identities;
@@ -288,7 +288,10 @@ $createSession = function (string $userId, string $secret, Request $request, Res
}
try {
$dbForProject->updateDocument('users', $user->getId(), $user);
$dbForProject->updateDocument('users', $user->getId(), new Document([
'emailVerification' => $user->getAttribute('emailVerification'),
'phoneVerification' => $user->getAttribute('phoneVerification'),
]));
} catch (\Throwable $th) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed saving user to DB');
}
@@ -1032,7 +1035,11 @@ Http::post('/v1/account/sessions/email')
->setAttribute('password', $proofForPasswordUpdated->hash($password))
->setAttribute('hash', $proofForPasswordUpdated->getHash()->getName())
->setAttribute('hashOptions', $proofForPasswordUpdated->getHash()->getOptions());
$dbForProject->updateDocument('users', $user->getId(), $user);
$dbForProject->updateDocument('users', $user->getId(), new Document([
'password' => $user->getAttribute('password'),
'hash' => $user->getAttribute('hash'),
'hashOptions' => $user->getAttribute('hashOptions'),
]));
}
$dbForProject->purgeCachedDocument('users', $user->getId());
@@ -1822,7 +1829,11 @@ Http::get('/v1/account/sessions/oauth2/:provider/redirect')
->setAttribute('providerAccessToken', $accessToken)
->setAttribute('providerRefreshToken', $refreshToken)
->setAttribute('providerAccessTokenExpiry', DateTime::addSeconds(new \DateTime(), (int) $accessTokenExpiry));
$dbForProject->updateDocument('identities', $identity->getId(), $identity);
$dbForProject->updateDocument('identities', $identity->getId(), new Document([
'providerAccessToken' => $identity->getAttribute('providerAccessToken'),
'providerRefreshToken' => $identity->getAttribute('providerRefreshToken'),
'providerAccessTokenExpiry' => $identity->getAttribute('providerAccessTokenExpiry'),
]));
}
if (empty($user->getAttribute('email'))) {
@@ -1960,7 +1971,10 @@ Http::get('/v1/account/sessions/oauth2/:provider/redirect')
->setAttribute('sessionId', $session->getId())
->setAttribute('sessionInternalId', $session->getSequence());
$dbForProject->updateDocument('targets', $target->getId(), $target);
$dbForProject->updateDocument('targets', $target->getId(), new Document([
'sessionId' => $target->getAttribute('sessionId'),
'sessionInternalId' => $target->getAttribute('sessionInternalId'),
]));
}
}
@@ -2787,12 +2801,12 @@ Http::post('/v1/account/tokens/phone')
->inject('queueForMessaging')
->inject('locale')
->inject('timelimit')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('store')
->inject('proofForCode')
->inject('authorization')
->action(function (string $userId, string $phone, Request $request, Response $response, User $user, Document $project, array $platform, Database $dbForProject, Event $queueForEvents, Messaging $queueForMessaging, Locale $locale, callable $timelimit, StatsUsage $queueForStatsUsage, array $plan, Store $store, ProofsCode $proofForCode, Authorization $authorization) {
->action(function (string $userId, string $phone, Request $request, Response $response, User $user, Document $project, array $platform, Database $dbForProject, Event $queueForEvents, Messaging $queueForMessaging, Locale $locale, callable $timelimit, Context $usage, array $plan, Store $store, ProofsCode $proofForCode, Authorization $authorization) {
if (empty(System::getEnv('_APP_SMS_PROVIDER'))) {
throw new Exception(Exception::GENERAL_PHONE_DISABLED, 'Phone provider not configured');
}
@@ -2941,16 +2955,12 @@ Http::post('/v1/account/tokens/phone')
$countryCode = $helper->parse($phone)->getCountryCode();
if (!empty($countryCode)) {
$queueForStatsUsage
->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
$usage->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
}
} catch (NumberParseException $e) {
// Ignore invalid phone number for country code stats
}
$queueForStatsUsage
->addMetric(METRIC_AUTH_METHOD_PHONE, 1)
->setProject($project)
->trigger();
$usage->addMetric(METRIC_AUTH_METHOD_PHONE, 1);
}
$token->setAttribute('secret', $secret);
@@ -3145,7 +3155,9 @@ Http::patch('/v1/account/name')
$user->setAttribute('name', $name);
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
$user = $dbForProject->updateDocument('users', $user->getId(), new Document([
'name' => $user->getAttribute('name'),
]));
$queueForEvents->setParam('userId', $user->getId());
@@ -3798,13 +3810,15 @@ Http::put('/v1/account/recovery')
$hooks->trigger('passwordValidator', [$dbForProject, $project, $password, &$user, true]);
$profile = $dbForProject->updateDocument('users', $profile->getId(), $profile
->setAttribute('password', $newPassword)
->setAttribute('passwordHistory', $history)
->setAttribute('passwordUpdate', DateTime::now())
->setAttribute('hash', $proofForPassword->getHash()->getName())
->setAttribute('hashOptions', $proofForPassword->getHash()->getOptions())
->setAttribute('emailVerification', true));
$profile = $dbForProject->updateDocument('users', $profile->getId(), new Document(
[
'password' => $newPassword,
'passwordHistory' => $history,
'passwordUpdate' => DateTime::now(),
'hash' => $proofForPassword->getHash()->getName(),
'hashOptions' => $proofForPassword->getHash()->getOptions(),
'emailVerification' => true]
));
$user->setAttributes($profile->getArrayCopy());
@@ -4126,7 +4140,7 @@ Http::put('/v1/account/verifications/email')
$authorization->addRole(Role::user($profile->getId())->toString());
$profile = $dbForProject->updateDocument('users', $profile->getId(), $profile->setAttribute('emailVerification', true));
$profile = $dbForProject->updateDocument('users', $profile->getId(), new Document(['emailVerification' => true]));
$user->setAttributes($profile->getArrayCopy());
@@ -4181,11 +4195,11 @@ Http::post('/v1/account/verifications/phone')
->inject('project')
->inject('locale')
->inject('timelimit')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('proofForCode')
->inject('authorization')
->action(function (Request $request, Response $response, User $user, Database $dbForProject, Event $queueForEvents, Messaging $queueForMessaging, Document $project, Locale $locale, callable $timelimit, StatsUsage $queueForStatsUsage, array $plan, ProofsCode $proofForCode, Authorization $authorization) {
->action(function (Request $request, Response $response, User $user, Database $dbForProject, Event $queueForEvents, Messaging $queueForMessaging, Document $project, Locale $locale, callable $timelimit, Context $usage, array $plan, ProofsCode $proofForCode, Authorization $authorization) {
if (empty(System::getEnv('_APP_SMS_PROVIDER'))) {
throw new Exception(Exception::GENERAL_PHONE_DISABLED, 'Phone provider not configured');
}
@@ -4270,16 +4284,12 @@ Http::post('/v1/account/verifications/phone')
$countryCode = $helper->parse($phone)->getCountryCode();
if (!empty($countryCode)) {
$queueForStatsUsage
->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
$usage->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
}
} catch (NumberParseException $e) {
// Ignore invalid phone number for country code stats
}
$queueForStatsUsage
->addMetric(METRIC_AUTH_METHOD_PHONE, 1)
->setProject($project)
->trigger();
$usage->addMetric(METRIC_AUTH_METHOD_PHONE, 1);
}
$verification->setAttribute('secret', $secret);
@@ -4342,7 +4352,7 @@ Http::put('/v1/account/verifications/phone')
$authorization->addRole(Role::user($profile->getId())->toString());
$profile = $dbForProject->updateDocument('users', $profile->getId(), $profile->setAttribute('phoneVerification', true));
$profile = $dbForProject->updateDocument('users', $profile->getId(), new Document(['phoneVerification' => true]));
$user->setAttributes($profile->getArrayCopy());
@@ -4500,7 +4510,11 @@ Http::put('/v1/account/targets/:targetId/push')
$target->setAttribute('name', "{$device['deviceBrand']} {$device['deviceModel']}");
$target = $dbForProject->updateDocument('targets', $target->getId(), $target);
$target = $dbForProject->updateDocument('targets', $target->getId(), new Document([
'identifier' => $target->getAttribute('identifier'),
'expired' => $target->getAttribute('expired'),
'name' => $target->getAttribute('name'),
]));
$dbForProject->purgeCachedDocument('users', $user->getId());
+51 -20
View File
@@ -1161,7 +1161,7 @@ Http::patch('/v1/users/:userId/status')
throw new Exception(Exception::USER_NOT_FOUND);
}
$user = $dbForProject->updateDocument('users', $user->getId(), $user->setAttribute('status', (bool) $status));
$user = $dbForProject->updateDocument('users', $user->getId(), new Document(['status' => (bool) $status]));
$queueForEvents
->setParam('userId', $user->getId());
@@ -1204,7 +1204,7 @@ Http::put('/v1/users/:userId/labels')
$user->setAttribute('labels', (array) \array_values(\array_unique($labels)));
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
$user = $dbForProject->updateDocument('users', $user->getId(), new Document(['labels' => $user->getAttribute('labels')]));
$queueForEvents
->setParam('userId', $user->getId());
@@ -1245,7 +1245,7 @@ Http::patch('/v1/users/:userId/verification/phone')
throw new Exception(Exception::USER_NOT_FOUND);
}
$user = $dbForProject->updateDocument('users', $user->getId(), $user->setAttribute('phoneVerification', $phoneVerification));
$user = $dbForProject->updateDocument('users', $user->getId(), new Document(['phoneVerification' => $phoneVerification]));
$queueForEvents
->setParam('userId', $user->getId());
@@ -1289,7 +1289,7 @@ Http::patch('/v1/users/:userId/name')
$user->setAttribute('name', $name);
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
$user = $dbForProject->updateDocument('users', $user->getId(), new Document(['name' => $user->getAttribute('name')]));
$queueForEvents->setParam('userId', $user->getId());
@@ -1344,7 +1344,10 @@ Http::patch('/v1/users/:userId/password')
->setAttribute('password', '')
->setAttribute('passwordUpdate', DateTime::now());
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
$user = $dbForProject->updateDocument('users', $user->getId(), new Document([
'password' => $user->getAttribute('password'),
'passwordUpdate' => $user->getAttribute('passwordUpdate'),
]));
$queueForEvents->setParam('userId', $user->getId());
$response->dynamic($user, Response::MODEL_USER);
}
@@ -1377,7 +1380,13 @@ Http::patch('/v1/users/:userId/password')
->setAttribute('hash', $hasher->getName())
->setAttribute('hashOptions', $hasher->getOptions());
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
$user = $dbForProject->updateDocument('users', $user->getId(), new Document([
'password' => $user->getAttribute('password'),
'passwordHistory' => $user->getAttribute('passwordHistory'),
'passwordUpdate' => $user->getAttribute('passwordUpdate'),
'hash' => $user->getAttribute('hash'),
'hashOptions' => $user->getAttribute('hashOptions'),
]));
$sessions = $user->getAttribute('sessions', []);
$invalidate = $project->getAttribute('auths', default: [])['invalidateSessions'] ?? false;
@@ -1469,7 +1478,15 @@ Http::patch('/v1/users/:userId/email')
;
try {
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
$user = $dbForProject->updateDocument('users', $user->getId(), new Document([
'email' => $user->getAttribute('email'),
'emailVerification' => $user->getAttribute('emailVerification'),
'emailCanonical' => $user->getAttribute('emailCanonical'),
'emailIsCanonical' => $user->getAttribute('emailIsCanonical'),
'emailIsCorporate' => $user->getAttribute('emailIsCorporate'),
'emailIsDisposable' => $user->getAttribute('emailIsDisposable'),
'emailIsFree' => $user->getAttribute('emailIsFree'),
]));
/**
* @var Document $oldTarget
*/
@@ -1477,7 +1494,8 @@ Http::patch('/v1/users/:userId/email')
if ($oldTarget instanceof Document && !$oldTarget->isEmpty()) {
if (\strlen($email) !== 0) {
$dbForProject->updateDocument('targets', $oldTarget->getId(), $oldTarget->setAttribute('identifier', $email));
$dbForProject->updateDocument('targets', $oldTarget->getId(), new Document(['identifier' => $email]));
$oldTarget->setAttribute('identifier', $email);
} else {
$dbForProject->deleteDocument('targets', $oldTarget->getId());
}
@@ -1542,12 +1560,15 @@ Http::patch('/v1/users/:userId/phone')
$oldPhone = $user->getAttribute('phone');
// Store null instead of empty string so unique constraint allows multiple users without phone
$phoneValue = $number !== '' ? $number : null;
$user
->setAttribute('phone', $number)
->setAttribute('phone', $phoneValue)
->setAttribute('phoneVerification', false)
;
if (\strlen($number) !== 0) {
if ($number !== '') {
$target = $dbForProject->findOne('targets', [
Query::equal('identifier', [$number]),
]);
@@ -1558,20 +1579,24 @@ Http::patch('/v1/users/:userId/phone')
}
try {
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
$user = $dbForProject->updateDocument('users', $user->getId(), new Document([
'phone' => $phoneValue,
'phoneVerification' => $user->getAttribute('phoneVerification'),
]));
/**
* @var Document $oldTarget
*/
$oldTarget = $user->find('identifier', $oldPhone, 'targets');
if ($oldTarget instanceof Document && !$oldTarget->isEmpty()) {
if (\strlen($number) !== 0) {
$dbForProject->updateDocument('targets', $oldTarget->getId(), $oldTarget->setAttribute('identifier', $number));
if ($number !== '') {
$dbForProject->updateDocument('targets', $oldTarget->getId(), new Document(['identifier' => $number]));
$oldTarget->setAttribute('identifier', $number);
} else {
$dbForProject->deleteDocument('targets', $oldTarget->getId());
}
} else {
if (\strlen($number) !== 0) {
if ($number !== '') {
$target = $dbForProject->createDocument('targets', new Document([
'$permissions' => [
Permission::read(Role::user($user->getId())),
@@ -1630,7 +1655,7 @@ Http::patch('/v1/users/:userId/verification')
throw new Exception(Exception::USER_NOT_FOUND);
}
$user = $dbForProject->updateDocument('users', $user->getId(), $user->setAttribute('emailVerification', $emailVerification));
$user = $dbForProject->updateDocument('users', $user->getId(), new Document(['emailVerification' => $emailVerification]));
$queueForEvents->setParam('userId', $user->getId());
@@ -1668,7 +1693,7 @@ Http::patch('/v1/users/:userId/prefs')
throw new Exception(Exception::USER_NOT_FOUND);
}
$user = $dbForProject->updateDocument('users', $user->getId(), $user->setAttribute('prefs', $prefs));
$user = $dbForProject->updateDocument('users', $user->getId(), new Document(['prefs' => $prefs]));
$queueForEvents
->setParam('userId', $user->getId());
@@ -1768,7 +1793,13 @@ Http::patch('/v1/users/:userId/targets/:targetId')
$target->setAttribute('name', $name);
}
$target = $dbForProject->updateDocument('targets', $target->getId(), $target);
$target = $dbForProject->updateDocument('targets', $target->getId(), new Document([
'identifier' => $target->getAttribute('identifier'),
'expired' => $target->getAttribute('expired'),
'providerId' => $target->getAttribute('providerId'),
'providerInternalId' => $target->getAttribute('providerInternalId'),
'name' => $target->getAttribute('name'),
]));
$dbForProject->purgeCachedDocument('users', $user->getId());
$queueForEvents
@@ -1836,7 +1867,7 @@ Http::patch('/v1/users/:userId/mfa')
$user->setAttribute('mfa', $mfa);
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
$user = $dbForProject->updateDocument('users', $user->getId(), new Document(['mfa' => $user->getAttribute('mfa')]));
$queueForEvents->setParam('userId', $user->getId());
@@ -2024,7 +2055,7 @@ Http::patch('/v1/users/:userId/mfa/recovery-codes')
$mfaRecoveryCodes = Type::generateBackupCodes();
$user->setAttribute('mfaRecoveryCodes', $mfaRecoveryCodes);
$dbForProject->updateDocument('users', $user->getId(), $user);
$dbForProject->updateDocument('users', $user->getId(), new Document(['mfaRecoveryCodes' => $mfaRecoveryCodes]));
$queueForEvents->setParam('userId', $user->getId());
@@ -2096,7 +2127,7 @@ Http::put('/v1/users/:userId/mfa/recovery-codes')
$mfaRecoveryCodes = Type::generateBackupCodes();
$user->setAttribute('mfaRecoveryCodes', $mfaRecoveryCodes);
$dbForProject->updateDocument('users', $user->getId(), $user);
$dbForProject->updateDocument('users', $user->getId(), new Document(['mfaRecoveryCodes' => $mfaRecoveryCodes]));
$queueForEvents->setParam('userId', $user->getId());
+29 -10
View File
@@ -30,6 +30,8 @@ use Appwrite\Utopia\Response\Filters\V16 as ResponseV16;
use Appwrite\Utopia\Response\Filters\V17 as ResponseV17;
use Appwrite\Utopia\Response\Filters\V18 as ResponseV18;
use Appwrite\Utopia\Response\Filters\V19 as ResponseV19;
use Appwrite\Utopia\Response\Filters\V20 as ResponseV20;
use Appwrite\Utopia\Response\Filters\V21 as ResponseV21;
use Appwrite\Utopia\View;
use Executor\Executor;
use MaxMind\Db\Reader;
@@ -132,8 +134,9 @@ function router(Http $utopia, Database $dbForPlatform, callable $getProjectDB, S
if (!$project->isEmpty() && $project->getId() !== 'console') {
$accessedAt = $project->getAttribute('accessedAt', 0);
if (DateTime::formatTz(DateTime::addSeconds(new \DateTime(), -APP_PROJECT_ACCESS)) > $accessedAt) {
$project->setAttribute('accessedAt', DateTime::now());
$authorization->skip(fn () => $dbForPlatform->updateDocument('projects', $project->getId(), $project));
$authorization->skip(fn () => $dbForPlatform->updateDocument('projects', $project->getId(), new Document([
'accessedAt' => DateTime::now()
])));
}
/**
@@ -321,7 +324,7 @@ function router(Http $utopia, Database $dbForPlatform, callable $getProjectDB, S
};
$runtimes = Config::getParam($version === 'v2' ? 'runtimes-v2' : 'runtimes', []);
$spec = Config::getParam('specifications')[$resource->getAttribute('specification', APP_COMPUTE_SPECIFICATION_DEFAULT)];
$spec = Config::getParam('specifications')[$resource->getAttribute('runtimeSpecification', APP_COMPUTE_SPECIFICATION_DEFAULT)];
$runtime = match ($type) {
'function' => $runtimes[$resource->getAttribute('runtime')] ?? null,
@@ -551,6 +554,10 @@ function router(Http $utopia, Database $dbForPlatform, callable $getProjectDB, S
}
}
if (!empty($deployment->getAttribute('startCommand', ''))) {
$startCommand = 'cd /usr/local/server/src/function/ && ' . $deployment->getAttribute('startCommand', '');
}
$runtimeEntrypoint = match ($version) {
'v2' => '',
default => "cp /tmp/code.$extension /mnt/code/code.$extension && nohup helpers/start.sh \"$startCommand\"",
@@ -932,17 +939,23 @@ Http::init()
*/
$responseFormat = $request->getHeader('x-appwrite-response-format', System::getEnv('_APP_SYSTEM_RESPONSE_FORMAT', ''));
if ($responseFormat) {
if (version_compare($responseFormat, '1.4.0', '<')) {
$response->addFilter(new ResponseV16());
if (version_compare($responseFormat, '1.9.0', '<')) {
$response->addFilter(new ResponseV21());
}
if (version_compare($responseFormat, '1.5.0', '<')) {
$response->addFilter(new ResponseV17());
if (version_compare($responseFormat, '1.8.0', '<')) {
$response->addFilter(new ResponseV20());
}
if (version_compare($responseFormat, '1.7.0', '<')) {
$response->addFilter(new ResponseV19());
}
if (version_compare($responseFormat, '1.6.0', '<')) {
$response->addFilter(new ResponseV18());
}
if (version_compare($responseFormat, '1.7.0', '<')) {
$response->addFilter(new ResponseV19());
if (version_compare($responseFormat, '1.5.0', '<')) {
$response->addFilter(new ResponseV17());
}
if (version_compare($responseFormat, '1.4.0', '<')) {
$response->addFilter(new ResponseV16());
}
if (version_compare($responseFormat, APP_VERSION_STABLE, '>')) {
$warnings[] = "The current SDK is built for Appwrite " . $responseFormat . ". However, the current Appwrite server version is " . APP_VERSION_STABLE . ". Please downgrade your SDK to match the Appwrite version: https://appwrite.io/docs/sdks";
@@ -1386,6 +1399,9 @@ Http::error()
$sdk = $route?->getLabel("sdk", false);
$action = 'UNKNOWN_NAMESPACE.UNKNOWN.METHOD';
if (!empty($sdk)) {
if (\is_array($sdk)) {
$sdk = $sdk[0];
}
/** @var \Appwrite\SDK\Method $sdk */
$action = $sdk->getNamespace() . '.' . $sdk->getMethodName();
} elseif ($route === null) {
@@ -1649,7 +1665,10 @@ Http::get('/v1/ping')
->setAttribute('pingedAt', $pingedAt);
$authorization->skip(function () use ($dbForPlatform, $project) {
$dbForPlatform->updateDocument('projects', $project->getId(), $project);
$dbForPlatform->updateDocument('projects', $project->getId(), new Document([
'pingCount' => $project->getAttribute('pingCount'),
'pingedAt' => $project->getAttribute('pingedAt')
]));
});
$queueForEvents
+120 -95
View File
@@ -10,14 +10,16 @@ use Appwrite\Event\Delete;
use Appwrite\Event\Event;
use Appwrite\Event\Func;
use Appwrite\Event\Mail;
use Appwrite\Event\Message\Usage as UsageMessage;
use Appwrite\Event\Messaging;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Realtime;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Webhook;
use Appwrite\Extend\Exception;
use Appwrite\Extend\Exception as AppwriteException;
use Appwrite\Functions\EventProcessor;
use Appwrite\SDK\Method;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Request;
use Appwrite\Utopia\Response;
@@ -53,7 +55,7 @@ $parseLabel = function (string $label, array $responsePayload, array $requestPar
$replace = $parts[1] ?? '';
$params = match ($namespace) {
'user' => (array)$user,
'user' => (array) $user,
'request' => $requestParams,
default => $responsePayload,
};
@@ -61,13 +63,13 @@ $parseLabel = function (string $label, array $responsePayload, array $requestPar
if (array_key_exists($replace, $params)) {
$replacement = $params[$replace];
// Convert to string if it's not already a string
if (!is_string($replacement)) {
if (! is_string($replacement)) {
if (is_array($replacement)) {
$replacement = json_encode($replacement);
} elseif (is_object($replacement) && method_exists($replacement, '__toString')) {
$replacement = (string)$replacement;
$replacement = (string) $replacement;
} elseif (is_scalar($replacement)) {
$replacement = (string)$replacement;
$replacement = (string) $replacement;
} else {
throw new Exception(Exception::GENERAL_SERVER_ERROR, "The server encountered an error while parsing the label: $label. Please create an issue on GitHub to allow us to investigate further https://github.com/appwrite/appwrite/issues/new/choose");
}
@@ -75,6 +77,7 @@ $parseLabel = function (string $label, array $responsePayload, array $requestPar
$label = \str_replace($find, $replacement, $label);
}
}
return $label;
};
@@ -160,7 +163,7 @@ Http::init()
$scopes = $roles[$role]['scopes'];
// Step 5: API Key Authentication
if (!empty($apiKey)) {
if (! empty($apiKey)) {
// Check if key is expired
if ($apiKey->isExpired()) {
throw new Exception(Exception::PROJECT_KEY_EXPIRED);
@@ -170,7 +173,6 @@ Http::init()
$role = $apiKey->getRole();
$scopes = $apiKey->getScopes();
// Handle special app role case
if ($apiKey->getRole() === User::ROLE_APPS) {
// Disable authorization checks for project API keys
@@ -193,19 +195,19 @@ Http::init()
// For standard keys, update last accessed time
if (\in_array($apiKey->getType(), [API_KEY_STANDARD, API_KEY_ORGANIZATION, API_KEY_ACCOUNT])) {
$dbKey = null;
if (!empty($apiKey->getProjectId())) {
if (! empty($apiKey->getProjectId())) {
$dbKey = $project->find(
key: 'secret',
find: $request->getHeader('x-appwrite-key', ''),
subject: 'keys'
);
} elseif (!empty($apiKey->getUserId())) {
} elseif (! empty($apiKey->getUserId())) {
$dbKey = $user->find(
key: 'secret',
find: $request->getHeader('x-appwrite-key', ''),
subject: 'keys'
);
} elseif (!empty($apiKey->getTeamId())) {
} elseif (! empty($apiKey->getTeamId())) {
$dbKey = $team->find(
key: 'secret',
find: $request->getHeader('x-appwrite-key', ''),
@@ -213,7 +215,7 @@ Http::init()
);
}
if (!$dbKey) {
if (! $dbKey) {
throw new Exception(Exception::USER_UNAUTHORIZED);
}
@@ -231,7 +233,7 @@ Http::init()
if ($sdk !== 'UNKNOWN' && $sdkValidator->isValid($sdk)) {
$sdks = $dbKey->getAttribute('sdks', []);
if (!in_array($sdk, $sdks)) {
if (! in_array($sdk, $sdks)) {
$sdks[] = $sdk;
$updates->setAttribute('sdks', $sdks);
@@ -239,14 +241,14 @@ Http::init()
}
}
if (!$updates->isEmpty()) {
if (! $updates->isEmpty()) {
$dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->updateDocument('keys', $dbKey->getId(), $updates));
if (!empty($apiKey->getProjectId())) {
if (! empty($apiKey->getProjectId())) {
$dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->purgeCachedDocument('projects', $project->getId()));
} elseif (!empty($apiKey->getUserId())) {
} elseif (! empty($apiKey->getUserId())) {
$dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->purgeCachedDocument('users', $user->getId()));
} elseif (!empty($apiKey->getTeamId())) {
} elseif (! empty($apiKey->getTeamId())) {
$dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->purgeCachedDocument('teams', $team->getId()));
}
}
@@ -283,7 +285,7 @@ Http::init()
}
}
} // Admin User Authentication
elseif (($project->getId() === 'console' && !$team->isEmpty() && !$user->isEmpty()) || ($project->getId() !== 'console' && !$user->isEmpty() && $mode === APP_MODE_ADMIN)) {
elseif (($project->getId() === 'console' && ! $team->isEmpty() && ! $user->isEmpty()) || ($project->getId() !== 'console' && ! $user->isEmpty() && $mode === APP_MODE_ADMIN)) {
$teamId = $team->getId();
$adminRoles = [];
$memberships = $user->getAttribute('memberships', []);
@@ -308,7 +310,7 @@ Http::init()
// Useful for those who have project-specific roles but don't have team-wide role.
$scopes = ['teams.read', 'projects.read'];
foreach ($adminRoles as $adminRole) {
$isTeamWideRole = !str_starts_with($adminRole, 'project-');
$isTeamWideRole = ! str_starts_with($adminRole, 'project-');
$isProjectSpecificRole = $projectId !== 'console' && str_starts_with($adminRole, 'project-' . $projectId);
if ($isTeamWideRole || $isProjectSpecificRole) {
@@ -346,34 +348,39 @@ Http::init()
* But, for actions on resources (sites, functions, etc.) in a non-console project, we explicitly check
* whether the admin user has necessary permission on the project (sites, functions, etc. don't have permissions associated to them).
*/
if (empty($apiKey) && !$user->isEmpty() && $project->getId() !== 'console' && $mode === APP_MODE_ADMIN) {
if (empty($apiKey) && ! $user->isEmpty() && $project->getId() !== 'console' && $mode === APP_MODE_ADMIN) {
$input = new Input(Database::PERMISSION_READ, $project->getPermissionsByType(Database::PERMISSION_READ));
$initialStatus = $authorization->getStatus();
$authorization->enable();
if (!$authorization->isValid($input)) {
if (! $authorization->isValid($input)) {
throw new Exception(Exception::PROJECT_NOT_FOUND);
}
$authorization->setStatus($initialStatus);
}
// Step 6: Update project and user last activity
if (!$project->isEmpty() && $project->getId() !== 'console') {
if (! $project->isEmpty() && $project->getId() !== 'console') {
$accessedAt = $project->getAttribute('accessedAt', 0);
if (DateTime::formatTz(DateTime::addSeconds(new \DateTime(), -APP_PROJECT_ACCESS)) > $accessedAt) {
$project->setAttribute('accessedAt', DateTime::now());
$authorization->skip(fn () => $dbForPlatform->updateDocument('projects', $project->getId(), $project));
$authorization->skip(fn () => $dbForPlatform->updateDocument('projects', $project->getId(), new Document([
'accessedAt' => DateTime::now()
])));
}
}
if (!empty($user->getId())) {
if (! empty($user->getId())) {
$accessedAt = $user->getAttribute('accessedAt', 0);
if (DateTime::formatTz(DateTime::addSeconds(new \DateTime(), -APP_USER_ACCESS)) > $accessedAt) {
$user->setAttribute('accessedAt', DateTime::now());
if ($project->getId() !== 'console' && APP_MODE_ADMIN !== $mode) {
$dbForProject->updateDocument('users', $user->getId(), $user);
if ($project->getId() !== 'console' && $mode !== APP_MODE_ADMIN) {
$dbForProject->updateDocument('users', $user->getId(), new Document([
'accessedAt' => $user->getAttribute('accessedAt')
]));
} else {
$authorization->skip(fn () => $dbForPlatform->updateDocument('users', $user->getId(), $user));
$authorization->skip(fn () => $dbForPlatform->updateDocument('users', $user->getId(), new Document([
'accessedAt' => $user->getAttribute('accessedAt')
])));
}
}
}
@@ -390,26 +397,26 @@ Http::init()
$method = $method[0];
}
if (!empty($method)) {
if (! empty($method)) {
$namespace = $method->getNamespace();
if (
array_key_exists($namespace, $project->getAttribute('services', []))
&& !$project->getAttribute('services', [])[$namespace]
&& !(User::isPrivileged($authorization->getRoles()) || User::isApp($authorization->getRoles()))
&& ! $project->getAttribute('services', [])[$namespace]
&& ! (User::isPrivileged($authorization->getRoles()) || User::isApp($authorization->getRoles()))
) {
throw new Exception(Exception::GENERAL_SERVICE_DISABLED);
}
}
// Step 9: Validate scope permissions
$allowed = (array)$route->getLabel('scope', 'none');
$allowed = (array) $route->getLabel('scope', 'none');
if (empty(\array_intersect($allowed, $scopes))) {
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE, $user->getAttribute('email', 'User') . ' (role: ' . \strtolower($roles[$role]['label']) . ') missing scopes (' . \json_encode($allowed) . ')');
}
// Step 10: Check if user is blocked
if (false === $user->getAttribute('status')) { // Account is blocked
if ($user->getAttribute('status') === false) { // Account is blocked
throw new Exception(Exception::USER_BLOCKED);
}
@@ -427,7 +434,7 @@ Http::init()
$minimumFactors = ($mfaEnabled && $hasMoreFactors) ? 2 : 1;
// Step 13: Handle Multi-Factor Authentication
if (!in_array('mfa', $route->getGroups())) {
if (! in_array('mfa', $route->getGroups())) {
if ($session && \count($session->getAttribute('factors', [])) < $minimumFactors) {
throw new Exception(Exception::USER_MORE_FACTORS_REQUIRED);
}
@@ -447,7 +454,7 @@ Http::init()
->inject('queueForDeletes')
->inject('queueForDatabase')
->inject('queueForBuilds')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForFunctions')
->inject('queueForMails')
->inject('dbForProject')
@@ -460,14 +467,14 @@ Http::init()
->inject('telemetry')
->inject('platform')
->inject('authorization')
->action(function (Http $utopia, Request $request, Response $response, Document $project, Document $user, Event $queueForEvents, Messaging $queueForMessaging, Audit $queueForAudits, Delete $queueForDeletes, EventDatabase $queueForDatabase, Build $queueForBuilds, StatsUsage $queueForStatsUsage, Func $queueForFunctions, Mail $queueForMails, Database $dbForProject, callable $timelimit, Document $resourceToken, string $mode, ?Key $apiKey, array $plan, Document $devKey, Telemetry $telemetry, array $platform, Authorization $authorization) {
->action(function (Http $utopia, Request $request, Response $response, Document $project, Document $user, Event $queueForEvents, Messaging $queueForMessaging, Audit $queueForAudits, Delete $queueForDeletes, EventDatabase $queueForDatabase, Build $queueForBuilds, Context $usage, Func $queueForFunctions, Mail $queueForMails, Database $dbForProject, callable $timelimit, Document $resourceToken, string $mode, ?Key $apiKey, array $plan, Document $devKey, Telemetry $telemetry, array $platform, Authorization $authorization) {
$route = $utopia->getRoute();
if (
array_key_exists('rest', $project->getAttribute('apis', []))
&& !$project->getAttribute('apis', [])['rest']
&& !(User::isPrivileged($authorization->getRoles()) || User::isApp($authorization->getRoles()))
&& ! $project->getAttribute('apis', [])['rest']
&& ! (User::isPrivileged($authorization->getRoles()) || User::isApp($authorization->getRoles()))
) {
throw new AppwriteException(AppwriteException::GENERAL_API_DISABLED);
}
@@ -479,7 +486,7 @@ Http::init()
$abuseKeyLabel = $route->getLabel('abuse-key', 'url:{url},ip:{ip}');
$timeLimitArray = [];
$abuseKeyLabel = (!is_array($abuseKeyLabel)) ? [$abuseKeyLabel] : $abuseKeyLabel;
$abuseKeyLabel = (! is_array($abuseKeyLabel)) ? [$abuseKeyLabel] : $abuseKeyLabel;
foreach ($abuseKeyLabel as $abuseKey) {
$start = $request->getContentRangeStart();
@@ -492,7 +499,7 @@ Http::init()
->setParam('{ip}', $request->getIP())
->setParam('{url}', $request->getHostname() . $route->getPath())
->setParam('{method}', $request->getMethod())
->setParam('{chunkId}', (int)($start / ($end + 1 - $start)));
->setParam('{chunkId}', (int) ($start / ($end + 1 - $start)));
$timeLimitArray[] = $timeLimit;
}
@@ -504,7 +511,7 @@ Http::init()
foreach ($timeLimitArray as $timeLimit) {
foreach ($request->getParams() as $key => $value) { // Set request params as potential abuse keys
if (!empty($value)) {
if (! empty($value)) {
$timeLimit->setParam('{param-' . $key . '}', (\is_array($value)) ? \json_encode($value) : $value);
}
}
@@ -527,8 +534,8 @@ Http::init()
if (
$enabled // Abuse is enabled
&& !$isAppUser // User is not API key
&& !$isPrivilegedUser // User is not an admin
&& ! $isAppUser // User is not API key
&& ! $isPrivilegedUser // User is not an admin
&& $devKey->isEmpty() // request doesn't not contain development key
&& $abuse->check() // Route is rate-limited
) {
@@ -557,19 +564,13 @@ Http::init()
->setProject($project);
/* If a session exists, use the user associated with the session */
if (!$user->isEmpty()) {
if (! $user->isEmpty()) {
$userClone = clone $user;
// $user doesn't support `type` and can cause unintended effects.
$userClone->setAttribute('type', ACTIVITY_TYPE_USER);
$queueForAudits->setUser($userClone);
}
if (!empty($apiKey) && !empty($apiKey->getDisabledMetrics())) {
foreach ($apiKey->getDisabledMetrics() as $key) {
$queueForStatsUsage->disableMetric($key);
}
}
/* Auto-set projects */
$queueForDeletes->setProject($project);
$queueForDatabase->setProject($project);
@@ -583,73 +584,70 @@ Http::init()
$queueForBuilds->setPlatform($platform);
$queueForMails->setPlatform($platform);
$useCache = $route->getLabel('cache', false);
$storageCacheOperationsCounter = $telemetry->createCounter('storage.cache.operations.load');
if ($useCache) {
$route = $utopia->match($request);
$isImageTransformation = $route->getPath() === '/v1/storage/buckets/:bucketId/files/:fileId/preview';
$isDisabled = isset($plan['imageTransformations']) && $plan['imageTransformations'] === -1 && !User::isPrivileged($authorization->getRoles());
$isDisabled = isset($plan['imageTransformations']) && $plan['imageTransformations'] === -1 && ! User::isPrivileged($authorization->getRoles());
$key = $request->cacheIdentifier();
$cacheLog = $authorization->skip(fn () => $dbForProject->getDocument('cache', $key));
$cacheLog = $authorization->skip(fn () => $dbForProject->getDocument('cache', $key));
$cache = new Cache(
new Filesystem(APP_STORAGE_CACHE . DIRECTORY_SEPARATOR . 'app-' . $project->getId())
);
$timestamp = 60 * 60 * 24 * 180; // Temporarily increase the TTL to 180 day to ensure files in the cache are still fetched.
$data = $cache->load($key, $timestamp);
if (!empty($data) && !$cacheLog->isEmpty()) {
$usageMetric = $route->getLabel('usage.metric', null);
if ($usageMetric === METRIC_AVATARS_SCREENSHOTS_GENERATED) {
$queueForStatsUsage->disableMetric(METRIC_AVATARS_SCREENSHOTS_GENERATED);
}
if (! empty($data) && ! $cacheLog->isEmpty()) {
$parts = explode('/', $cacheLog->getAttribute('resourceType', ''));
$type = $parts[0] ?? null;
if ($type === 'bucket' && (!$isImageTransformation || !$isDisabled)) {
if ($type === 'bucket' && (! $isImageTransformation || ! $isDisabled)) {
$bucketId = $parts[1] ?? null;
$bucket = $authorization->skip(fn () => $dbForProject->getDocument('buckets', $bucketId));
$isToken = !$resourceToken->isEmpty() && $resourceToken->getAttribute('bucketInternalId') === $bucket->getSequence();
$isToken = ! $resourceToken->isEmpty() && $resourceToken->getAttribute('bucketInternalId') === $bucket->getSequence();
$isPrivilegedUser = User::isPrivileged($authorization->getRoles());
if ($bucket->isEmpty() || (!$bucket->getAttribute('enabled') && !$isAppUser && !$isPrivilegedUser)) {
if ($bucket->isEmpty() || (! $bucket->getAttribute('enabled') && ! $isAppUser && ! $isPrivilegedUser)) {
throw new Exception(Exception::STORAGE_BUCKET_NOT_FOUND);
}
if (!$bucket->getAttribute('transformations', true) && !$isAppUser && !$isPrivilegedUser) {
if (! $bucket->getAttribute('transformations', true) && ! $isAppUser && ! $isPrivilegedUser) {
throw new Exception(Exception::STORAGE_BUCKET_TRANSFORMATIONS_DISABLED);
}
$fileSecurity = $bucket->getAttribute('fileSecurity', false);
$valid = $authorization->isValid(new Input(Database::PERMISSION_READ, $bucket->getRead()));
if (!$fileSecurity && !$valid && !$isToken) {
if (! $fileSecurity && ! $valid && ! $isToken) {
throw new Exception(Exception::USER_UNAUTHORIZED);
}
$parts = explode('/', $cacheLog->getAttribute('resource'));
$fileId = $parts[1] ?? null;
if ($fileSecurity && !$valid && !$isToken) {
if ($fileSecurity && ! $valid && ! $isToken) {
$file = $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId);
} else {
$file = $authorization->skip(fn () => $dbForProject->getDocument('bucket_' . $bucket->getSequence(), $fileId));
}
if (!$resourceToken->isEmpty() && $resourceToken->getAttribute('fileInternalId') !== $file->getSequence()) {
if (! $resourceToken->isEmpty() && $resourceToken->getAttribute('fileInternalId') !== $file->getSequence()) {
throw new Exception(Exception::USER_UNAUTHORIZED);
}
if ($file->isEmpty()) {
throw new Exception(Exception::STORAGE_FILE_NOT_FOUND);
}
//Do not update transformedAt if it's a console user
if (!User::isPrivileged($authorization->getRoles())) {
// Do not update transformedAt if it's a console user
if (! User::isPrivileged($authorization->getRoles())) {
$transformedAt = $file->getAttribute('transformedAt', '');
if (DateTime::formatTz(DateTime::addSeconds(new \DateTime(), -APP_PROJECT_ACCESS)) > $transformedAt) {
$file->setAttribute('transformedAt', DateTime::now());
$authorization->skip(fn () => $dbForProject->updateDocument('bucket_' . $file->getAttribute('bucketInternalId'), $file->getId(), $file));
$authorization->skip(fn () => $dbForProject->updateDocument('bucket_' . $file->getAttribute('bucketInternalId'), $file->getId(), new Document([
'transformedAt' => $file->getAttribute('transformedAt')
])));
}
}
}
@@ -659,7 +657,7 @@ Http::init()
->addHeader('X-Appwrite-Cache', 'hit')
->setContentType($cacheLog->getAttribute('mimeType'));
$storageCacheOperationsCounter->add(1, ['result' => 'hit']);
if (!$isImageTransformation || !$isDisabled) {
if (! $isImageTransformation || ! $isDisabled) {
$response->send($data);
}
} else {
@@ -682,7 +680,7 @@ Http::init()
return;
}
if (!$user->isEmpty()) {
if (! $user->isEmpty()) {
throw new Exception(Exception::USER_SESSION_ALREADY_EXISTS);
}
});
@@ -736,7 +734,8 @@ Http::shutdown()
->inject('user')
->inject('queueForEvents')
->inject('queueForAudits')
->inject('queueForStatsUsage')
->inject('usage')
->inject('publisherForUsage')
->inject('queueForDeletes')
->inject('queueForDatabase')
->inject('queueForBuilds')
@@ -749,11 +748,12 @@ Http::shutdown()
->inject('timelimit')
->inject('eventProcessor')
->inject('bus')
->action(function (Http $utopia, Request $request, Response $response, Document $project, User $user, Event $queueForEvents, Audit $queueForAudits, StatsUsage $queueForStatsUsage, Delete $queueForDeletes, EventDatabase $queueForDatabase, Build $queueForBuilds, Messaging $queueForMessaging, Func $queueForFunctions, Event $queueForWebhooks, Realtime $queueForRealtime, Database $dbForProject, Authorization $authorization, callable $timelimit, EventProcessor $eventProcessor, Bus $bus) use ($parseLabel) {
->inject('apiKey')
->action(function (Http $utopia, Request $request, Response $response, Document $project, User $user, Event $queueForEvents, Audit $queueForAudits, Context $usage, UsagePublisher $publisherForUsage, Delete $queueForDeletes, EventDatabase $queueForDatabase, Build $queueForBuilds, Messaging $queueForMessaging, Func $queueForFunctions, Event $queueForWebhooks, Realtime $queueForRealtime, Database $dbForProject, Authorization $authorization, callable $timelimit, EventProcessor $eventProcessor, Bus $bus, ?Key $apiKey) use ($parseLabel) {
$responsePayload = $response->getPayload();
if (!empty($queueForEvents->getEvent())) {
if (! empty($queueForEvents->getEvent())) {
if (empty($queueForEvents->getPayload())) {
$queueForEvents->setPayload($responsePayload);
}
@@ -775,7 +775,7 @@ Http::shutdown()
}
// Only trigger functions if there are matching function events
if (!empty($functionsEvents)) {
if (! empty($functionsEvents)) {
foreach ($generatedEvents as $event) {
if (isset($functionsEvents[$event])) {
$queueForFunctions
@@ -787,7 +787,7 @@ Http::shutdown()
}
// Only trigger webhooks if there are matching webhook events
if (!empty($webhooksEvents)) {
if (! empty($webhooksEvents)) {
foreach ($generatedEvents as $event) {
if (isset($webhooksEvents[$event])) {
$queueForWebhooks
@@ -811,7 +811,7 @@ Http::shutdown()
if ($abuseEnabled && \count($abuseResetCode) > 0 && \in_array($response->getStatusCode(), $abuseResetCode)) {
$abuseKeyLabel = $route->getLabel('abuse-key', 'url:{url},ip:{ip}');
$abuseKeyLabel = (!is_array($abuseKeyLabel)) ? [$abuseKeyLabel] : $abuseKeyLabel;
$abuseKeyLabel = (! is_array($abuseKeyLabel)) ? [$abuseKeyLabel] : $abuseKeyLabel;
foreach ($abuseKeyLabel as $abuseKey) {
$start = $request->getContentRangeStart();
@@ -824,10 +824,10 @@ Http::shutdown()
->setParam('{ip}', $request->getIP())
->setParam('{url}', $request->getHostname() . $route->getPath())
->setParam('{method}', $request->getMethod())
->setParam('{chunkId}', (int)($start / ($end + 1 - $start)));
->setParam('{chunkId}', (int) ($start / ($end + 1 - $start)));
foreach ($request->getParams() as $key => $value) { // Set request params as potential abuse keys
if (!empty($value)) {
if (! empty($value)) {
$timeLimit->setParam('{param-' . $key . '}', (\is_array($value)) ? \json_encode($value) : $value);
}
}
@@ -841,14 +841,14 @@ Http::shutdown()
* Audit labels
*/
$pattern = $route->getLabel('audits.resource', null);
if (!empty($pattern)) {
if (! empty($pattern)) {
$resource = $parseLabel($pattern, $responsePayload, $requestParams, $user);
if (!empty($resource) && $resource !== $pattern) {
if (! empty($resource) && $resource !== $pattern) {
$queueForAudits->setResource($resource);
}
}
if (!$user->isEmpty()) {
if (! $user->isEmpty()) {
$userClone = clone $user;
// $user doesn't support `type` and can cause unintended effects.
$userClone->setAttribute('type', ACTIVITY_TYPE_USER);
@@ -874,13 +874,13 @@ Http::shutdown()
$queueForAudits->setUser($user);
}
if (!empty($queueForAudits->getResource()) && !$queueForAudits->getUser()->isEmpty()) {
if (! empty($queueForAudits->getResource()) && ! $queueForAudits->getUser()->isEmpty()) {
/**
* audits.payload is switched to default true
* in order to auto audit payload for all endpoints
*/
$pattern = $route->getLabel('audits.payload', true);
if (!empty($pattern)) {
if (! empty($pattern)) {
$queueForAudits->setPayload($responsePayload);
}
@@ -891,19 +891,19 @@ Http::shutdown()
$queueForAudits->trigger();
}
if (!empty($queueForDeletes->getType())) {
if (! empty($queueForDeletes->getType())) {
$queueForDeletes->trigger();
}
if (!empty($queueForDatabase->getType())) {
if (! empty($queueForDatabase->getType())) {
$queueForDatabase->trigger();
}
if (!empty($queueForBuilds->getType())) {
if (! empty($queueForBuilds->getType())) {
$queueForBuilds->trigger();
}
if (!empty($queueForMessaging->getType())) {
if (! empty($queueForMessaging->getType())) {
$queueForMessaging->trigger();
}
@@ -912,14 +912,14 @@ Http::shutdown()
if ($useCache) {
$resource = $resourceType = null;
$data = $response->getPayload();
if (!empty($data['payload'])) {
if (! empty($data['payload'])) {
$pattern = $route->getLabel('cache.resource', null);
if (!empty($pattern)) {
if (! empty($pattern)) {
$resource = $parseLabel($pattern, $responsePayload, $requestParams, $user);
}
$pattern = $route->getLabel('cache.resourceType', null);
if (!empty($pattern)) {
if (! empty($pattern)) {
$resourceType = $parseLabel($pattern, $responsePayload, $requestParams, $user);
}
@@ -929,7 +929,7 @@ Http::shutdown()
$key = $request->cacheIdentifier();
$signature = md5($data['payload']);
$cacheLog = $authorization->skip(fn () => $dbForProject->getDocument('cache', $key));
$cacheLog = $authorization->skip(fn () => $dbForProject->getDocument('cache', $key));
$accessedAt = $cacheLog->getAttribute('accessedAt', 0);
$now = DateTime::now();
if ($cacheLog->isEmpty()) {
@@ -948,7 +948,9 @@ Http::shutdown()
}
} elseif (DateTime::formatTz(DateTime::addSeconds(new \DateTime(), -APP_CACHE_UPDATE)) > $accessedAt) {
$cacheLog->setAttribute('accessedAt', $now);
$authorization->skip(fn () => $dbForProject->updateDocument('cache', $cacheLog->getId(), $cacheLog));
$authorization->skip(fn () => $dbForProject->updateDocument('cache', $cacheLog->getId(), new Document([
'accessedAt' => $cacheLog->getAttribute('accessedAt')
])));
// Overwrite the file every APP_CACHE_UPDATE seconds to update the file modified time that is used in the TTL checks in cache->load()
$cache->save($key, $data['payload']);
}
@@ -960,7 +962,7 @@ Http::shutdown()
}
if ($project->getId() !== 'console') {
if (!User::isPrivileged($authorization->getRoles())) {
if (! User::isPrivileged($authorization->getRoles())) {
$bus->dispatch(new RequestCompleted(
project: $project->getArrayCopy(),
request: $request,
@@ -968,9 +970,32 @@ Http::shutdown()
));
}
$queueForStatsUsage
->setProject($project)
->trigger();
// Publish usage metrics if context has data
if (! $usage->isEmpty()) {
$metrics = $usage->getMetrics();
// Filter out API key disabled metrics using suffix pattern matching
$disabledMetrics = $apiKey?->getDisabledMetrics() ?? [];
if (! empty($disabledMetrics)) {
$metrics = array_values(array_filter($metrics, function ($metric) use ($disabledMetrics) {
foreach ($disabledMetrics as $pattern) {
if (str_ends_with($metric['key'], $pattern)) {
return false;
}
}
return true;
}));
}
$message = new UsageMessage(
project: $project,
metrics: $metrics,
reduce: $usage->getReduce()
);
$publisherForUsage->enqueue($message);
}
}
});
+3
View File
@@ -581,6 +581,9 @@ $http->on(Constant::EVENT_REQUEST, function (SwooleRequest $swooleRequest, Swool
$action = 'UNKNOWN_NAMESPACE.UNKNOWN.METHOD';
if (!empty($sdk)) {
if (\is_array($sdk)) {
$sdk = $sdk[0];
}
/** @var Appwrite\SDK\Method $sdk */
$action = $sdk->getNamespace() . '.' . $sdk->getMethodName();
} elseif ($route === null) {
+7
View File
@@ -93,6 +93,7 @@ const APP_SOCIAL_YOUTUBE = 'https://www.youtube.com/c/appwrite?sub_confirmation=
const APP_COMPUTE_CPUS_DEFAULT = 0.5;
const APP_COMPUTE_MEMORY_DEFAULT = 512;
const APP_COMPUTE_SPECIFICATION_DEFAULT = Specification::S_1VCPU_512MB;
const APP_COMPUTE_DEPLOYMENT_MAX_RETENTION = 100 * 365; // 100 years
const APP_SDK_PLATFORM_SERVER = 'server';
const APP_SDK_PLATFORM_CLIENT = 'client';
const APP_SDK_PLATFORM_CONSOLE = 'console';
@@ -361,6 +362,12 @@ const METRIC_AVATARS_SCREENSHOTS_GENERATED = 'avatars.screenshotsGenerated';
const METRIC_FUNCTIONS_RUNTIME = 'functions.runtimes.{runtime}';
const METRIC_SITES_FRAMEWORK = 'sites.frameworks.{framework}';
// Realtime metrics
const METRIC_REALTIME_CONNECTIONS = 'realtime.connections';
const METRIC_REALTIME_CONNECTIONS_MESSAGES_SENT = 'realtime.messages.sent';
const METRIC_REALTIME_INBOUND = 'realtime.inbound';
const METRIC_REALTIME_OUTBOUND = 'realtime.outbound';
// Resource types
const RESOURCE_TYPE_PROJECTS = 'projects';
const RESOURCE_TYPE_FUNCTIONS = 'functions';
+1
View File
@@ -420,6 +420,7 @@ $register->set('smtp', function () {
$mail->Password = $password;
$mail->SMTPSecure = System::getEnv('_APP_SMTP_SECURE', '');
$mail->SMTPAutoTLS = false;
$mail->SMTPKeepAlive = true;
$mail->CharSet = 'UTF-8';
$mail->Timeout = 10; /* Connection timeout */
$mail->getSMTPInstance()->Timelimit = 30; /* Timeout for each individual SMTP command (e.g. HELO, EHLO, etc.) */
+130 -114
View File
@@ -14,10 +14,10 @@ use Appwrite\Event\Func;
use Appwrite\Event\Mail;
use Appwrite\Event\Messaging;
use Appwrite\Event\Migration;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Realtime;
use Appwrite\Event\Screenshot;
use Appwrite\Event\StatsResources;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Webhook;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
@@ -26,6 +26,7 @@ use Appwrite\Network\Cors;
use Appwrite\Network\Platform;
use Appwrite\Network\Validator\Origin;
use Appwrite\Network\Validator\Redirect;
use Appwrite\Usage\Context as UsageContext;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Request;
use Appwrite\Utopia\Response;
@@ -57,6 +58,7 @@ use Utopia\Logger\Log;
use Utopia\Pools\Group;
use Utopia\Queue\Broker\Pool as BrokerPool;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
use Utopia\Storage\Device;
use Utopia\Storage\Device\AWS;
use Utopia\Storage\Device\Backblaze;
@@ -88,6 +90,7 @@ Http::setResource('register', fn () => $register);
Http::setResource('locale', function () {
$locale = new Locale(System::getEnv('_APP_LOCALE', 'en'));
$locale->setFallback(System::getEnv('_APP_LOCALE', 'en'));
return $locale;
});
@@ -108,9 +111,6 @@ Http::setResource('publisherFunctions', function (Publisher $publisher) {
Http::setResource('publisherMigrations', function (Publisher $publisher) {
return $publisher;
}, ['publisher']);
Http::setResource('publisherStatsUsage', function (Publisher $publisher) {
return $publisher;
}, ['publisher']);
Http::setResource('publisherMails', function (Publisher $publisher) {
return $publisher;
}, ['publisher']);
@@ -150,9 +150,13 @@ Http::setResource('queueForWebhooks', function (Publisher $publisher) {
Http::setResource('queueForRealtime', function () {
return new Realtime();
}, []);
Http::setResource('queueForStatsUsage', function (Publisher $publisher) {
return new StatsUsage($publisher);
}, ['publisher']);
Http::setResource('usage', function () {
return new UsageContext();
}, []);
Http::setResource('publisherForUsage', fn (Publisher $publisher) => new UsagePublisher(
$publisher,
new Queue(System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME))
), ['publisher']);
Http::setResource('queueForAudits', function (Publisher $publisher) {
return new AuditEvent($publisher);
}, ['publisher']);
@@ -186,14 +190,14 @@ Http::setResource('allowedHostnames', function (array $platform, Document $proje
$allowed = [...($platform['hostnames'] ?? [])];
/* Add platform configured hostnames */
if (!$project->isEmpty() && $project->getId() !== 'console') {
if (! $project->isEmpty() && $project->getId() !== 'console') {
$platforms = $project->getAttribute('platforms', []);
$hostnames = Platform::getHostnames($platforms);
$allowed = [...$allowed, ...$hostnames];
}
/* Add the request hostname if a dev key is found */
if (!$devKey->isEmpty()) {
if (! $devKey->isEmpty()) {
$allowed[] = $request->getHostname();
}
@@ -211,12 +215,12 @@ Http::setResource('allowedHostnames', function (array $platform, Document $proje
}
/* Allow the request origin of rule */
if (!$rule->isEmpty() && !empty($rule->getAttribute('domain', ''))) {
if (! $rule->isEmpty() && ! empty($rule->getAttribute('domain', ''))) {
$allowed[] = $rule->getAttribute('domain', '');
}
/* Allow the request origin if a dev key is found */
if (!$devKey->isEmpty() && !empty($hostname)) {
if (! $devKey->isEmpty() && ! empty($hostname)) {
$allowed[] = $hostname;
}
@@ -229,7 +233,7 @@ Http::setResource('allowedHostnames', function (array $platform, Document $proje
Http::setResource('allowedSchemes', function (array $platform, Document $project) {
$allowed = [...($platform['schemas'] ?? [])];
if (!$project->isEmpty() && $project->getId() !== 'console') {
if (! $project->isEmpty() && $project->getId() !== 'console') {
/* Add hardcoded schemes */
$allowed[] = 'exp';
$allowed[] = 'appwrite-callback-' . $project->getId();
@@ -273,7 +277,7 @@ Http::setResource('rule', function (Request $request, Database $dbForPlatform, D
// Temporary implementation until custom wildcard domains are an official feature
// Allow trusted projects; Used for Console (website) previews
if (!$permitsCurrentProject && !$rule->isEmpty() && !empty($rule->getAttribute('projectId', ''))) {
if (! $permitsCurrentProject && ! $rule->isEmpty() && ! empty($rule->getAttribute('projectId', ''))) {
$trustedProjects = [];
foreach (\explode(',', System::getEnv('_APP_CONSOLE_TRUSTED_PROJECTS', '')) as $trustedProject) {
if (empty($trustedProject)) {
@@ -286,7 +290,7 @@ Http::setResource('rule', function (Request $request, Database $dbForPlatform, D
}
}
if (!$permitsCurrentProject) {
if (! $permitsCurrentProject) {
return new Document();
}
@@ -309,16 +313,18 @@ Http::setResource('cors', function (array $allowedHostnames) {
}, ['allowedHostnames']);
Http::setResource('originValidator', function (Document $devKey, array $allowedHostnames, array $allowedSchemes) {
if (!$devKey->isEmpty()) {
if (! $devKey->isEmpty()) {
return new URL();
}
return new Origin($allowedHostnames, $allowedSchemes);
}, ['devKey', 'allowedHostnames', 'allowedSchemes']);
Http::setResource('redirectValidator', function (Document $devKey, array $allowedHostnames, array $allowedSchemes) {
if (!$devKey->isEmpty()) {
if (! $devKey->isEmpty()) {
return new URL();
}
return new Redirect($allowedHostnames, $allowedSchemes);
}, ['devKey', 'allowedHostnames', 'allowedSchemes']);
@@ -342,12 +348,11 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
* overwriting the previous value.
* 7. If account API key is passed, use user of the account API key as long as user ID header matches too
*/
$authorization->setDefaultStatus(true);
$store->setKey('a_session_' . $project->getId());
if (APP_MODE_ADMIN === $mode) {
if ($mode === APP_MODE_ADMIN) {
$store->setKey('a_session_' . $console->getId());
}
@@ -362,7 +367,7 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
if (empty($store->getProperty('id', '')) && empty($store->getProperty('secret', ''))) {
$sessionHeader = $request->getHeader('x-appwrite-session', '');
if (!empty($sessionHeader)) {
if (! empty($sessionHeader)) {
$store->decode($sessionHeader);
}
}
@@ -382,14 +387,14 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
}
$user = null;
if (APP_MODE_ADMIN === $mode) {
if ($mode === APP_MODE_ADMIN) {
/** @var User $user */
$user = $dbForPlatform->getDocument('users', $store->getProperty('id', ''));
} else {
if ($project->isEmpty()) {
$user = new User([]);
} else {
if (!empty($store->getProperty('id', ''))) {
if (! empty($store->getProperty('id', ''))) {
if ($project->getId() === 'console') {
/** @var User $user */
$user = $dbForPlatform->getDocument('users', $store->getProperty('id', ''));
@@ -402,16 +407,16 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
}
if (
!$user ||
! $user ||
$user->isEmpty() // Check a document has been found in the DB
|| !$user->sessionVerify($store->getProperty('secret', ''), $proofForToken)
|| ! $user->sessionVerify($store->getProperty('secret', ''), $proofForToken)
) { // Validate user has valid login token
$user = new User([]);
}
$authJWT = $request->getHeader('x-appwrite-jwt', '');
if (!empty($authJWT) && !$project->isEmpty()) { // JWT authentication
if (!$user->isEmpty()) {
if (! empty($authJWT) && ! $project->isEmpty()) { // JWT authentication
if (! $user->isEmpty()) {
throw new Exception(Exception::USER_JWT_AND_COOKIE_SET);
}
@@ -423,7 +428,7 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
}
$jwtUserId = $payload['userId'] ?? '';
if (!empty($jwtUserId)) {
if (! empty($jwtUserId)) {
if ($mode === APP_MODE_ADMIN) {
$user = $dbForPlatform->getDocument('users', $jwtUserId);
} else {
@@ -431,7 +436,7 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
}
}
$jwtSessionId = $payload['sessionId'] ?? '';
if (!empty($jwtSessionId)) {
if (! empty($jwtSessionId)) {
if (empty($user->find('$id', $jwtSessionId, 'sessions'))) { // Match JWT to active token
$user = new User([]);
}
@@ -441,22 +446,22 @@ Http::setResource('user', function (string $mode, Document $project, Document $c
// Account based on account API key
$accountKey = $request->getHeader('x-appwrite-key', '');
$accountKeyUserId = $request->getHeader('x-appwrite-user', '');
if (!empty($accountKeyUserId) && !empty($accountKey)) {
if (!$user->isEmpty()) {
if (! empty($accountKeyUserId) && ! empty($accountKey)) {
if (! $user->isEmpty()) {
throw new Exception(Exception::USER_API_KEY_AND_SESSION_SET);
}
$accountKeyUser = $dbForPlatform->getAuthorization()->skip(fn () => $dbForPlatform->getDocument('users', $accountKeyUserId));
if (!$accountKeyUser->isEmpty()) {
if (! $accountKeyUser->isEmpty()) {
$key = $accountKeyUser->find(
key: 'secret',
find: $accountKey,
subject: 'keys'
);
if (!empty($key)) {
if (! empty($key)) {
$expire = $key->getAttribute('expire');
if (!empty($expire) && $expire < DatabaseDateTime::formatTz(DatabaseDateTime::now())) {
if (! empty($expire) && $expire < DatabaseDateTime::formatTz(DatabaseDateTime::now())) {
throw new Exception(Exception::ACCOUNT_KEY_EXPIRED);
}
@@ -475,10 +480,9 @@ Http::setResource('project', function ($dbForPlatform, $request, $console, $auth
/** @var Appwrite\Utopia\Request $request */
/** @var Utopia\Database\Database $dbForPlatform */
/** @var Utopia\Database\Document $console */
$projectId = $request->getParam('project', $request->getHeader('x-appwrite-project', ''));
// Realtime channel "project" can send project=Query array
if (!\is_string($projectId)) {
if (! \is_string($projectId)) {
$projectId = $request->getHeader('x-appwrite-project', '');
}
@@ -499,7 +503,7 @@ Http::setResource('session', function (User $user, Store $store, Token $proofFor
$sessions = $user->getAttribute('sessions', []);
$sessionId = $user->sessionVerify($store->getProperty('secret', ''), $proofForToken);
if (!$sessionId) {
if (! $sessionId) {
return;
}
foreach ($sessions as $session) {
@@ -509,7 +513,6 @@ Http::setResource('session', function (User $user, Store $store, Token $proofFor
}
}
return;
}, ['user', 'store', 'proofForToken']);
Http::setResource('store', function (): Store {
@@ -533,12 +536,14 @@ Http::setResource('proofForPassword', function (): Password {
Http::setResource('proofForToken', function (): Token {
$token = new Token();
$token->setHash(new Sha());
return $token;
});
Http::setResource('proofForCode', function (): Code {
$code = new Code();
$code->setHash(new Sha());
return $code;
});
@@ -550,7 +555,7 @@ Http::setResource('authorization', function () {
return new Authorization();
}, []);
Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatform, Cache $cache, Document $project, Response $response, Publisher $publisher, Publisher $publisherFunctions, Publisher $publisherWebhooks, Event $queueForEvents, Func $queueForFunctions, Webhook $queueForWebhooks, Realtime $queueForRealtime, StatsUsage $queueForStatsUsage, Authorization $authorization) {
Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatform, Cache $cache, Document $project, Response $response, Publisher $publisher, Publisher $publisherFunctions, Publisher $publisherWebhooks, Event $queueForEvents, Func $queueForFunctions, Webhook $queueForWebhooks, Realtime $queueForRealtime, UsageContext $usage, Authorization $authorization) {
if ($project->isEmpty() || $project->getId() === 'console') {
return $dbForPlatform;
}
@@ -615,9 +620,8 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
->from($queueForEvents)
->trigger();
/** Trigger webhooks events only if a project has them enabled */
if (!empty($project->getAttribute('webhooks'))) {
if (! empty($project->getAttribute('webhooks'))) {
$queueForWebhooks
->from($queueForEvents)
->trigger();
@@ -636,7 +640,6 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
*/
$functionsEventsCacheListener = function (string $event, Document $document, Document $project, Database $dbForProject) {
if ($document->getCollection() !== 'functions') {
return;
}
@@ -658,7 +661,7 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
$dbForProject->getCache()->purge($cacheKey);
};
$usageDatabaseListener = function (string $event, Document $document, StatsUsage $queueForStatsUsage) {
$usageDatabaseListener = function (string $event, Document $document, UsageContext $usage) {
$value = 1;
switch ($event) {
@@ -678,81 +681,78 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
switch (true) {
case $document->getCollection() === 'teams':
$queueForStatsUsage->addMetric(METRIC_TEAMS, $value); // per project
$usage->addMetric(METRIC_TEAMS, $value); // per project
break;
case $document->getCollection() === 'users':
$queueForStatsUsage->addMetric(METRIC_USERS, $value); // per project
$usage->addMetric(METRIC_USERS, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage->addReduce($document);
$usage->addReduce($document);
}
break;
case $document->getCollection() === 'sessions': // sessions
$queueForStatsUsage->addMetric(METRIC_SESSIONS, $value); //per project
$usage->addMetric(METRIC_SESSIONS, $value); // per project
break;
case $document->getCollection() === 'databases': // databases
$queueForStatsUsage->addMetric(METRIC_DATABASES, $value); // per project
$usage->addMetric(METRIC_DATABASES, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage->addReduce($document);
$usage->addReduce($document);
}
break;
case str_starts_with($document->getCollection(), 'database_') && !str_contains($document->getCollection(), 'collection'): //collections
case str_starts_with($document->getCollection(), 'database_') && ! str_contains($document->getCollection(), 'collection'): // collections
$parts = explode('_', $document->getCollection());
$databaseInternalId = $parts[1] ?? 0;
$queueForStatsUsage
$usage
->addMetric(METRIC_COLLECTIONS, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, METRIC_DATABASE_ID_COLLECTIONS), $value);
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage->addReduce($document);
$usage->addReduce($document);
}
break;
case str_starts_with($document->getCollection(), 'database_') && str_contains($document->getCollection(), '_collection_'): //documents
case str_starts_with($document->getCollection(), 'database_') && str_contains($document->getCollection(), '_collection_'): // documents
$parts = explode('_', $document->getCollection());
$databaseInternalId = $parts[1] ?? 0;
$databaseInternalId = $parts[1] ?? 0;
$collectionInternalId = $parts[3] ?? 0;
$queueForStatsUsage
$usage
->addMetric(METRIC_DOCUMENTS, $value) // per project
->addMetric(str_replace('{databaseInternalId}', $databaseInternalId, METRIC_DATABASE_ID_DOCUMENTS), $value) // per database
->addMetric(str_replace(['{databaseInternalId}', '{collectionInternalId}'], [$databaseInternalId, $collectionInternalId], METRIC_DATABASE_ID_COLLECTION_ID_DOCUMENTS), $value); // per collection
break;
case $document->getCollection() === 'buckets': //buckets
$queueForStatsUsage
->addMetric(METRIC_BUCKETS, $value); // per project
case $document->getCollection() === 'buckets': // buckets
$usage->addMetric(METRIC_BUCKETS, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage
$usage
->addReduce($document);
}
break;
case str_starts_with($document->getCollection(), 'bucket_'): // files
$parts = explode('_', $document->getCollection());
$bucketInternalId = $parts[1];
$queueForStatsUsage
$bucketInternalId = $parts[1];
$usage
->addMetric(METRIC_FILES, $value) // per project
->addMetric(METRIC_FILES_STORAGE, $document->getAttribute('sizeOriginal') * $value) // per project
->addMetric(str_replace('{bucketInternalId}', $bucketInternalId, METRIC_BUCKET_ID_FILES), $value) // per bucket
->addMetric(str_replace('{bucketInternalId}', $bucketInternalId, METRIC_BUCKET_ID_FILES_STORAGE), $document->getAttribute('sizeOriginal') * $value); // per bucket
break;
case $document->getCollection() === 'functions':
$queueForStatsUsage
->addMetric(METRIC_FUNCTIONS, $value); // per project
$usage->addMetric(METRIC_FUNCTIONS, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage
$usage
->addReduce($document);
}
break;
case $document->getCollection() === 'sites':
$queueForStatsUsage
->addMetric(METRIC_SITES, $value); // per project
$usage->addMetric(METRIC_SITES, $value); // per project
if ($event === Database::EVENT_DOCUMENT_DELETE) {
$queueForStatsUsage
$usage
->addReduce($document);
}
break;
case $document->getCollection() === 'deployments':
$queueForStatsUsage
$usage
->addMetric(METRIC_DEPLOYMENTS, $value) // per project
->addMetric(METRIC_DEPLOYMENTS_STORAGE, $document->getAttribute('size') * $value) // per project
->addMetric(str_replace(['{resourceType}'], [$document->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_DEPLOYMENTS), $value) // per function
@@ -772,30 +772,27 @@ Http::setResource('dbForProject', function (Group $pools, Database $dbForPlatfor
$queueForWebhooks = new Webhook($publisherWebhooks);
$queueForRealtime = new Realtime();
$database
->on(Database::EVENT_DOCUMENT_CREATE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENT_DELETE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENTS_CREATE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENTS_DELETE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENTS_UPSERT, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $queueForStatsUsage))
->on(Database::EVENT_DOCUMENT_CREATE, 'create-trigger-events', fn ($event, $document) => $eventDatabaseListener(
$project,
$document,
$response,
$queueForEventsClone->from($queueForEvents),
$queueForFunctions->from($queueForEvents),
$queueForWebhooks->from($queueForEvents),
$queueForRealtime->from($queueForEvents)
))
->on(Database::EVENT_DOCUMENT_CREATE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
->on(Database::EVENT_DOCUMENT_UPDATE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
->on(Database::EVENT_DOCUMENT_DELETE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
;
->on(Database::EVENT_DOCUMENT_CREATE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENT_DELETE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENTS_CREATE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENTS_DELETE, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENTS_UPSERT, 'calculate-usage', fn ($event, $document) => $usageDatabaseListener($event, $document, $usage))
->on(Database::EVENT_DOCUMENT_CREATE, 'create-trigger-events', fn ($event, $document) => $eventDatabaseListener(
$project,
$document,
$response,
$queueForEventsClone->from($queueForEvents),
$queueForFunctions->from($queueForEvents),
$queueForWebhooks->from($queueForEvents),
$queueForRealtime->from($queueForEvents)
))
->on(Database::EVENT_DOCUMENT_CREATE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
->on(Database::EVENT_DOCUMENT_UPDATE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database))
->on(Database::EVENT_DOCUMENT_DELETE, 'purge-function-events-cache', fn ($event, $document) => $functionsEventsCacheListener($event, $document, $project, $database));
return $database;
}, ['pools', 'dbForPlatform', 'cache', 'project', 'response', 'publisher', 'publisherFunctions', 'publisherWebhooks', 'queueForEvents', 'queueForFunctions', 'queueForWebhooks', 'queueForRealtime', 'queueForStatsUsage', 'authorization']);
}, ['pools', 'dbForPlatform', 'cache', 'project', 'response', 'publisher', 'publisherFunctions', 'publisherWebhooks', 'queueForEvents', 'queueForFunctions', 'queueForWebhooks', 'queueForRealtime', 'usage', 'authorization']);
Http::setResource('dbForPlatform', function (Group $pools, Cache $cache, Authorization $authorization) {
@@ -844,8 +841,7 @@ Http::setResource('getProjectDB', function (Group $pools, Database $dbForPlatfor
->setMetadata('project', $project->getId())
->setTimeout(APP_DATABASE_TIMEOUT_MILLISECONDS_API)
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES)
->setDocumentType('users', User::class)
;
->setDocumentType('users', User::class);
$sharedTables = \explode(',', System::getEnv('_APP_DATABASE_SHARED_TABLES', ''));
@@ -865,6 +861,7 @@ Http::setResource('getProjectDB', function (Group $pools, Database $dbForPlatfor
if (isset($databases[$dsn->getHost()])) {
$database = $databases[$dsn->getHost()];
$configure($database);
return $database;
}
@@ -881,8 +878,9 @@ Http::setResource('getLogsDB', function (Group $pools, Cache $cache, Authorizati
$database = null;
return function (?Document $project = null) use ($pools, $cache, $authorization, &$database) {
if ($database !== null && $project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
if ($database !== null && $project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
return $database;
}
@@ -898,7 +896,7 @@ Http::setResource('getLogsDB', function (Group $pools, Cache $cache, Authorizati
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES);
// set tenant
if ($project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
if ($project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
}
@@ -908,6 +906,7 @@ Http::setResource('getLogsDB', function (Group $pools, Cache $cache, Authorizati
Http::setResource('audit', function ($dbForProject) {
$adapter = new AdapterDatabase($dbForProject);
return new Audit($adapter);
}, ['dbForProject']);
@@ -923,6 +922,7 @@ Http::setResource('cache', function (Group $pools, Telemetry $telemetry) {
$cache = new Cache(new Sharding($adapters));
$cache->setTelemetry($telemetry);
return $cache;
}, ['pools', 'telemetry']);
@@ -968,9 +968,9 @@ Http::setResource('deviceForBuilds', function ($project, Telemetry $telemetry) {
function getDevice(string $root, string $connection = ''): Device
{
$connection = !empty($connection) ? $connection : System::getEnv('_APP_CONNECTIONS_STORAGE', '');
$connection = ! empty($connection) ? $connection : System::getEnv('_APP_CONNECTIONS_STORAGE', '');
if (!empty($connection)) {
if (! empty($connection)) {
$acl = 'private';
$device = Storage::DEVICE_LOCAL;
$accessKey = '';
@@ -992,8 +992,9 @@ function getDevice(string $root, string $connection = ''): Device
switch ($device) {
case Storage::DEVICE_S3:
if (!empty($url)) {
$bucketRoot = (!empty($bucket) ? $bucket . '/' : '') . \ltrim($root, '/');
if (! empty($url)) {
$bucketRoot = (! empty($bucket) ? $bucket . '/' : '') . \ltrim($root, '/');
return new S3($bucketRoot, $accessKey, $accessSecret, $url, $region, $acl);
} else {
return new AWS($root, $accessKey, $accessSecret, $bucket, $region, $acl);
@@ -1002,6 +1003,7 @@ function getDevice(string $root, string $connection = ''): Device
case STORAGE::DEVICE_DO_SPACES:
$device = new DOSpaces($root, $accessKey, $accessSecret, $bucket, $region, $acl);
$device->setHttpVersion(S3::HTTP_VERSION_1_1);
return $device;
case Storage::DEVICE_BACKBLAZE:
return new Backblaze($root, $accessKey, $accessSecret, $bucket, $region, $acl);
@@ -1025,8 +1027,9 @@ function getDevice(string $root, string $connection = ''): Device
$s3Bucket = System::getEnv('_APP_STORAGE_S3_BUCKET', '');
$s3Acl = 'private';
$s3EndpointUrl = System::getEnv('_APP_STORAGE_S3_ENDPOINT', '');
if (!empty($s3EndpointUrl)) {
$bucketRoot = (!empty($s3Bucket) ? $s3Bucket . '/' : '') . \ltrim($root, '/');
if (! empty($s3EndpointUrl)) {
$bucketRoot = (! empty($s3Bucket) ? $s3Bucket . '/' : '') . \ltrim($root, '/');
return new S3($bucketRoot, $s3AccessKey, $s3SecretKey, $s3EndpointUrl, $s3Region, $s3Acl);
} else {
return new AWS($root, $s3AccessKey, $s3SecretKey, $s3Bucket, $s3Region, $s3Acl);
@@ -1040,6 +1043,7 @@ function getDevice(string $root, string $connection = ''): Device
$doSpacesAcl = 'private';
$device = new DOSpaces($root, $doSpacesAccessKey, $doSpacesSecretKey, $doSpacesBucket, $doSpacesRegion, $doSpacesAcl);
$device->setHttpVersion(S3::HTTP_VERSION_1_1);
return $device;
case Storage::DEVICE_BACKBLAZE:
$backblazeAccessKey = System::getEnv('_APP_STORAGE_BACKBLAZE_ACCESS_KEY', '');
@@ -1047,6 +1051,7 @@ function getDevice(string $root, string $connection = ''): Device
$backblazeRegion = System::getEnv('_APP_STORAGE_BACKBLAZE_REGION', '');
$backblazeBucket = System::getEnv('_APP_STORAGE_BACKBLAZE_BUCKET', '');
$backblazeAcl = 'private';
return new Backblaze($root, $backblazeAccessKey, $backblazeSecretKey, $backblazeBucket, $backblazeRegion, $backblazeAcl);
case Storage::DEVICE_LINODE:
$linodeAccessKey = System::getEnv('_APP_STORAGE_LINODE_ACCESS_KEY', '');
@@ -1054,6 +1059,7 @@ function getDevice(string $root, string $connection = ''): Device
$linodeRegion = System::getEnv('_APP_STORAGE_LINODE_REGION', '');
$linodeBucket = System::getEnv('_APP_STORAGE_LINODE_BUCKET', '');
$linodeAcl = 'private';
return new Linode($root, $linodeAccessKey, $linodeSecretKey, $linodeBucket, $linodeRegion, $linodeAcl);
case Storage::DEVICE_WASABI:
$wasabiAccessKey = System::getEnv('_APP_STORAGE_WASABI_ACCESS_KEY', '');
@@ -1061,6 +1067,7 @@ function getDevice(string $root, string $connection = ''): Device
$wasabiRegion = System::getEnv('_APP_STORAGE_WASABI_REGION', '');
$wasabiBucket = System::getEnv('_APP_STORAGE_WASABI_BUCKET', '');
$wasabiAcl = 'private';
return new Wasabi($root, $wasabiAccessKey, $wasabiSecretKey, $wasabiBucket, $wasabiRegion, $wasabiAcl);
}
}
@@ -1087,7 +1094,6 @@ Http::setResource('passwordsDictionary', function ($register) {
return $register->get('passwordsDictionary');
}, ['register']);
Http::setResource('servers', function () {
$platforms = Config::getParam('sdks');
$server = $platforms[APP_SDK_PLATFORM_SERVER];
@@ -1195,16 +1201,17 @@ Http::setResource('gitHub', function (Cache $cache) {
}, ['cache']);
Http::setResource('requestTimestamp', function ($request) {
//TODO: Move this to the Request class itself
// TODO: Move this to the Request class itself
$timestampHeader = $request->getHeader('x-appwrite-timestamp');
$requestTimestamp = null;
if (!empty($timestampHeader)) {
if (! empty($timestampHeader)) {
try {
$requestTimestamp = new \DateTime($timestampHeader);
} catch (\Throwable $e) {
throw new Exception(Exception::GENERAL_ARGUMENT_INVALID, 'Invalid X-Appwrite-Timestamp header value');
}
}
return $requestTimestamp;
}, ['request']);
@@ -1221,13 +1228,13 @@ Http::setResource('devKey', function (Request $request, Document $project, array
// Check if given key match project's development keys
$key = $project->find('secret', $devKey, 'devKeys');
if (!$key) {
if (! $key) {
return new Document([]);
}
// check expiration
$expire = $key->getAttribute('expire');
if (!empty($expire) && $expire < DatabaseDateTime::formatTz(DatabaseDateTime::now())) {
if (! empty($expire) && $expire < DatabaseDateTime::formatTz(DatabaseDateTime::now())) {
return new Document([]);
}
@@ -1235,7 +1242,9 @@ Http::setResource('devKey', function (Request $request, Document $project, array
$accessedAt = $key->getAttribute('accessedAt', 0);
if (empty($accessedAt) || DatabaseDateTime::formatTz(DatabaseDateTime::addSeconds(new \DateTime(), -APP_KEY_ACCESS)) > $accessedAt) {
$key->setAttribute('accessedAt', DatabaseDateTime::now());
$authorization->skip(fn () => $dbForPlatform->updateDocument('devKeys', $key->getId(), $key));
$authorization->skip(fn () => $dbForPlatform->updateDocument('devKeys', $key->getId(), new Document([
'accessedAt' => $key->getAttribute('accessedAt')
])));
$dbForPlatform->purgeCachedDocument('projects', $project->getId());
}
@@ -1246,13 +1255,16 @@ Http::setResource('devKey', function (Request $request, Document $project, array
if ($sdk !== 'UNKNOWN' && $sdkValidator->isValid($sdk)) {
$sdks = $key->getAttribute('sdks', []);
if (!in_array($sdk, $sdks)) {
if (! in_array($sdk, $sdks)) {
$sdks[] = $sdk;
$key->setAttribute('sdks', $sdks);
/** Update access time as well */
$key->setAttribute('accessedAt', DatabaseDateTime::now());
$key = $authorization->skip(fn () => $dbForPlatform->updateDocument('devKeys', $key->getId(), $key));
$key = $authorization->skip(fn () => $dbForPlatform->updateDocument('devKeys', $key->getId(), new Document([
'sdks' => $key->getAttribute('sdks'),
'accessedAt' => $key->getAttribute('accessedAt')
])));
$dbForPlatform->purgeCachedDocument('projects', $project->getId());
}
}
@@ -1266,7 +1278,7 @@ Http::setResource('team', function (Document $project, Database $dbForPlatform,
$teamInternalId = $project->getAttribute('teamInternalId', '');
} else {
$route = $utopia->match($request);
$path = !empty($route) ? $route->getPath() : $request->getURI();
$path = ! empty($route) ? $route->getPath() : $request->getURI();
$orgHeader = $request->getHeader('x-appwrite-organization', '');
if (str_starts_with($path, '/v1/projects/:projectId')) {
$uri = $request->getURI();
@@ -1281,8 +1293,9 @@ Http::setResource('team', function (Document $project, Database $dbForPlatform,
}
$team = $authorization->skip(fn () => $dbForPlatform->getDocument('teams', $teamId));
return $team;
} elseif (!empty($orgHeader)) {
} elseif (! empty($orgHeader)) {
return $authorization->skip(fn () => $dbForPlatform->getDocument('teams', $orgHeader));
}
}
@@ -1312,13 +1325,13 @@ Http::setResource('previewHostname', function (Request $request, ?Key $apiKey) {
if (Http::isDevelopment()) {
$allowed = true;
} elseif (!\is_null($apiKey) && $apiKey->getHostnameOverride() === true) {
} elseif (! \is_null($apiKey) && $apiKey->getHostnameOverride() === true) {
$allowed = true;
}
if ($allowed) {
$host = $request->getQuery('appwrite-hostname', $request->getHeader('x-appwrite-hostname', '')) ?? '';
if (!empty($host)) {
if (! empty($host)) {
return $host;
}
}
@@ -1339,19 +1352,19 @@ Http::setResource('apiKey', function (Request $request, Document $project, Docum
$organizationHeader = $request->getHeader('x-appwrite-organization');
$projectHeader = $request->getHeader('x-appwrite-project');
if (!empty($key->getProjectId())) {
if (! empty($key->getProjectId())) {
if (empty($projectHeader) || $projectHeader !== $key->getProjectId()) {
throw new Exception(Exception::PROJECT_ID_MISSING);
}
}
if (!empty($key->getUserId())) {
if (! empty($key->getUserId())) {
if (empty($userHeader) || $userHeader !== $key->getUserId()) {
throw new Exception(Exception::USER_ID_MISSING);
}
}
if (!empty($key->getTeamId())) {
if (! empty($key->getTeamId())) {
if (empty($organizationHeader) || $organizationHeader !== $key->getTeamId()) {
throw new Exception(Exception::ORGANIZATION_ID_MISSING);
}
@@ -1365,7 +1378,7 @@ Http::setResource('executor', fn () => new Executor());
Http::setResource('resourceToken', function ($project, $dbForProject, $request, Authorization $authorization) {
$tokenJWT = $request->getParam('token');
if (!empty($tokenJWT) && !$project->isEmpty()) { // JWT authentication
if (! empty($tokenJWT) && ! $project->isEmpty()) { // JWT authentication
// Use a large but reasonable maxAge to avoid auto-exp when token has no expiry
$jwt = new JWT(System::getEnv('_APP_OPENSSL_KEY_V1'), RESOURCE_TOKEN_ALGORITHM, RESOURCE_TOKEN_MAX_AGE, RESOURCE_TOKEN_LEEWAY); // Instantiate with key, algo, maxAge and leeway.
@@ -1409,7 +1422,9 @@ Http::setResource('resourceToken', function ($project, $dbForProject, $request,
$accessedAt = $token->getAttribute('accessedAt', 0);
if (empty($accessedAt) || DatabaseDateTime::formatTz(DatabaseDateTime::addSeconds(new \DateTime(), -APP_RESOURCE_TOKEN_ACCESS)) > $accessedAt) {
$token->setAttribute('accessedAt', DatabaseDateTime::now());
$authorization->skip(fn () => $dbForProject->updateDocument('resourceTokens', $token->getId(), $token));
$authorization->skip(fn () => $dbForProject->updateDocument('resourceTokens', $token->getId(), new Document([
'accessedAt' => $token->getAttribute('accessedAt')
])));
}
return new Document([
@@ -1423,6 +1438,7 @@ Http::setResource('resourceToken', function ($project, $dbForProject, $request,
default => throw new Exception(Exception::TOKEN_RESOURCE_TYPE_INVALID),
};
}
return new Document([]);
}, ['project', 'dbForProject', 'request', 'authorization']);
+6 -1
View File
@@ -5,4 +5,9 @@ use Utopia\Span\Span;
use Utopia\Span\Storage;
Span::setStorage(new Storage\Coroutine());
Span::addExporter(new Exporter\Pretty());
Span::addExporter(new Exporter\Pretty(), function (Span $span): bool {
if (\str_starts_with($span->getAction(), 'listener.')) {
return $span->getError() !== null;
}
return true;
});
+100 -11
View File
@@ -224,6 +224,13 @@ if (!function_exists('getTelemetry')) {
}
}
if (!function_exists('triggerStats')) {
function triggerStats(array $event, string $projectId): void
{
return;
}
}
$realtime = getRealtime();
/**
@@ -356,7 +363,10 @@ $server->onStart(function () use ($stats, $register, $containerId, &$statsDocume
->setAttribute('timestamp', DateTime::now())
->setAttribute('value', json_encode($payload));
$database->getAuthorization()->skip(fn () => $database->updateDocument('realtime', $statsDocument->getId(), $statsDocument));
$database->getAuthorization()->skip(fn () => $database->updateDocument('realtime', $statsDocument->getId(), new Document([
'timestamp' => $statsDocument->getAttribute('timestamp'),
'value' => $statsDocument->getAttribute('value')
])));
} catch (Throwable $th) {
logError($th, "updateWorkerDocument");
}
@@ -545,20 +555,41 @@ $server->onWorkerStart(function (int $workerId) use ($server, $register, $stats,
}
$total = 0;
$outboundBytes = 0;
foreach ($groups as $group) {
$data = $event['data'];
$data['subscriptions'] = $group['subscriptions'];
$server->send($group['ids'], json_encode([
$payloadJson = json_encode([
'type' => 'event',
'data' => $data
]));
$total += count($group['ids']);
]);
$server->send($group['ids'], $payloadJson);
$count = count($group['ids']);
$total += $count;
$outboundBytes += strlen($payloadJson) * $count;
}
if ($total > 0) {
$register->get('telemetry.messageSentCounter')->add($total);
$stats->incr($event['project'], 'messages', $total);
$projectId = $event['project'] ?? null;
if (!empty($projectId)) {
$metrics = [
METRIC_REALTIME_CONNECTIONS_MESSAGES_SENT => $total,
];
if ($outboundBytes > 0) {
$metrics[METRIC_REALTIME_OUTBOUND] = $outboundBytes;
}
triggerStats($metrics, $projectId);
}
}
});
} catch (Throwable $th) {
@@ -635,6 +666,12 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
throw new Exception(Exception::REALTIME_TOO_MANY_MESSAGES, 'Too many requests');
}
$rawSize = $request->getSize();
triggerStats([
METRIC_REALTIME_INBOUND => $rawSize,
], $project->getId());
/*
* Validate Client Domain - Check to avoid CSRF attack.
* Adding Appwrite API domains to allow XDOMAIN communication.
@@ -689,14 +726,16 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
$user = empty($user->getId()) ? null : $response->output($user, Response::MODEL_ACCOUNT);
$server->send([$connection], json_encode([
$connectedPayloadJson = json_encode([
'type' => 'connected',
'data' => [
'channels' => $names,
'subscriptions' => $mapping,
'user' => $user
]
]));
]);
$server->send([$connection], $connectedPayloadJson);
$register->get('telemetry.connectionCounter')->add(1);
$register->get('telemetry.connectionCreatedCounter')->add(1);
@@ -707,6 +746,12 @@ $server->onOpen(function (int $connection, SwooleRequest $request) use ($server,
]);
$stats->incr($project->getId(), 'connections');
$stats->incr($project->getId(), 'connectionsTotal');
$connectedOutboundBytes = \strlen($connectedPayloadJson);
triggerStats([METRIC_REALTIME_CONNECTIONS => 1, METRIC_REALTIME_OUTBOUND => $connectedOutboundBytes], $project->getId());
} catch (Throwable $th) {
logError($th, 'realtime', project: $project, user: $logUser, authorization: $authorization);
@@ -748,6 +793,7 @@ $server->onMessage(function (int $connection, string $message) use ($server, $re
$authorization = null;
try {
$rawSize = \strlen($message);
$response = new Response(new SwooleResponse());
$projectId = $realtime->connections[$connection]['projectId'] ?? null;
@@ -760,7 +806,7 @@ $server->onMessage(function (int $connection, string $message) use ($server, $re
$database = getConsoleDB();
$database->setAuthorization($authorization);
if ($projectId !== 'console') {
if (!empty($projectId) && $projectId !== 'console') {
$project = $authorization->skip(fn () => $database->getDocument('projects', $projectId));
$database = getProjectDB($project);
@@ -786,17 +832,41 @@ $server->onMessage(function (int $connection, string $message) use ($server, $re
throw new Exception(Exception::REALTIME_TOO_MANY_MESSAGES, 'Too many messages.');
}
// Record realtime inbound bytes for this project
if ($project !== null && !$project->isEmpty()) {
triggerStats([
METRIC_REALTIME_INBOUND => $rawSize,
], $project->getId());
}
$message = json_decode($message, true);
if (is_null($message) || (!array_key_exists('type', $message) && !array_key_exists('data', $message))) {
throw new Exception(Exception::REALTIME_MESSAGE_FORMAT_INVALID, 'Message format is not valid.');
}
// Ping does not require project context; other messages do (e.g. after unsubscribe during auth)
if (empty($projectId) && ($message['type'] ?? '') !== 'ping') {
throw new Exception(Exception::REALTIME_POLICY_VIOLATION, 'Missing project context. Reconnect to the project first.');
}
switch ($message['type']) {
case 'ping':
$server->send([$connection], json_encode([
$pongPayloadJson = json_encode([
'type' => 'pong'
]));
]);
$server->send([$connection], $pongPayloadJson);
if ($project !== null && !$project->isEmpty()) {
$pongOutboundBytes = \strlen($pongPayloadJson);
if ($pongOutboundBytes > 0) {
triggerStats([
METRIC_REALTIME_OUTBOUND => $pongOutboundBytes,
], $project->getId());
}
}
break;
case 'authentication':
@@ -857,14 +927,27 @@ $server->onMessage(function (int $connection, string $message) use ($server, $re
}
$user = $response->output($user, Response::MODEL_ACCOUNT);
$server->send([$connection], json_encode([
$authResponsePayloadJson = json_encode([
'type' => 'response',
'data' => [
'to' => 'authentication',
'success' => true,
'user' => $user
]
]));
]);
$server->send([$connection], $authResponsePayloadJson);
if ($project !== null && !$project->isEmpty()) {
$authOutboundBytes = \strlen($authResponsePayloadJson);
if ($authOutboundBytes > 0) {
triggerStats([
METRIC_REALTIME_OUTBOUND => $authOutboundBytes,
], $project->getId());
}
}
break;
@@ -905,6 +988,12 @@ $server->onClose(function (int $connection) use ($realtime, $stats, $register) {
if (array_key_exists($connection, $realtime->connections)) {
$stats->decr($realtime->connections[$connection]['projectId'], 'connectionsTotal');
$register->get('telemetry.connectionCounter')->add(-1);
$projectId = $realtime->connections[$connection]['projectId'];
triggerStats([
METRIC_REALTIME_CONNECTIONS => -1,
], $projectId);
}
$realtime->unsubscribe($connection);
+31 -27
View File
@@ -13,11 +13,12 @@ use Appwrite\Event\Func;
use Appwrite\Event\Mail;
use Appwrite\Event\Messaging;
use Appwrite\Event\Migration;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Realtime;
use Appwrite\Event\Screenshot;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Webhook;
use Appwrite\Platform\Appwrite;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Executor\Executor;
use Swoole\Runtime;
@@ -42,6 +43,7 @@ use Utopia\Pools\Group;
use Utopia\Queue\Broker\Pool as BrokerPool;
use Utopia\Queue\Message;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
use Utopia\Queue\Server;
use Utopia\Registry\Registry;
use Utopia\Storage\Device\Telemetry as TelemetryDevice;
@@ -58,7 +60,8 @@ Server::setResource('register', fn () => $register);
Server::setResource('authorization', function () {
$authorization = new Authorization();
$authorization->disable();
return $authorization;
return $authorization;
}, []);
Server::setResource('dbForPlatform', function (Cache $cache, Registry $register, Authorization $authorization) {
@@ -70,9 +73,7 @@ Server::setResource('dbForPlatform', function (Cache $cache, Registry $register,
->setDatabase(APP_DATABASE)
->setAuthorization($authorization)
->setNamespace('_console')
->setDocumentType('users', User::class)
;
->setDocumentType('users', User::class);
return $dbForPlatform;
}, ['cache', 'register', 'authorization']);
@@ -111,7 +112,7 @@ Server::setResource('dbForProject', function (Cache $cache, Registry $register,
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -151,7 +152,7 @@ Server::setResource('getProjectDB', function (Group $pools, Database $dbForPlatf
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -173,7 +174,7 @@ Server::setResource('getProjectDB', function (Group $pools, Database $dbForPlatf
if (\in_array($dsn->getHost(), $sharedTables)) {
$database
->setSharedTables(true)
->setTenant((int)$project->getSequence())
->setTenant((int) $project->getSequence())
->setNamespace($dsn->getParam('namespace'));
} else {
$database
@@ -193,9 +194,11 @@ Server::setResource('getProjectDB', function (Group $pools, Database $dbForPlatf
Server::setResource('getLogsDB', function (Group $pools, Cache $cache, Authorization $authorization) {
$database = null;
return function (?Document $project = null) use ($pools, $cache, $database, $authorization) {
if ($database !== null && $project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int)$project->getSequence());
if ($database !== null && $project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
return $database;
}
@@ -211,8 +214,8 @@ Server::setResource('getLogsDB', function (Group $pools, Cache $cache, Authoriza
->setMaxQueryValues(APP_DATABASE_QUERY_MAX_VALUES_WORKER);
// set tenant
if ($project !== null && !$project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int)$project->getSequence());
if ($project !== null && ! $project->isEmpty() && $project->getId() !== 'console') {
$database->setTenant((int) $project->getSequence());
}
return $database;
@@ -227,6 +230,7 @@ Server::setResource('auditRetention', function (Document $project) {
if ($project->getId() === 'console') {
return DateTime::addSeconds(new \DateTime(), -1 * System::getEnv('_APP_MAINTENANCE_RETENTION_AUDIT_CONSOLE', 15778800)); // 6 months
}
return DateTime::addSeconds(new \DateTime(), -1 * System::getEnv('_APP_MAINTENANCE_RETENTION_AUDIT', 1209600)); // 14 days
}, ['project']);
@@ -252,7 +256,7 @@ Server::setResource('redis', function () {
$pass = System::getEnv('_APP_REDIS_PASS', '');
$redis = new \Redis();
@$redis->pconnect($host, (int)$port);
@$redis->pconnect($host, (int) $port);
if ($pass) {
$redis->auth($pass);
}
@@ -269,7 +273,6 @@ Server::setResource('timelimit', function (\Redis $redis) {
Server::setResource('log', fn () => new Log());
Server::setResource('publisher', function (Group $pools) {
return new BrokerPool(publisher: $pools->get('publisher'));
}, ['pools']);
@@ -286,10 +289,6 @@ Server::setResource('publisherMigrations', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
Server::setResource('publisherStatsUsage', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
Server::setResource('publisherMessaging', function (BrokerPool $publisher) {
return $publisher;
}, ['publisher']);
@@ -310,9 +309,13 @@ Server::setResource('consumerStatsUsage', function (BrokerPool $consumer) {
return $consumer;
}, ['consumer']);
Server::setResource('queueForStatsUsage', function (Publisher $publisher) {
return new StatsUsage($publisher);
}, ['publisher']);
Server::setResource('usage', function () {
return new Context();
}, []);
Server::setResource('publisherForUsage', fn (Publisher $publisher) => new UsagePublisher(
$publisher,
new Queue(System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME))
), ['publisher']);
Server::setResource('queueForDatabase', function (Publisher $publisher) {
return new EventDatabase($publisher);
@@ -354,7 +357,6 @@ Server::setResource('queueForFunctions', function (Publisher $publisher) {
return new Func($publisher);
}, ['publisher']);
Server::setResource('queueForRealtime', function () {
return new Realtime();
}, []);
@@ -484,11 +486,13 @@ Server::setResource('getAudit', function (Database $dbForPlatform, callable $get
return function (Document $project) use ($dbForPlatform, $getProjectDB) {
if ($project->isEmpty() || $project->getId() === 'console') {
$adapter = new AdapterDatabase($dbForPlatform);
return new UtopiaAudit($adapter);
}
$dbForProject = $getProjectDB($project);
$adapter = new AdapterDatabase($dbForProject);
return new UtopiaAudit($adapter);
};
}, ['dbForPlatform', 'getProjectDB']);
@@ -505,7 +509,7 @@ $pools = $register->get('pools');
$platform = new Appwrite();
$args = $platform->getEnv('argv');
if (!isset($args[1])) {
if (! isset($args[1])) {
Console::error('Missing worker name');
Console::exit(1);
}
@@ -530,10 +534,10 @@ try {
'workersNum' => System::getEnv('_APP_WORKERS_NUM', 1),
'connection' => $pools->get('consumer')->pop()->getResource(),
'workerName' => strtolower($workerName) ?? null,
'queueName' => $queueName
'queueName' => $queueName,
]);
} catch (\Throwable $e) {
Console::error($e->getMessage() . ', File: ' . $e->getFile() . ', Line: ' . $e->getLine());
Console::error($e->getMessage() . ', File: ' . $e->getFile() . ', Line: ' . $e->getLine());
}
$worker = $platform->getWorker();
@@ -550,11 +554,11 @@ $worker
->inject('pools')
->inject('project')
->inject('authorization')
->action(function (Throwable $error, ?Logger $logger, Log $log, Group $pools, Document $project, Authorization $authorization) use ($worker, $queueName) {
->action(function (Throwable $error, ?Logger $logger, Log $log, Group $pools, Document $project, Authorization $authorization) use ($queueName) {
$version = System::getEnv('_APP_VERSION', 'UNKNOWN');
if ($logger) {
$log->setNamespace("appwrite-worker");
$log->setNamespace('appwrite-worker');
$log->setServer(System::getEnv('_APP_LOGGING_SERVICE_IDENTIFIER', \gethostname()));
$log->setVersion($version);
$log->setType(Log::TYPE_ERROR);
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
exec php /usr/src/code/app/cli.php time-travel "$@"
+7 -7
View File
@@ -13,8 +13,7 @@
"test": "vendor/bin/phpunit",
"lint": "vendor/bin/pint --test --config pint.json",
"format": "vendor/bin/pint --config pint.json",
"bench": "vendor/bin/phpbench run --report=benchmark",
"check": "./vendor/bin/phpstan analyse -c phpstan.neon"
"analyze": "./vendor/bin/phpstan analyse -c phpstan.neon --memory-limit=1G"
},
"autoload": {
"psr-4": {
@@ -58,7 +57,7 @@
"utopia-php/compression": "0.1.*",
"utopia-php/config": "1.*",
"utopia-php/console": "0.1.*",
"utopia-php/database": "5.*",
"utopia-php/database": "dev-chore/remove-databases as 5.3.12",
"utopia-php/detector": "0.2.*",
"utopia-php/domains": "1.*",
"utopia-php/emails": "0.6.*",
@@ -70,12 +69,13 @@
"utopia-php/locale": "0.8.*",
"utopia-php/logger": "0.6.*",
"utopia-php/messaging": "0.20.*",
"utopia-php/migration": "1.6.*",
"utopia-php/migration": "1.7.*",
"utopia-php/platform": "0.7.*",
"utopia-php/pools": "1.*",
"utopia-php/span": "1.1.*",
"utopia-php/preloader": "0.2.*",
"utopia-php/queue": "0.15.*",
"utopia-php/servers": "0.2.5",
"utopia-php/registry": "0.5.*",
"utopia-php/storage": "1.0.*",
"utopia-php/system": "0.10.*",
@@ -98,10 +98,10 @@
"brianium/paratest": "7.*",
"phpunit/phpunit": "12.*",
"swoole/ide-helper": "6.*",
"phpstan/phpstan": "1.12.*",
"phpstan/phpstan": "^2.0",
"textalk/websocket": "1.5.*",
"laravel/pint": "1.*",
"phpbench/phpbench": "1.*"
"czproject/git-php": "4.*",
"laravel/pint": "1.*"
},
"provide": {
"ext-phpiredis": "*"
Generated
+98 -698
View File
File diff suppressed because it is too large Load Diff
+144
View File
@@ -0,0 +1,144 @@
# Dev tools for local development only.
# This file is automatically loaded by `docker compose` alongside docker-compose.yml.
# CI sets COMPOSE_FILE=docker-compose.yml explicitly, so these services are excluded from test runs.
services:
appwrite-mongo-express:
profiles: ["mongodb"]
image: mongo-express
container_name: appwrite-mongo-express
networks:
- appwrite
ports:
- "8082:8081"
environment:
ME_CONFIG_MONGODB_URL: "mongodb://root:${_APP_DB_ROOT_PASS}@appwrite-mongodb:27017/?replicaSet=rs0&directConnection=true"
ME_CONFIG_BASICAUTH_USERNAME: ${_APP_DB_USER}
ME_CONFIG_BASICAUTH_PASSWORD: ${_APP_DB_PASS}
depends_on:
- mongodb
adminer:
image: adminer
container_name: appwrite-adminer
restart: always
ports:
- 9506:8080
networks:
- appwrite
- gateway
environment:
- ADMINER_DESIGN=pepa-linha
- ADMINER_DEFAULT_SERVER=mariadb
- ADMINER_DEFAULT_USERNAME=root
- ADMINER_DEFAULT_PASSWORD=rootsecretpassword
- ADMINER_DEFAULT_DB=appwrite
configs:
- source: adminer-index.php
target: /var/www/html/index.php
mode: 0755
labels:
- "traefik.enable=true"
- "traefik.constraint-label-stack=appwrite"
- "traefik.docker.network=gateway"
- "traefik.http.services.appwrite_adminer.loadbalancer.server.port=8080"
- "traefik.http.routers.appwrite_adminer_http.entrypoints=appwrite_web"
- "traefik.http.routers.appwrite_adminer_http.rule=Host(`mysql.localhost`)"
- "traefik.http.routers.appwrite_adminer_http.service=appwrite_adminer"
- "traefik.http.routers.appwrite_adminer_https.entrypoints=appwrite_websecure"
- "traefik.http.routers.appwrite_adminer_https.rule=Host(`mysql.localhost`)"
- "traefik.http.routers.appwrite_adminer_https.service=appwrite_adminer"
- "traefik.http.routers.appwrite_adminer_https.tls=true"
redis-insight:
image: redis/redisinsight:latest
restart: unless-stopped
networks:
- appwrite
- gateway
environment:
- RI_PRE_SETUP_DATABASES_PATH=/mnt/connections.json
configs:
- source: redisinsight-connections.json
target: /mnt/connections.json
mode: 0755
labels:
- "traefik.enable=true"
- "traefik.constraint-label-stack=appwrite"
- "traefik.docker.network=gateway"
- "traefik.http.services.appwrite_redisinsight.loadbalancer.server.port=5540"
- "traefik.http.routers.appwrite_redisinsight_http.entrypoints=appwrite_web"
- "traefik.http.routers.appwrite_redisinsight_http.rule=Host(`redis.localhost`)"
- "traefik.http.routers.appwrite_redisinsight_http.service=appwrite_redisinsight"
- "traefik.http.routers.appwrite_redisinsight_https.entrypoints=appwrite_websecure"
- "traefik.http.routers.appwrite_redisinsight_https.rule=Host(`redis.localhost`)"
- "traefik.http.routers.appwrite_redisinsight_https.service=appwrite_redisinsight"
- "traefik.http.routers.appwrite_redisinsight_https.tls=true"
ports:
- "8081:5540"
graphql-explorer:
container_name: appwrite-graphql-explorer
image: appwrite/altair:0.3.0
restart: unless-stopped
networks:
- appwrite
ports:
- "9509:3000"
environment:
- SERVER_URL=http://localhost/v1/graphql
configs:
redisinsight-connections.json:
content: |
[
{
"compressor": "NONE",
"id": "104dc90a-21ef-4d5e-8912-b30baabb152f",
"host": "redis",
"port": 6379,
"name": "redis:6379",
"db": 0,
"username": "default",
"password": null,
"connectionType": "STANDALONE",
"nameFromProvider": null,
"provider": "REDIS",
"lastConnection": "2025-10-16T09:22:02.591Z",
"modules": [
{
"name": "ReJSON",
"version": 20808,
"semanticVersion": "2.8.8"
},
{
"name": "search",
"version": 21015,
"semanticVersion": "2.10.15"
}
],
"tls": false,
"tlsServername": null,
"verifyServerCert": null,
"caCert": null,
"clientCert": null,
"ssh": false,
"sshOptions": null,
"forceStandalone": false,
"tags": []
}
]
adminer-index.php:
content: |
<?php
if(!count($$_GET)) {
$$_POST['auth'] = [
'server' => $$_ENV['ADMINER_DEFAULT_SERVER'],
'driver' => 'server', /* seems to autodetect the driver from server settings */
'username' => $$_ENV['ADMINER_DEFAULT_USERNAME'],
'password' => $$_ENV['ADMINER_DEFAULT_PASSWORD'],
'db' => $$_ENV['ADMINER_DEFAULT_DB'],
];
}
include './adminer.php';
+58 -190
View File
@@ -10,6 +10,15 @@ x-logging: &x-logging
max-file: "5"
max-size: "10m"
x-build: &x-build
build:
context: .
target: development
args:
DEBUG: false
TESTING: true
VERSION: dev
services:
traefik:
image: traefik:3.6
@@ -50,15 +59,13 @@ services:
appwrite:
container_name: appwrite
<<: *x-logging
<<: [*x-logging, *x-build]
image: appwrite-dev
build:
context: .
target: development
args:
DEBUG: false
TESTING: true
VERSION: dev
healthcheck:
test: ["CMD", "doctor"]
interval: 5s
timeout: 5s
retries: 12
ports:
- 9501:80
networks:
@@ -101,10 +108,10 @@ services:
- ./dev:/usr/src/code/dev
depends_on:
- ${_APP_DB_HOST:-mongodb}
- redis
- coredns
# - clamav
redis:
condition: service_healthy
coredns:
condition: service_started
entrypoint:
- php
- -e
@@ -152,7 +159,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_SMTP_HOST
- _APP_SMTP_PORT
- _APP_SMTP_SECURE
@@ -261,7 +267,7 @@ services:
appwrite-realtime:
entrypoint: realtime
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-realtime
image: appwrite-dev
restart: unless-stopped
@@ -305,7 +311,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_USAGE_STATS
- _APP_LOGGING_CONFIG
- _APP_LOGGING_CONFIG_REALTIME
@@ -314,7 +319,7 @@ services:
appwrite-worker-audits:
entrypoint: worker-audits
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-audits
image: appwrite-dev
networks:
@@ -340,13 +345,12 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_LOGGING_CONFIG
- _APP_DATABASE_SHARED_TABLES
appwrite-worker-webhooks:
entrypoint: worker-webhooks
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-webhooks
image: appwrite-dev
networks:
@@ -371,7 +375,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_REDIS_HOST
- _APP_REDIS_PORT
- _APP_REDIS_USER
@@ -382,7 +385,7 @@ services:
appwrite-worker-deletes:
entrypoint: worker-deletes
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-deletes
image: appwrite-dev
networks:
@@ -414,7 +417,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_STORAGE_DEVICE
- _APP_STORAGE_S3_ACCESS_KEY
- _APP_STORAGE_S3_SECRET
@@ -448,7 +450,7 @@ services:
appwrite-worker-databases:
entrypoint: worker-databases
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-databases
image: appwrite-dev
networks:
@@ -474,7 +476,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_LOGGING_CONFIG
- _APP_WORKERS_NUM
- _APP_QUEUE_NAME
@@ -482,7 +483,7 @@ services:
appwrite-worker-builds:
entrypoint: worker-builds
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-builds
image: appwrite-dev
networks:
@@ -513,7 +514,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_LOGGING_CONFIG
- _APP_VCS_GITHUB_APP_NAME
- _APP_VCS_GITHUB_PRIVATE_KEY
@@ -558,7 +558,7 @@ services:
appwrite-worker-screenshots:
entrypoint: worker-screenshots
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-screenshots
image: appwrite-dev
networks:
@@ -621,7 +621,7 @@ services:
appwrite-worker-certificates:
entrypoint: worker-certificates
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-certificates
image: appwrite-dev
networks:
@@ -658,13 +658,12 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_LOGGING_CONFIG
- _APP_DATABASE_SHARED_TABLES
appwrite-worker-executions:
entrypoint: worker-executions
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-executions
image: appwrite-dev
networks:
@@ -694,7 +693,7 @@ services:
appwrite-worker-functions:
entrypoint: worker-functions
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-functions
image: appwrite-dev
networks:
@@ -722,7 +721,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_FUNCTIONS_TIMEOUT
- _APP_SITES_TIMEOUT
- _APP_COMPUTE_BUILD_TIMEOUT
@@ -739,7 +737,7 @@ services:
appwrite-worker-mails:
entrypoint: worker-mails
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-mails
image: appwrite-dev
networks:
@@ -781,7 +779,7 @@ services:
appwrite-worker-messaging:
entrypoint: worker-messaging
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-messaging
restart: unless-stopped
image: appwrite-dev
@@ -808,7 +806,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_LOGGING_CONFIG
- _APP_SMS_FROM
- _APP_SMS_PROVIDER
@@ -839,7 +836,7 @@ services:
appwrite-worker-migrations:
entrypoint: worker-migrations
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-migrations
restart: unless-stopped
image: appwrite-dev
@@ -875,7 +872,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_LOGGING_CONFIG
- _APP_MIGRATIONS_FIREBASE_CLIENT_ID
- _APP_MIGRATIONS_FIREBASE_CLIENT_SECRET
@@ -885,7 +881,7 @@ services:
appwrite-task-maintenance:
entrypoint: maintenance
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-maintenance
image: appwrite-dev
networks:
@@ -918,7 +914,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_MAINTENANCE_INTERVAL
- _APP_MAINTENANCE_RETENTION_EXECUTION
- _APP_MAINTENANCE_RETENTION_CACHE
@@ -932,7 +927,7 @@ services:
appwrite-task-interval:
entrypoint: interval
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-interval
image: appwrite-dev
networks:
@@ -973,7 +968,7 @@ services:
appwrite-task-stats-resources:
container_name: appwrite-task-stats-resources
entrypoint: stats-resources
<<: *x-logging
<<: [*x-logging, *x-build]
image: appwrite-dev
networks:
- appwrite
@@ -994,7 +989,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_REDIS_HOST
- _APP_REDIS_PORT
- _APP_REDIS_USER
@@ -1006,7 +1000,7 @@ services:
appwrite-worker-stats-resources:
entrypoint: worker-stats-resources
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-stats-resources
image: appwrite-dev
networks:
@@ -1028,7 +1022,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_REDIS_HOST
- _APP_REDIS_PORT
- _APP_REDIS_USER
@@ -1040,7 +1033,7 @@ services:
appwrite-worker-stats-usage:
entrypoint: worker-stats-usage
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-worker-stats-usage
image: appwrite-dev
networks:
@@ -1062,7 +1055,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_REDIS_HOST
- _APP_REDIS_PORT
- _APP_REDIS_USER
@@ -1074,7 +1066,7 @@ services:
appwrite-task-scheduler-functions:
entrypoint: schedule-functions
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-scheduler-functions
image: appwrite-dev
networks:
@@ -1100,12 +1092,11 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_DATABASE_SHARED_TABLES
appwrite-task-scheduler-executions:
entrypoint: schedule-executions
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-scheduler-executions
image: appwrite-dev
networks:
@@ -1131,11 +1122,10 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
appwrite-task-scheduler-messages:
entrypoint: schedule-messages
<<: *x-logging
<<: [*x-logging, *x-build]
container_name: appwrite-task-scheduler-messages
image: appwrite-dev
networks:
@@ -1161,7 +1151,6 @@ services:
- _APP_DB_SCHEMA
- _APP_DB_USER
- _APP_DB_PASS
- _APP_DB_ADAPTER
- _APP_DATABASE_SHARED_TABLES
appwrite-assistant:
@@ -1255,6 +1244,11 @@ services:
- MYSQL_PASSWORD=${_APP_DB_PASS}
- MARIADB_AUTO_UPGRADE=1
command: "mysqld --innodb-flush-method=fsync"
healthcheck:
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
interval: 5s
timeout: 5s
retries: 12
mongodb:
profiles: ["mongodb"]
@@ -1293,20 +1287,7 @@ services:
retries: 10
start_period: 30s
appwrite-mongo-express:
profiles: ["mongodb"]
image: mongo-express
container_name: appwrite-mongo-express
networks:
- appwrite
ports:
- "8082:8081"
environment:
ME_CONFIG_MONGODB_URL: "mongodb://root:${_APP_DB_ROOT_PASS}@appwrite-mongodb:27017/?replicaSet=rs0&directConnection=true"
ME_CONFIG_BASICAUTH_USERNAME: ${_APP_DB_USER}
ME_CONFIG_BASICAUTH_PASSWORD: ${_APP_DB_PASS}
depends_on:
- mongodb
postgresql:
profiles: ["postgresql"]
@@ -1327,6 +1308,11 @@ services:
- POSTGRES_USER=${_APP_DB_USER}
- POSTGRES_PASSWORD=${_APP_DB_PASS}
command: "postgres"
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${_APP_DB_USER}"]
interval: 5s
timeout: 5s
retries: 12
redis:
image: redis:7.4.7-alpine
@@ -1343,6 +1329,11 @@ services:
- appwrite
volumes:
- appwrite-redis:/data:rw
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 5s
retries: 12
coredns: # DNS server for testing purposes (Proxy APIs)
image: coredns/coredns:1.12.4
@@ -1414,78 +1405,8 @@ services:
networks:
- appwrite
adminer:
image: adminer
container_name: appwrite-adminer
<<: *x-logging
restart: always
ports:
- 9506:8080
networks:
- appwrite
- gateway
environment:
- ADMINER_DESIGN=pepa-linha
- ADMINER_DEFAULT_SERVER=mariadb
- ADMINER_DEFAULT_USERNAME=root
- ADMINER_DEFAULT_PASSWORD=rootsecretpassword
- ADMINER_DEFAULT_DB=appwrite
configs:
- source: adminer-index.php
target: /var/www/html/index.php
mode: 0755
labels:
- "traefik.enable=true"
- "traefik.constraint-label-stack=appwrite"
- "traefik.docker.network=gateway"
- "traefik.http.services.appwrite_adminer.loadbalancer.server.port=8080"
- "traefik.http.routers.appwrite_adminer_http.entrypoints=appwrite_web"
- "traefik.http.routers.appwrite_adminer_http.rule=Host(`mysql.localhost`)"
- "traefik.http.routers.appwrite_adminer_http.service=appwrite_adminer"
- "traefik.http.routers.appwrite_adminer_https.entrypoints=appwrite_websecure"
- "traefik.http.routers.appwrite_adminer_https.rule=Host(`mysql.localhost`)"
- "traefik.http.routers.appwrite_adminer_https.service=appwrite_adminer"
- "traefik.http.routers.appwrite_adminer_https.tls=true"
redis-insight:
image: redis/redisinsight:latest
restart: unless-stopped
networks:
- appwrite
- gateway
environment:
- RI_PRE_SETUP_DATABASES_PATH=/mnt/connections.json
configs:
- source: redisinsight-connections.json
target: /mnt/connections.json
mode: 0755
labels:
- "traefik.enable=true"
- "traefik.constraint-label-stack=appwrite"
- "traefik.docker.network=gateway"
- "traefik.http.services.appwrite_redisinsight.loadbalancer.server.port=5540"
- "traefik.http.routers.appwrite_redisinsight_http.entrypoints=appwrite_web"
- "traefik.http.routers.appwrite_redisinsight_http.rule=Host(`redis.localhost`)"
- "traefik.http.routers.appwrite_redisinsight_http.service=appwrite_redisinsight"
- "traefik.http.routers.appwrite_redisinsight_https.entrypoints=appwrite_websecure"
- "traefik.http.routers.appwrite_redisinsight_https.rule=Host(`redis.localhost`)"
- "traefik.http.routers.appwrite_redisinsight_https.service=appwrite_redisinsight"
- "traefik.http.routers.appwrite_redisinsight_https.tls=true"
ports:
- "8081:5540"
graphql-explorer:
container_name: appwrite-graphql-explorer
image: appwrite/altair:0.3.0
restart: unless-stopped
networks:
- appwrite
ports:
- "9509:3000"
environment:
- SERVER_URL=http://localhost/v1/graphql
# Dev Tools End ------------------------------------------------------------------------------------------
# Dev tools (adminer, redis-insight, mongo-express, graphql-explorer)
# are defined in docker-compose.override.yml
networks:
gateway:
@@ -1498,60 +1419,7 @@ networks:
runtimes:
name: runtimes
configs:
redisinsight-connections.json:
content: |
[
{
"compressor": "NONE",
"id": "104dc90a-21ef-4d5e-8912-b30baabb152f",
"host": "redis",
"port": 6379,
"name": "redis:6379",
"db": 0,
"username": "default",
"password": null,
"connectionType": "STANDALONE",
"nameFromProvider": null,
"provider": "REDIS",
"lastConnection": "2025-10-16T09:22:02.591Z",
"modules": [
{
"name": "ReJSON",
"version": 20808,
"semanticVersion": "2.8.8"
},
{
"name": "search",
"version": 21015,
"semanticVersion": "2.10.15"
}
],
"tls": false,
"tlsServername": null,
"verifyServerCert": null,
"caCert": null,
"clientCert": null,
"ssh": false,
"sshOptions": null,
"forceStandalone": false,
"tags": []
}
]
adminer-index.php:
content: |
<?php
if(!count($$_GET)) {
$$_POST['auth'] = [
'server' => $$_ENV['ADMINER_DEFAULT_SERVER'],
'driver' => 'server', /* seems to autodetect the driver from server settings */
'username' => $$_ENV['ADMINER_DEFAULT_USERNAME'],
'password' => $$_ENV['ADMINER_DEFAULT_PASSWORD'],
'db' => $$_ENV['ADMINER_DEFAULT_DB'],
];
}
include './adminer.php';
volumes:
appwrite-mariadb:
+10
View File
@@ -0,0 +1,10 @@
{
"name": "@appwrite.io/repo",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@appwrite.io/repo"
}
}
}
-6
View File
@@ -1,6 +0,0 @@
{
"$schema":"vendor/phpbench/phpbench/phpbench.schema.json",
"runner.bootstrap": "vendor/autoload.php",
"runner.path": "tests",
"runner.file_pattern": "*Bench.php"
}
+83215
View File
File diff suppressed because it is too large Load Diff
+9 -5
View File
@@ -1,13 +1,17 @@
includes:
- phpstan-baseline.neon
parameters:
level: 8
level: max
paths:
- src/Utopia/Bus
- src/Appwrite/Bus
- src/Appwrite/Transformation
- src
- app
- bin
- tests
bootstrapFiles:
- app/init/constants.php
scanDirectories:
- vendor/swoole/ide-helper
excludePaths:
- tests/resources
- app/sdks
+1
View File
@@ -6,6 +6,7 @@
colors="true"
processIsolation="false"
stopOnFailure="false"
stopOnError="false"
cacheDirectory=".phpunit.cache"
>
<extensions>
+22 -21
View File
@@ -4,11 +4,12 @@ namespace Appwrite\Bus\Listeners;
use Appwrite\Bus\Events\ExecutionCompleted;
use Appwrite\Bus\Events\RequestCompleted;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Message\Usage as UsageMessage;
use Appwrite\Event\Publisher\Usage as Publisher;
use Appwrite\Usage\Context;
use Utopia\Bus\Event;
use Utopia\Bus\Listener;
use Utopia\Database\Document;
use Utopia\Queue\Publisher;
class Usage extends Listener
{
@@ -29,20 +30,21 @@ class Usage extends Listener
{
$this
->desc('Records usage metrics')
->inject('publisherStatsUsage')
->inject('publisherForUsage')
->inject('usage')
->callback($this->handle(...));
}
public function handle(Event $event, Publisher $publisher): void
public function handle(Event $event, Publisher $publisherForUsage, Context $usage): void
{
match (true) {
$event instanceof ExecutionCompleted => $this->handleExecutionCompleted($event, $publisher),
$event instanceof RequestCompleted => $this->handleRequestCompleted($event, $publisher),
$event instanceof ExecutionCompleted => $this->handleExecutionCompleted($event, $publisherForUsage),
$event instanceof RequestCompleted => $this->handleRequestCompleted($event, $usage),
default => null,
};
}
private function handleExecutionCompleted(ExecutionCompleted $event, Publisher $publisher): void
private function handleExecutionCompleted(ExecutionCompleted $event, Publisher $publisherForUsage): void
{
$execution = new Document($event->execution);
$resource = new Document($event->resource);
@@ -61,9 +63,7 @@ class Usage extends Listener
$compute = (int)($duration * 1000);
$mbSeconds = (int)(($spec['memory'] ?? APP_COMPUTE_MEMORY_DEFAULT) * $duration * ($spec['cpus'] ?? APP_COMPUTE_CPUS_DEFAULT));
$queueForStatsUsage = new StatsUsage($publisher);
$queueForStatsUsage
->setProject($project)
$context = (new Context())
->addMetric(METRIC_EXECUTIONS, 1)
->addMetric(str_replace(['{resourceType}'], [$resourceType], METRIC_RESOURCE_TYPE_EXECUTIONS), 1)
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$resourceType, $resourceInternalId], METRIC_RESOURCE_TYPE_ID_EXECUTIONS), 1)
@@ -72,11 +72,18 @@ class Usage extends Listener
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$resourceType, $resourceInternalId], METRIC_RESOURCE_TYPE_ID_EXECUTIONS_COMPUTE), $compute)
->addMetric(METRIC_EXECUTIONS_MB_SECONDS, $mbSeconds)
->addMetric(str_replace(['{resourceType}'], [$resourceType], METRIC_RESOURCE_TYPE_EXECUTIONS_MB_SECONDS), $mbSeconds)
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$resourceType, $resourceInternalId], METRIC_RESOURCE_TYPE_ID_EXECUTIONS_MB_SECONDS), $mbSeconds)
->trigger();
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$resourceType, $resourceInternalId], METRIC_RESOURCE_TYPE_ID_EXECUTIONS_MB_SECONDS), $mbSeconds);
$message = new UsageMessage(
project: $project,
metrics: $context->getMetrics(),
reduce: $context->getReduce()
);
$publisherForUsage->enqueue($message);
}
private function handleRequestCompleted(RequestCompleted $event, Publisher $publisher): void
private function handleRequestCompleted(RequestCompleted $event, Context $usage): void
{
$fileSize = 0;
$file = $event->request->getFiles('file');
@@ -84,18 +91,14 @@ class Usage extends Listener
$fileSize = (\is_array($file['size']) && isset($file['size'][0])) ? $file['size'][0] : $file['size'];
}
$project = new Document($event->project);
$deployment = new Document($event->deployment);
$queueForStatsUsage = new StatsUsage($publisher);
$inbound = $event->request->getSize() + $fileSize;
$outbound = $event->response->getSize();
$queueForStatsUsage->setProject($project);
if ($deployment->getAttribute('resourceType') === 'sites') {
$siteInternalId = $deployment->getAttribute('resourceInternalId', '');
$queueForStatsUsage
$usage
->addMetric(METRIC_SITES_REQUESTS, 1)
->addMetric(METRIC_SITES_INBOUND, $inbound)
->addMetric(METRIC_SITES_OUTBOUND, $outbound)
@@ -103,12 +106,10 @@ class Usage extends Listener
->addMetric(str_replace('{siteInternalId}', $siteInternalId, METRIC_SITES_ID_INBOUND), $inbound)
->addMetric(str_replace('{siteInternalId}', $siteInternalId, METRIC_SITES_ID_OUTBOUND), $outbound);
} else {
$queueForStatsUsage
$usage
->addMetric(METRIC_NETWORK_REQUESTS, 1)
->addMetric(METRIC_NETWORK_INBOUND, $inbound)
->addMetric(METRIC_NETWORK_OUTBOUND, $outbound);
}
$queueForStatsUsage->trigger();
}
}
+21
View File
@@ -0,0 +1,21 @@
<?php
namespace Appwrite\Event\Message;
abstract class Base
{
/**
* Serialize message to array for queue
*
* @return array
*/
abstract public function toArray(): array;
/**
* Deserialize message from array
*
* @param array $data
* @return static
*/
abstract public static function fromArray(array $data): static;
}
+49
View File
@@ -0,0 +1,49 @@
<?php
namespace Appwrite\Event\Message;
use Utopia\Database\Document;
class Usage extends Base
{
/**
* @param Document $project
* @param array<array{key: string, value: int}> $metrics
* @param array<Document> $reduce
*/
public function __construct(
public readonly Document $project,
public readonly array $metrics,
public readonly array $reduce = [],
) {
}
/**
* @return array
*/
public function toArray(): array
{
return [
'project' => [
'$id' => $this->project->getId(),
'$sequence' => $this->project->getSequence(),
'database' => $this->project->getAttribute('database', ''),
],
'metrics' => $this->metrics,
'reduce' => array_map(fn (Document $doc) => $doc->getArrayCopy(), $this->reduce),
];
}
/**
* @param array $data
* @return static
*/
public static function fromArray(array $data): static
{
return new self(
project: new Document($data['project'] ?? []),
metrics: $data['metrics'] ?? [],
reduce: array_map(fn (array $doc) => new Document($doc), $data['reduce'] ?? []),
);
}
}
+33
View File
@@ -0,0 +1,33 @@
<?php
namespace Appwrite\Event\Publisher;
use Appwrite\Event\Message\Base as BaseMessage;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
readonly class Base
{
public function __construct(
protected Publisher $publisher
) {
}
/**
* Publish a message to the queue
*/
public function publish(Queue $queue, BaseMessage $message): string|bool
{
$payload = $message->toArray();
return $this->publisher->enqueue($queue, $payload);
}
/**
* Get the size of a queue
*/
public function getQueueSize(Queue $queue, bool $failed = false): int
{
return $this->publisher->getQueueSize($queue, $failed);
}
}
+39
View File
@@ -0,0 +1,39 @@
<?php
namespace Appwrite\Event\Publisher;
use Appwrite\Event\Message\Usage as UsageMessage;
use Utopia\Console;
use Utopia\Queue\Publisher;
use Utopia\Queue\Queue;
readonly class Usage extends Base
{
public function __construct(
Publisher $publisher,
protected Queue $queue
) {
parent::__construct($publisher);
}
/**
* Enqueue a usage message
*/
public function enqueue(UsageMessage $message): string|bool
{
try {
return $this->publish($this->queue, $message);
} catch (\Throwable $th) {
Console::error('[Usage] Failed to publish usage message: ' . $th->getMessage());
return false;
}
}
/**
* Get the size of the usage queue
*/
public function getSize(bool $failed = false): int
{
return $this->getQueueSize($this->queue, $failed);
}
}
-96
View File
@@ -1,96 +0,0 @@
<?php
namespace Appwrite\Event;
use Utopia\Database\Document;
use Utopia\Queue\Publisher;
use Utopia\System\System;
class StatsUsage extends Event
{
protected array $metrics = [];
protected array $reduce = [];
protected array $disabled = [];
protected bool $critical = false;
public function __construct(protected Publisher $publisher)
{
parent::__construct($publisher);
$this
->setQueue(System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME))
->setClass(System::getEnv('_APP_STATS_USAGE_CLASS_NAME', Event::STATS_USAGE_CLASS_NAME));
}
/**
* Add reduce.
*
* @param Document $document
* @return self
*/
public function addReduce(Document $document): self
{
$this->reduce[] = $document;
return $this;
}
/**
* Add metric.
*
* @param string $key
* @param int $value
* @return self
*/
public function addMetric(string $key, int $value): self
{
$this->metrics[] = [
'key' => $key,
'value' => $value,
];
return $this;
}
/**
* Set disabled metrics.
*
* @param string $key
* @return self
*/
public function disableMetric(string $key): self
{
$this->disabled[] = $key;
return $this;
}
/**
* Prepare the payload for the event
*
* @return array
*/
protected function preparePayload(): array
{
return [
'project' => $this->getProject(),
'reduce' => $this->reduce,
'metrics' => \array_filter($this->metrics, function ($metric) {
foreach ($this->disabled as $disabledMetric) {
if (\str_ends_with($metric['key'], $disabledMetric)) {
return false;
}
}
return true;
}),
];
}
public function reset(): Event
{
$this->metrics = [];
parent::reset();
return $this;
}
}
@@ -119,7 +119,7 @@ class Update extends Action
$authenticator->setAttribute('verified', true);
$dbForProject->updateDocument('authenticators', $authenticator->getId(), $authenticator);
$dbForProject->updateDocument('authenticators', $authenticator->getId(), new Document(['verified' => true]));
$dbForProject->purgeCachedDocument('users', $user->getId());
$factors = $session->getAttribute('factors', []);
@@ -127,7 +127,7 @@ class Update extends Action
$factors = \array_values(\array_unique($factors));
$session->setAttribute('factors', $factors);
$dbForProject->updateDocument('sessions', $session->getId(), $session);
$dbForProject->updateDocument('sessions', $session->getId(), new Document(['factors' => $factors]));
$queueForEvents->setParam('userId', $user->getId());
@@ -7,7 +7,6 @@ use Appwrite\Detector\Detector;
use Appwrite\Event\Event;
use Appwrite\Event\Mail;
use Appwrite\Event\Messaging;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
@@ -15,6 +14,7 @@ use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Template\Template;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Request;
use Appwrite\Utopia\Response;
use libphonenumber\NumberParseException;
@@ -104,7 +104,7 @@ class Create extends Action
->inject('queueForMessaging')
->inject('queueForMails')
->inject('timelimit')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('proofForToken')
->inject('proofForCode')
@@ -124,7 +124,7 @@ class Create extends Action
Messaging $queueForMessaging,
Mail $queueForMails,
callable $timelimit,
StatsUsage $queueForStatsUsage,
Context $usage,
array $plan,
ProofsToken $proofForToken,
ProofsCode $proofForCode
@@ -201,16 +201,12 @@ class Create extends Action
$countryCode = $helper->parse($phone)->getCountryCode();
if (!empty($countryCode)) {
$queueForStatsUsage
->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
$usage->addMetric(str_replace('{countryCode}', $countryCode, METRIC_AUTH_METHOD_PHONE_COUNTRY_CODE), 1);
}
} catch (NumberParseException $e) {
// Ignore invalid phone number for country code stats
}
$queueForStatsUsage
->addMetric(METRIC_AUTH_METHOD_PHONE, 1)
->setProject($project)
->trigger();
$usage->addMetric(METRIC_AUTH_METHOD_PHONE, 1);
break;
case Type::EMAIL:
if (empty(System::getEnv('_APP_SMTP_HOST'))) {
@@ -117,7 +117,7 @@ class Update extends Action
$mfaRecoveryCodes = \array_diff($mfaRecoveryCodes, [$otp]);
$mfaRecoveryCodes = \array_values($mfaRecoveryCodes);
$user->setAttribute('mfaRecoveryCodes', $mfaRecoveryCodes);
$dbForProject->updateDocument('users', $user->getId(), $user);
$dbForProject->updateDocument('users', $user->getId(), new Document(['mfaRecoveryCodes' => $mfaRecoveryCodes]));
return true;
}
@@ -147,11 +147,13 @@ class Update extends Action
$factors[] = $type;
$factors = \array_values(\array_unique($factors));
$mfaUpdatedAt = DateTime::now();
$session
->setAttribute('factors', $factors)
->setAttribute('mfaUpdatedAt', DateTime::now());
->setAttribute('mfaUpdatedAt', $mfaUpdatedAt);
$dbForProject->updateDocument('sessions', $session->getId(), $session);
$dbForProject->updateDocument('sessions', $session->getId(), new Document(['factors' => $factors, 'mfaUpdatedAt' => $mfaUpdatedAt]));
$queueForEvents
->setParam('userId', $user->getId())
@@ -93,7 +93,7 @@ class Create extends Action
$mfaRecoveryCodes = Type::generateBackupCodes();
$user->setAttribute('mfaRecoveryCodes', $mfaRecoveryCodes);
$dbForProject->updateDocument('users', $user->getId(), $user);
$dbForProject->updateDocument('users', $user->getId(), new Document(['mfaRecoveryCodes' => $mfaRecoveryCodes]));
$queueForEvents->setParam('userId', $user->getId());
@@ -91,7 +91,7 @@ class Update extends Action
$mfaRecoveryCodes = Type::generateBackupCodes();
$user->setAttribute('mfaRecoveryCodes', $mfaRecoveryCodes);
$dbForProject->updateDocument('users', $user->getId(), $user);
$dbForProject->updateDocument('users', $user->getId(), new Document(['mfaRecoveryCodes' => $mfaRecoveryCodes]));
$queueForEvents->setParam('userId', $user->getId());
@@ -72,7 +72,7 @@ class Update extends Action
): void {
$user->setAttribute('mfa', $mfa);
$user = $dbForProject->updateDocument('users', $user->getId(), $user);
$user = $dbForProject->updateDocument('users', $user->getId(), new Document(['mfa' => $mfa]));
if ($mfa) {
$factors = $session->getAttribute('factors', []);
@@ -89,7 +89,7 @@ class Update extends Action
$factors = \array_values(\array_unique($factors));
$session->setAttribute('factors', $factors);
$dbForProject->updateDocument('sessions', $session->getId(), $session);
$dbForProject->updateDocument('sessions', $session->getId(), new Document(['factors' => $factors]));
}
$queueForEvents->setParam('userId', $user->getId());
@@ -2,7 +2,6 @@
namespace Appwrite\Platform\Modules\Avatars\Http\Screenshots;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Platform\Modules\Avatars\Http\Action;
use Appwrite\SDK\AuthType;
@@ -10,6 +9,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Method;
use Appwrite\SDK\MethodType;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Response;
use Utopia\Config\Config;
use Utopia\Domains\Domain;
@@ -84,11 +84,11 @@ class Get extends Action
->param('quality', -1, new Range(-1, 100), 'Screenshot quality. Pass an integer between 0 to 100. Defaults to keep existing image quality.', true, example: '85')
->param('output', '', new WhiteList(\array_keys(Config::getParam('storage-outputs')), true), 'Output format type (jpeg, jpg, png, gif and webp).', true, example: 'jpeg')
->inject('response')
->inject('queueForStatsUsage')
->inject('usage')
->callback($this->action(...));
}
public function action(string $url, array $headers, int $viewportWidth, int $viewportHeight, float $scale, string $theme, string $userAgent, bool $fullpage, string $locale, string $timezone, float $latitude, float $longitude, float $accuracy, bool $touch, array $permissions, int $sleep, int $width, int $height, int $quality, string $output, Response $response, StatsUsage $queueForStatsUsage)
public function action(string $url, array $headers, int $viewportWidth, int $viewportHeight, float $scale, string $theme, string $userAgent, bool $fullpage, string $locale, string $timezone, float $latitude, float $longitude, float $accuracy, bool $touch, array $permissions, int $sleep, int $width, int $height, int $quality, string $output, Response $response, Context $usage)
{
if (!\extension_loaded('imagick')) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Imagick extension is missing');
@@ -210,7 +210,7 @@ class Get extends Action
$outputs = Config::getParam('storage-outputs');
$contentType = $outputs[$output] ?? $outputs['png'];
$queueForStatsUsage->addMetric(METRIC_AVATARS_SCREENSHOTS_GENERATED, 1);
$usage->addMetric(METRIC_AVATARS_SCREENSHOTS_GENERATED, 1);
$response
->addHeader('Cache-Control', 'private, max-age=2592000') // 30 days
+12 -2
View File
@@ -143,7 +143,12 @@ class Base extends Action
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), $function);
$dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
$queueForBuilds
->setType(BUILD_TYPE_DEPLOYMENT)
@@ -249,7 +254,12 @@ class Base extends Action
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('sites', $site->getId(), $site);
$dbForProject->updateDocument('sites', $site->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
$sitesDomain = $platform['sitesDomain'];
$domain = ID::unique() . "." . $sitesDomain;
@@ -12,6 +12,7 @@ use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\IndexDependency as IndexDependencyValidator;
use Utopia\Database\Validator\Key;
@@ -98,7 +99,8 @@ class Delete extends Action
}
if ($attribute->getAttribute('status') === 'available') {
$attribute = $dbForProject->updateDocument('attributes', $attribute->getId(), $attribute->setAttribute('status', 'deleting'));
$attribute->setAttribute('status', 'deleting');
$attribute = $dbForProject->updateDocument('attributes', $attribute->getId(), new Document(['status' => 'deleting']));
}
$dbForProject->purgeCachedDocument('database_' . $db->getSequence(), $collectionId);
@@ -118,7 +120,8 @@ class Delete extends Action
}
if ($relatedAttribute->getAttribute('status') === 'available') {
$dbForProject->updateDocument('attributes', $relatedAttribute->getId(), $relatedAttribute->setAttribute('status', 'deleting'));
$relatedAttribute->setAttribute('status', 'deleting');
$dbForProject->updateDocument('attributes', $relatedAttribute->getId(), new Document(['status' => 'deleting']));
}
$dbForProject->purgeCachedDocument('database_' . $db->getSequence(), $options['relatedCollection']);
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Attribute;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
use Appwrite\SDK\AuthType;
@@ -11,6 +10,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use InvalidArgumentException;
@@ -83,13 +83,13 @@ class Decrement extends Action
->inject('response')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string $documentId, string $attribute, int|float $value, int|float|null $min, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Event $queueForEvents, StatsUsage $queueForStatsUsage, array $plan, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, string $documentId, string $attribute, int|float $value, int|float|null $min, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Event $queueForEvents, Context $usage, array $plan, Authorization $authorization): void
{
$isAPIKey = User::isApp($authorization->getRoles());
$isPrivilegedUser = User::isPrivileged($authorization->getRoles());
@@ -200,7 +200,7 @@ class Decrement extends Action
)
);
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, 1)
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), 1);
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Attribute;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
use Appwrite\SDK\AuthType;
@@ -11,6 +10,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use InvalidArgumentException;
@@ -83,13 +83,13 @@ class Increment extends Action
->inject('response')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string $documentId, string $attribute, int|float $value, int|float|null $max, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Event $queueForEvents, StatsUsage $queueForStatsUsage, array $plan, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, string $documentId, string $attribute, int|float $value, int|float|null $max, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Event $queueForEvents, Context $usage, array $plan, Authorization $authorization): void
{
$isAPIKey = User::isApp($authorization->getRoles());
$isPrivilegedUser = User::isPrivileged($authorization->getRoles());
@@ -200,7 +200,7 @@ class Increment extends Action
)
);
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, 1)
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), 1);
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Bulk;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
@@ -12,6 +11,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
use Utopia\Database\Document;
@@ -76,7 +76,7 @@ class Delete extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForEvents')
->inject('queueForRealtime')
->inject('queueForFunctions')
@@ -86,7 +86,7 @@ class Delete extends Action
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, StatsUsage $queueForStatsUsage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
public function action(string $databaseId, string $collectionId, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Context $usage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
{
$database = $dbForProject->getDocument('databases', $databaseId);
if ($database->isEmpty()) {
@@ -185,10 +185,10 @@ class Delete extends Action
foreach ($documents as $document) {
$document->setAttribute('$databaseId', $database->getId());
$document->setAttribute('$'.$this->getCollectionsEventsContext().'Id', $collection->getId());
$document->setAttribute('$' . $this->getCollectionsEventsContext() . 'Id', $collection->getId());
}
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, \max(1, $modified))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), \max(1, $modified));
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Bulk;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
@@ -12,6 +11,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
use Utopia\Database\Document;
@@ -80,7 +80,7 @@ class Update extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForEvents')
->inject('queueForRealtime')
->inject('queueForFunctions')
@@ -90,7 +90,7 @@ class Update extends Action
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string|array $data, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, StatsUsage $queueForStatsUsage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
public function action(string $databaseId, string $collectionId, string|array $data, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Context $usage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
{
$data = \is_string($data)
? \json_decode($data, true)
@@ -216,10 +216,10 @@ class Update extends Action
foreach ($documents as $document) {
$document->setAttribute('$databaseId', $database->getId());
$document->setAttribute('$'.$this->getCollectionsEventsContext().'Id', $collection->getId());
$document->setAttribute('$' . $this->getCollectionsEventsContext() . 'Id', $collection->getId());
}
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, \max(1, $modified))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), \max(1, $modified));
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Bulk;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
use Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents\Action;
@@ -12,6 +11,7 @@ use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
use Utopia\Database\Document;
@@ -78,7 +78,7 @@ class Upsert extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForEvents')
->inject('queueForRealtime')
->inject('queueForFunctions')
@@ -88,7 +88,7 @@ class Upsert extends Action
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, array $documents, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, StatsUsage $queueForStatsUsage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
public function action(string $databaseId, string $collectionId, array $documents, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Context $usage, Event $queueForEvents, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, EventProcessor $eventProcessor): void
{
$database = $dbForProject->getDocument('databases', $databaseId);
if ($database->isEmpty()) {
@@ -106,7 +106,7 @@ class Upsert extends Action
);
if ($hasRelationships) {
throw new Exception(Exception::GENERAL_BAD_REQUEST, 'Bulk upsert is not supported for ' . $this->getSDKNamespace() . ' with relationship attributes');
throw new Exception(Exception::GENERAL_BAD_REQUEST, 'Bulk upsert is not supported for ' . $this->getSDKNamespace() . ' with relationship attributes');
}
foreach ($documents as $key => $document) {
@@ -191,10 +191,10 @@ class Upsert extends Action
foreach ($upserted as $document) {
$document->setAttribute('$databaseId', $database->getId());
$document->setAttribute('$'.$this->getCollectionsEventsContext().'Id', $collection->getId());
$document->setAttribute('$' . $this->getCollectionsEventsContext() . 'Id', $collection->getId());
}
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, \max(1, $modified))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), \max(1, $modified));
@@ -3,7 +3,6 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
use Appwrite\SDK\AuthType;
@@ -12,6 +11,7 @@ use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Parameter;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Database\Validator\CustomId;
use Appwrite\Utopia\Response as UtopiaResponse;
@@ -129,7 +129,7 @@ class Create extends Action
->inject('dbForProject')
->inject('user')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForRealtime')
->inject('queueForFunctions')
->inject('queueForWebhooks')
@@ -138,7 +138,7 @@ class Create extends Action
->inject('eventProcessor')
->callback($this->action(...));
}
public function action(string $databaseId, string $documentId, string $collectionId, string|array $data, ?array $permissions, ?array $documents, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Document $user, Event $queueForEvents, StatsUsage $queueForStatsUsage, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, Authorization $authorization, EventProcessor $eventProcessor): void
public function action(string $databaseId, string $documentId, string $collectionId, string|array $data, ?array $permissions, ?array $documents, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Document $user, Event $queueForEvents, Context $usage, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, array $plan, Authorization $authorization, EventProcessor $eventProcessor): void
{
$data = \is_string($data)
? \json_decode($data, true)
@@ -205,7 +205,7 @@ class Create extends Action
);
if ($isBulk && $hasRelationships) {
throw new Exception(Exception::GENERAL_BAD_REQUEST, 'Bulk create is not supported for ' . $this->getSDKNamespace() .' with relationship ' . $this->getStructureContext());
throw new Exception(Exception::GENERAL_BAD_REQUEST, 'Bulk create is not supported for ' . $this->getSDKNamespace() . ' with relationship ' . $this->getStructureContext());
}
$setPermissions = function (Document $document, ?array $permissions) use ($user, $isAPIKey, $isPrivilegedUser, $isBulk, $dbForProject, $authorization) {
@@ -489,7 +489,7 @@ class Create extends Action
);
}
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, \max(1, $operations))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), \max(1, $operations)); // per collection
@@ -4,13 +4,13 @@ namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documen
use Appwrite\Databases\TransactionState;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
@@ -80,7 +80,7 @@ class Delete extends Action
->inject('response')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('plan')
->inject('authorization')
@@ -96,7 +96,7 @@ class Delete extends Action
UtopiaResponse $response,
Database $dbForProject,
Event $queueForEvents,
StatsUsage $queueForStatsUsage,
Context $usage,
TransactionState $transactionState,
array $plan,
Authorization $authorization
@@ -210,7 +210,7 @@ class Delete extends Action
authorization: $authorization
);
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, 1)
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), 1); // per collection
@@ -3,13 +3,13 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents;
use Appwrite\Databases\TransactionState;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
@@ -68,13 +68,13 @@ class Get extends Action
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID to read uncommitted changes within the transaction.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string $documentId, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, StatsUsage $queueForStatsUsage, TransactionState $transactionState, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, string $documentId, array $queries, ?string $transactionId, UtopiaResponse $response, Database $dbForProject, Context $usage, TransactionState $transactionState, Authorization $authorization): void
{
$isAPIKey = User::isApp($authorization->getRoles());
$isPrivilegedUser = User::isPrivileged($authorization->getRoles());
@@ -130,7 +130,7 @@ class Get extends Action
operations: $operations
);
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_READS, max($operations, 1))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_READS), $operations);
@@ -4,13 +4,13 @@ namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documen
use Appwrite\Databases\TransactionState;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
@@ -84,14 +84,14 @@ class Update extends Action
->inject('response')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('plan')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string $documentId, string|array $data, ?array $permissions, ?string $transactionId, ?\DateTime $requestTimestamp, UtopiaResponse $response, Database $dbForProject, Event $queueForEvents, StatsUsage $queueForStatsUsage, TransactionState $transactionState, array $plan, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, string $documentId, string|array $data, ?array $permissions, ?string $transactionId, ?\DateTime $requestTimestamp, UtopiaResponse $response, Database $dbForProject, Event $queueForEvents, Context $usage, TransactionState $transactionState, array $plan, Authorization $authorization): void
{
$data = (\is_string($data)) ? \json_decode($data, true) : $data; // Cast to JSON array
@@ -246,7 +246,7 @@ class Update extends Action
$setCollection($collection, $newDocument);
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, max($operations, 1))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), $operations);
@@ -4,13 +4,13 @@ namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documen
use Appwrite\Databases\TransactionState;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Database\Validator\CustomId;
use Appwrite\Utopia\Response as UtopiaResponse;
@@ -88,14 +88,14 @@ class Upsert extends Action
->inject('user')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('plan')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, string $documentId, string|array $data, ?array $permissions, ?string $transactionId, ?\DateTime $requestTimestamp, UtopiaResponse $response, Document $user, Database $dbForProject, Event $queueForEvents, StatsUsage $queueForStatsUsage, TransactionState $transactionState, array $plan, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, string $documentId, string|array $data, ?array $permissions, ?string $transactionId, ?\DateTime $requestTimestamp, UtopiaResponse $response, Document $user, Database $dbForProject, Event $queueForEvents, Context $usage, TransactionState $transactionState, array $plan, Authorization $authorization): void
{
$data = (\is_string($data)) ? \json_decode($data, true) : $data; // Cast to JSON array
@@ -256,7 +256,7 @@ class Upsert extends Action
$setCollection($collection, $newDocument);
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, \max(1, $operations))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_WRITES), \max(1, $operations));
@@ -3,13 +3,13 @@
namespace Appwrite\Platform\Modules\Databases\Http\Databases\Collections\Documents;
use Appwrite\Databases\TransactionState;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Deprecated;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
@@ -75,13 +75,13 @@ class XList extends Action
->inject('response')
->inject('dbForProject')
->inject('user')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('authorization')
->callback($this->action(...));
}
public function action(string $databaseId, string $collectionId, array $queries, ?string $transactionId, bool $includeTotal, int $ttl, UtopiaResponse $response, Database $dbForProject, Document $user, StatsUsage $queueForStatsUsage, TransactionState $transactionState, Authorization $authorization): void
public function action(string $databaseId, string $collectionId, array $queries, ?string $transactionId, bool $includeTotal, int $ttl, UtopiaResponse $response, Database $dbForProject, Document $user, Context $usage, TransactionState $transactionState, Authorization $authorization): void
{
$isAPIKey = User::isApp($authorization->getRoles());
$isPrivilegedUser = User::isPrivileged($authorization->getRoles());
@@ -228,7 +228,7 @@ class XList extends Action
);
}
$queueForStatsUsage
$usage
->addMetric(METRIC_DATABASES_OPERATIONS_READS, max($operations, 1))
->addMetric(str_replace('{databaseInternalId}', $database->getSequence(), METRIC_DATABASE_ID_OPERATIONS_READS), $operations);
@@ -12,6 +12,7 @@ use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\Key;
use Utopia\Database\Validator\UID;
@@ -96,7 +97,8 @@ class Delete extends Action
// Only update status if removing available index
if ($index->getAttribute('status') === 'available') {
$index = $dbForProject->updateDocument('indexes', $index->getId(), $index->setAttribute('status', 'deleting'));
$index->setAttribute('status', 'deleting');
$index = $dbForProject->updateDocument('indexes', $index->getId(), new Document(['status' => 'deleting']));
}
$dbForProject->purgeCachedDocument('database_' . $db->getSequence(), $collectionId);
@@ -60,7 +60,6 @@ class Delete extends Action
->inject('dbForProject')
->inject('queueForDatabase')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->callback($this->action(...));
}
@@ -5,13 +5,13 @@ namespace Appwrite\Platform\Modules\Databases\Http\Databases\Transactions;
use Appwrite\Databases\TransactionState;
use Appwrite\Event\Delete;
use Appwrite\Event\Event;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Functions\EventProcessor;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response as UtopiaResponse;
use Utopia\Database\Database;
@@ -73,7 +73,7 @@ class Update extends Action
->inject('transactionState')
->inject('queueForDeletes')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForRealtime')
->inject('queueForFunctions')
->inject('queueForWebhooks')
@@ -92,7 +92,7 @@ class Update extends Action
* @param TransactionState $transactionState
* @param Delete $queueForDeletes
* @param Event $queueForEvents
* @param StatsUsage $queueForStatsUsage
* @param Context $usage
* @param Event $queueForRealtime
* @param Event $queueForFunctions
* @param Event $queueForWebhooks
@@ -106,7 +106,7 @@ class Update extends Action
* @throws Structure
* @throws \Utopia\Http\Exception
*/
public function action(string $transactionId, bool $commit, bool $rollback, UtopiaResponse $response, Database $dbForProject, Document $user, TransactionState $transactionState, Delete $queueForDeletes, Event $queueForEvents, StatsUsage $queueForStatsUsage, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, Authorization $authorization, EventProcessor $eventProcessor): void
public function action(string $transactionId, bool $commit, bool $rollback, UtopiaResponse $response, Database $dbForProject, Document $user, TransactionState $transactionState, Delete $queueForDeletes, Event $queueForEvents, Context $usage, Event $queueForRealtime, Event $queueForFunctions, Event $queueForWebhooks, Authorization $authorization, EventProcessor $eventProcessor): void
{
if (!$commit && !$rollback) {
throw new Exception(Exception::GENERAL_BAD_REQUEST, 'Either commit or rollback must be true');
@@ -142,7 +142,7 @@ class Update extends Action
$currentDocumentId = null;
try {
$dbForProject->withTransaction(function () use ($dbForProject, $transactionState, $queueForDeletes, $transactionId, &$transaction, &$operations, &$totalOperations, &$databaseOperations, &$currentDocumentId, $queueForEvents, $queueForStatsUsage, $queueForRealtime, $queueForFunctions, $queueForWebhooks, $authorization) {
$dbForProject->withTransaction(function () use ($dbForProject, $transactionState, $queueForDeletes, $transactionId, &$transaction, &$operations, &$totalOperations, &$databaseOperations, &$currentDocumentId, $queueForEvents, $usage, $queueForRealtime, $queueForFunctions, $queueForWebhooks, $authorization) {
$authorization->skip(fn () => $dbForProject->updateDocument('transactions', $transactionId, new Document([
'status' => 'committing',
])));
@@ -279,11 +279,10 @@ class Update extends Action
throw new Exception(Exception::GENERAL_QUERY_INVALID, $e->getMessage());
}
$queueForStatsUsage
->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, $totalOperations);
$usage->addMetric(METRIC_DATABASES_OPERATIONS_WRITES, $totalOperations);
foreach ($databaseOperations as $sequence => $count) {
$queueForStatsUsage->addMetric(
$usage->addMetric(
str_replace('{databaseInternalId}', $sequence, METRIC_DATABASE_ID_OPERATIONS_WRITES),
$count
);
@@ -50,7 +50,6 @@ class Delete extends DatabaseDelete
->inject('dbForProject')
->inject('queueForDatabase')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->callback($this->action(...));
}
}
@@ -61,7 +61,7 @@ class Delete extends DocumentsDelete
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForEvents')
->inject('queueForRealtime')
->inject('queueForFunctions')
@@ -63,7 +63,7 @@ class Update extends DocumentsUpdate
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForEvents')
->inject('queueForRealtime')
->inject('queueForFunctions')
@@ -63,7 +63,7 @@ class Upsert extends DocumentsUpsert
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID for staging the operation.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForEvents')
->inject('queueForRealtime')
->inject('queueForFunctions')
@@ -66,7 +66,7 @@ class Decrement extends DecrementDocumentAttribute
->inject('response')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('authorization')
->callback($this->action(...));
@@ -66,7 +66,7 @@ class Increment extends IncrementDocumentAttribute
->inject('response')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('plan')
->inject('authorization')
->callback($this->action(...));
@@ -106,7 +106,7 @@ class Create extends DocumentCreate
->inject('dbForProject')
->inject('user')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForRealtime')
->inject('queueForFunctions')
->inject('queueForWebhooks')
@@ -68,7 +68,7 @@ class Delete extends DocumentDelete
->inject('response')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('plan')
->inject('authorization')
@@ -57,7 +57,7 @@ class Get extends DocumentGet
->param('transactionId', null, fn (Database $dbForProject) => new Nullable(new UID($dbForProject->getAdapter()->getMaxUIDLength())), 'Transaction ID to read uncommitted changes within the transaction.', true, ['dbForProject'])
->inject('response')
->inject('dbForProject')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('authorization')
->callback($this->action(...));
@@ -66,7 +66,7 @@ class Update extends DocumentUpdate
->inject('response')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('plan')
->inject('authorization')
@@ -69,7 +69,7 @@ class Upsert extends DocumentUpsert
->inject('user')
->inject('dbForProject')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('plan')
->inject('authorization')
@@ -61,7 +61,7 @@ class XList extends DocumentXList
->inject('response')
->inject('dbForProject')
->inject('user')
->inject('queueForStatsUsage')
->inject('usage')
->inject('transactionState')
->inject('authorization')
->callback($this->action(...));
@@ -57,7 +57,7 @@ class Update extends TransactionsUpdate
->inject('transactionState')
->inject('queueForDeletes')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForRealtime')
->inject('queueForFunctions')
->inject('queueForWebhooks')
@@ -322,13 +322,19 @@ class Databases extends Action
$dbForProject->updateDocument(
'attributes',
$attribute->getId(),
$attribute->setAttribute('status', 'stuck')
new Document([
'error' => $attribute->getAttribute('error'),
'status' => 'stuck',
])
);
if (!$relatedAttribute->isEmpty()) {
$dbForProject->updateDocument(
'attributes',
$relatedAttribute->getId(),
$relatedAttribute->setAttribute('status', 'stuck')
new Document([
'error' => $relatedAttribute->getAttribute('error'),
'status' => 'stuck',
])
);
}
@@ -382,7 +388,11 @@ class Databases extends Action
if ($exists) { // Delete the duplicate if created, else update in db
$this->deleteIndex($database, $collection, $index, $project, $dbForPlatform, $dbForProject, $queueForRealtime);
} else {
$dbForProject->updateDocument('indexes', $index->getId(), $index);
$dbForProject->updateDocument('indexes', $index->getId(), new Document([
'attributes' => $index->getAttribute('attributes'),
'lengths' => $index->getAttribute('lengths'),
'orders' => $index->getAttribute('orders'),
]));
}
}
}
@@ -227,7 +227,9 @@ class Create extends Action
foreach ($activeDeployments as $activeDeployment) {
$activeDeployment->setAttribute('activate', false);
$dbForProject->updateDocument('deployments', $activeDeployment->getId(), $activeDeployment);
$dbForProject->updateDocument('deployments', $activeDeployment->getId(), new Document([
'activate' => false,
]));
}
}
@@ -255,14 +257,17 @@ class Create extends Action
'type' => $type
]));
$function = $function
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), $function);
$function = $dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
} else {
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, $deployment->setAttribute('sourceSize', $fileSize)->setAttribute('sourceMetadata', $metadata));
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'sourceSize' => $fileSize,
'sourceMetadata' => $metadata,
]));
}
// Start the build
@@ -295,14 +300,17 @@ class Create extends Action
'type' => $type
]));
$function = $function
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), $function);
$function = $dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
} else {
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, $deployment->setAttribute('sourceChunksUploaded', $chunksUploaded)->setAttribute('sourceMetadata', $metadata));
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'sourceChunksUploaded' => $chunksUploaded,
'sourceMetadata' => $metadata,
]));
}
}
@@ -107,11 +107,12 @@ class Delete extends Action
$function = $dbForProject->updateDocument(
'functions',
$function->getId(),
$function
->setAttribute('latestDeploymentCreatedAt', $latestDeployment->isEmpty() ? '' : $latestDeployment->getCreatedAt())
->setAttribute('latestDeploymentInternalId', $latestDeployment->isEmpty() ? '' : $latestDeployment->getSequence())
->setAttribute('latestDeploymentId', $latestDeployment->isEmpty() ? '' : $latestDeployment->getId())
->setAttribute('latestDeploymentStatus', $latestDeployment->isEmpty() ? '' : $latestDeployment->getAttribute('status', ''))
new Document([
'latestDeploymentCreatedAt' => $latestDeployment->isEmpty() ? '' : $latestDeployment->getCreatedAt(),
'latestDeploymentInternalId' => $latestDeployment->isEmpty() ? '' : $latestDeployment->getSequence(),
'latestDeploymentId' => $latestDeployment->isEmpty() ? '' : $latestDeployment->getId(),
'latestDeploymentStatus' => $latestDeployment->isEmpty() ? '' : $latestDeployment->getAttribute('status', ''),
])
);
}
@@ -10,6 +10,7 @@ use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Helpers\ID;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
@@ -119,7 +120,12 @@ class Create extends Action
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), $function);
$dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $function->getAttribute('latestDeploymentId'),
'latestDeploymentInternalId' => $function->getAttribute('latestDeploymentInternalId'),
'latestDeploymentCreatedAt' => $function->getAttribute('latestDeploymentCreatedAt'),
'latestDeploymentStatus' => $function->getAttribute('latestDeploymentStatus'),
]));
$queueForBuilds
->setType(BUILD_TYPE_DEPLOYMENT)
@@ -91,7 +91,7 @@ class Update extends Action
$endTime = new \DateTime('now');
$duration = $endTime->getTimestamp() - $startTime->getTimestamp();
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment->setAttributes([
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildEndedAt' => DateTime::now(),
'buildDuration' => $duration,
'status' => 'canceled'
@@ -99,7 +99,9 @@ class Update extends Action
if ($deployment->getSequence() === $function->getAttribute('latestDeploymentInternalId', '')) {
$function = $function->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), $function);
$dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentStatus' => $function->getAttribute('latestDeploymentStatus'),
]));
}
try {
@@ -174,7 +174,12 @@ class Create extends Base
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), $function);
$dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $function->getAttribute('latestDeploymentId'),
'latestDeploymentInternalId' => $function->getAttribute('latestDeploymentInternalId'),
'latestDeploymentCreatedAt' => $function->getAttribute('latestDeploymentCreatedAt'),
'latestDeploymentStatus' => $function->getAttribute('latestDeploymentStatus'),
]));
$this->updateEmptyManualRule($project, $function, $deployment, $dbForPlatform, $authorization);
@@ -6,7 +6,6 @@ use Ahc\Jwt\JWT;
use Appwrite\Event\Delete as DeleteEvent;
use Appwrite\Event\Event;
use Appwrite\Event\Func;
use Appwrite\Event\StatsUsage;
use Appwrite\Extend\Exception;
use Appwrite\Extend\Exception as AppwriteException;
use Appwrite\Functions\Validator\Headers;
@@ -15,6 +14,7 @@ use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Database\Documents\User;
use Appwrite\Utopia\Response;
use Executor\Executor;
@@ -93,7 +93,7 @@ class Create extends Base
->inject('dbForPlatform')
->inject('user')
->inject('queueForEvents')
->inject('queueForStatsUsage')
->inject('usage')
->inject('queueForFunctions')
->inject('geodb')
->inject('store')
@@ -121,7 +121,7 @@ class Create extends Base
Database $dbForPlatform,
Document $user,
Event $queueForEvents,
StatsUsage $queueForStatsUsage,
Context $usage,
Func $queueForFunctions,
Reader $geodb,
Store $store,
@@ -180,7 +180,8 @@ class Create extends Base
$version = $function->getAttribute('version', 'v2');
$runtimes = Config::getParam($version === 'v2' ? 'runtimes-v2' : 'runtimes', []);
$spec = Config::getParam('specifications')[$function->getAttribute('specification', APP_COMPUTE_SPECIFICATION_DEFAULT)];
$spec = Config::getParam('specifications')[$function->getAttribute('runtimeSpecification', APP_COMPUTE_SPECIFICATION_DEFAULT)];
$runtime = (isset($runtimes[$function->getAttribute('runtime', '')])) ? $runtimes[$function->getAttribute('runtime', '')] : null;
@@ -420,6 +421,11 @@ class Create extends Base
try {
$version = $function->getAttribute('version', 'v2');
$command = $runtime['startCommand'];
if (!empty($deployment->getAttribute('startCommand', ''))) {
$command = 'cd /usr/local/server/src/function/ && ' . $deployment->getAttribute('startCommand', '');
}
$source = $deployment->getAttribute('buildPath', '');
$extension = str_ends_with($source, '.tar') ? 'tar' : 'tar.gz';
$command = $version === 'v2' ? '' : "cp /tmp/code.$extension /mnt/code/code.$extension && nohup helpers/start.sh \"$command\"";
@@ -493,7 +499,7 @@ class Create extends Base
throw $th;
}
} finally {
$queueForStatsUsage
$usage
->addMetric(METRIC_EXECUTIONS, 1)
->addMetric(str_replace(['{resourceType}'], [RESOURCE_TYPE_FUNCTIONS], METRIC_RESOURCE_TYPE_EXECUTIONS), 1)
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [RESOURCE_TYPE_FUNCTIONS, $function->getSequence()], METRIC_RESOURCE_TYPE_ID_EXECUTIONS), 1)
@@ -12,6 +12,7 @@ use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\DateTime;
use Utopia\Database\Document;
use Utopia\Database\Query;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\UID;
@@ -119,7 +120,10 @@ class Delete extends Base
->setAttribute('resourceUpdatedAt', DateTime::now())
->setAttribute('active', false);
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), $schedule));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), new Document([
'resourceUpdatedAt' => $schedule->getAttribute('resourceUpdatedAt'),
'active' => $schedule->getAttribute('active'),
])));
}
}
@@ -93,16 +93,23 @@ class Create extends Base
->param('providerBranch', '', new Text(128, 0), 'Production branch for the repo linked to the function.', true)
->param('providerSilentMode', false, new Boolean(), 'Is the VCS (Version Control System) connection in silent mode for the repo linked to the function? In silent mode, comments will not be made on commits and pull requests.', true)
->param('providerRootDirectory', '', new Text(128, 0), 'Path to function code in the linked repo.', true)
->param('specification', fn (array $plan) => $this->getDefaultSpecification($plan), fn (array $plan) => new Specification(
->param('buildSpecification', fn (array $plan) => $this->getDefaultSpecification($plan), fn (array $plan) => new Specification(
$plan,
Config::getParam('specifications', []),
System::getEnv('_APP_COMPUTE_CPUS', 0),
System::getEnv('_APP_COMPUTE_MEMORY', 0)
), 'Runtime specification for the function and builds.', true, ['plan'])
), 'Build specification for the function deployments.', true, ['plan'])
->param('runtimeSpecification', fn (array $plan) => $this->getDefaultSpecification($plan), fn (array $plan) => new Specification(
$plan,
Config::getParam('specifications', []),
System::getEnv('_APP_COMPUTE_CPUS', 0),
System::getEnv('_APP_COMPUTE_MEMORY', 0)
), 'Runtime specification for the function executions.', true, ['plan'])
->param('templateRepository', '', new Text(128, 0), 'Repository name of the template.', true, deprecated: true)
->param('templateOwner', '', new Text(128, 0), 'The name of the owner of the template.', true, deprecated: true)
->param('templateRootDirectory', '', new Text(128, 0), 'Path to function code in the template repo.', true, deprecated: true)
->param('templateVersion', '', new Text(128, 0), 'Version (tag) for the repo linked to the function template.', true, deprecated: true)
->param('deploymentRetention', 0, new Range(0, APP_COMPUTE_DEPLOYMENT_MAX_RETENTION), 'Days to keep non-active deployments before deletion. Value 0 means all deployments will be kept.', true)
->inject('response')
->inject('dbForProject')
->inject('timelimit')
@@ -138,11 +145,13 @@ class Create extends Base
string $providerBranch,
bool $providerSilentMode,
string $providerRootDirectory,
string $specification,
string $buildSpecification,
string $runtimeSpecification,
string $templateRepository,
string $templateOwner,
string $templateRootDirectory,
string $templateVersion,
int $deploymentRetention,
Response $response,
Database $dbForProject,
callable $timelimit,
@@ -215,6 +224,7 @@ class Create extends Base
'logging' => $logging,
'name' => $name,
'runtime' => $runtime,
'deploymentRetention' => $deploymentRetention,
'deploymentInternalId' => '',
'deploymentId' => '',
'events' => $events,
@@ -225,7 +235,6 @@ class Create extends Base
'entrypoint' => $entrypoint,
'commands' => $commands,
'scopes' => $scopes,
'deploymentRetention' => 0,
'startCommand' => '',
'search' => implode(' ', [$functionId, $name, $runtime]),
'version' => 'v5',
@@ -237,9 +246,8 @@ class Create extends Base
'providerBranch' => $providerBranch,
'providerRootDirectory' => $providerRootDirectory,
'providerSilentMode' => $providerSilentMode,
'specification' => $specification,
'buildSpecification' => $specification,
'runtimeSpecification' => $specification,
'buildSpecification' => $buildSpecification,
'runtimeSpecification' => $runtimeSpecification,
]));
} catch (DuplicateException) {
throw new Exception(Exception::FUNCTION_ALREADY_EXISTS);
@@ -283,7 +291,12 @@ class Create extends Base
$function->setAttribute('repositoryInternalId', $repository->getSequence());
}
$function = $dbForProject->updateDocument('functions', $function->getId(), $function);
$function = $dbForProject->updateDocument('functions', $function->getId(), new Document([
'scheduleId' => $function->getAttribute('scheduleId'),
'scheduleInternalId' => $function->getAttribute('scheduleInternalId'),
'repositoryId' => $function->getAttribute('repositoryId'),
'repositoryInternalId' => $function->getAttribute('repositoryInternalId'),
]));
// Backwards compatibility with 1.6 behaviour
$requestFormat = $request->getHeader('x-appwrite-response-format', System::getEnv('_APP_SYSTEM_RESPONSE_FORMAT', ''));
@@ -321,12 +334,12 @@ class Create extends Base
referenceType: 'branch'
);
$function = $function
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), $function);
$function = $dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
} elseif (!$template->isEmpty()) {
// Deploy non-VCS from template
$deploymentId = ID::unique();
@@ -347,12 +360,12 @@ class Create extends Base
'activate' => true,
]));
$function = $function
->setAttribute('latestDeploymentId', $deployment->getId())
->setAttribute('latestDeploymentInternalId', $deployment->getSequence())
->setAttribute('latestDeploymentCreatedAt', $deployment->getCreatedAt())
->setAttribute('latestDeploymentStatus', $deployment->getAttribute('status', ''));
$dbForProject->updateDocument('functions', $function->getId(), $function);
$function = $dbForProject->updateDocument('functions', $function->getId(), new Document([
'latestDeploymentId' => $deployment->getId(),
'latestDeploymentInternalId' => $deployment->getSequence(),
'latestDeploymentCreatedAt' => $deployment->getCreatedAt(),
'latestDeploymentStatus' => $deployment->getAttribute('status', ''),
]));
$queueForBuilds
->setType(BUILD_TYPE_DEPLOYMENT)
@@ -13,6 +13,7 @@ use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\DateTime;
use Utopia\Database\Document;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
@@ -90,7 +91,10 @@ class Delete extends Base
$schedule
->setAttribute('resourceUpdatedAt', DateTime::now())
->setAttribute('active', false);
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), $schedule));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), new Document([
'resourceUpdatedAt' => $schedule->getAttribute('resourceUpdatedAt'),
'active' => $schedule->getAttribute('active'),
])));
}
$queueForDeletes
@@ -103,7 +103,11 @@ class Update extends Base
->setAttribute('resourceUpdatedAt', DateTime::now())
->setAttribute('schedule', $function->getAttribute('schedule'))
->setAttribute('active', !empty($function->getAttribute('schedule')) && !empty($function->getAttribute('deploymentId')));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), $schedule));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), new Document([
'resourceUpdatedAt' => $schedule->getAttribute('resourceUpdatedAt'),
'schedule' => $schedule->getAttribute('schedule'),
'active' => $schedule->getAttribute('active'),
])));
$queries = [
Query::equal('trigger', ['manual']),
@@ -119,7 +123,10 @@ class Update extends Base
->setAttribute('deploymentId', $deployment->getId())
->setAttribute('deploymentInternalId', $deployment->getSequence());
$authorization->skip(fn () => $dbForPlatform->updateDocument('rules', $rule->getId(), $rule));
$authorization->skip(fn () => $dbForPlatform->updateDocument('rules', $rule->getId(), new Document([
'deploymentId' => $rule->getAttribute('deploymentId'),
'deploymentInternalId' => $rule->getAttribute('deploymentInternalId'),
])));
}, $queries));
$queueForEvents
@@ -87,12 +87,19 @@ class Update extends Base
->param('providerBranch', '', new Text(128, 0), 'Production branch for the repo linked to the function', true)
->param('providerSilentMode', false, new Boolean(), 'Is the VCS (Version Control System) connection in silent mode for the repo linked to the function? In silent mode, comments will not be made on commits and pull requests.', true)
->param('providerRootDirectory', '', new Text(128, 0), 'Path to function code in the linked repo.', true)
->param('specification', fn (array $plan) => $this->getDefaultSpecification($plan), fn (array $plan) => new Specification(
->param('buildSpecification', fn (array $plan) => $this->getDefaultSpecification($plan), fn (array $plan) => new Specification(
$plan,
Config::getParam('specifications', []),
System::getEnv('_APP_COMPUTE_CPUS', 0),
System::getEnv('_APP_COMPUTE_MEMORY', 0)
), 'Runtime specification for the function and builds.', true, ['plan'])
), 'Build specification for the function deployments.', true, ['plan'])
->param('runtimeSpecification', fn (array $plan) => $this->getDefaultSpecification($plan), fn (array $plan) => new Specification(
$plan,
Config::getParam('specifications', []),
System::getEnv('_APP_COMPUTE_CPUS', 0),
System::getEnv('_APP_COMPUTE_MEMORY', 0)
), 'Runtime specification for the function executions.', true, ['plan'])
->param('deploymentRetention', 0, new Range(0, APP_COMPUTE_DEPLOYMENT_MAX_RETENTION), 'Days to keep non-active deployments before deletion. Value 0 means all deployments will be kept.', true)
->inject('request')
->inject('response')
->inject('dbForProject')
@@ -124,7 +131,9 @@ class Update extends Base
string $providerBranch,
bool $providerSilentMode,
string $providerRootDirectory,
string $specification,
string $buildSpecification,
string $runtimeSpecification,
int $deploymentRetention,
Request $request,
Response $response,
Database $dbForProject,
@@ -209,7 +218,7 @@ class Update extends Base
'resourceId' => $function->getId(),
'resourceInternalId' => $function->getSequence(),
'resourceType' => 'function',
'providerPullRequestIds' => []
'providerPullRequestIds' => [],
]));
$repositoryId = $repository->getId();
@@ -229,13 +238,22 @@ class Update extends Base
}
// Enforce Cold Start if spec limits change.
if ($function->getAttribute('specification') !== $specification && !empty($function->getAttribute('deploymentId'))) {
try {
$executor->deleteRuntime($project->getId(), $function->getAttribute('deploymentId'));
} catch (\Throwable $th) {
// Don't throw if the deployment doesn't exist
if ($th->getCode() !== 404) {
throw $th;
if (!empty($function->getAttribute('deploymentId'))) {
$specsChanged = false;
if ($function->getAttribute('runtimeSpecification', '') !== $runtimeSpecification) {
$specsChanged = true;
} elseif ($function->getAttribute('buildSpecification', '') !== $buildSpecification) {
$specsChanged = true;
}
if ($specsChanged) {
try {
$executor->deleteRuntime($project->getId(), $function->getAttribute('deploymentId'));
} catch (\Throwable $th) {
// Don't throw if the deployment doesn't exist
if ($th->getCode() !== 404) {
throw $th;
}
}
}
}
@@ -253,8 +271,7 @@ class Update extends Base
'entrypoint' => $entrypoint,
'commands' => $commands,
'scopes' => $scopes,
'deploymentRetention' => 0,
'startCommand' => '',
'deploymentRetention' => $deploymentRetention,
'installationId' => $installation->getId(),
'installationInternalId' => $installation->getSequence(),
'providerRepositoryId' => $providerRepositoryId,
@@ -263,9 +280,8 @@ class Update extends Base
'providerBranch' => $providerBranch,
'providerRootDirectory' => $providerRootDirectory,
'providerSilentMode' => $providerSilentMode,
'specification' => $specification,
'buildSpecification' => $specification,
'runtimeSpecification' => $specification,
'buildSpecification' => $buildSpecification,
'runtimeSpecification' => $runtimeSpecification,
'search' => implode(' ', [$functionId, $name, $runtime]),
])));
@@ -105,7 +105,8 @@ class Create extends Base
throw new Exception(Exception::VARIABLE_ALREADY_EXISTS);
}
$dbForProject->updateDocument('functions', $function->getId(), $function->setAttribute('live', false));
$function->setAttribute('live', false);
$dbForProject->updateDocument('functions', $function->getId(), new Document(['live' => false]));
// Inform scheduler to pull the latest changes
$schedule = $dbForPlatform->getDocument('schedules', $function->getAttribute('scheduleId'));
@@ -113,7 +114,11 @@ class Create extends Base
->setAttribute('resourceUpdatedAt', DateTime::now())
->setAttribute('schedule', $function->getAttribute('schedule'))
->setAttribute('active', !empty($function->getAttribute('schedule')) && !empty($function->getAttribute('deploymentId')));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), $schedule));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), new Document([
'resourceUpdatedAt' => $schedule->getAttribute('resourceUpdatedAt'),
'schedule' => $schedule->getAttribute('schedule'),
'active' => $schedule->getAttribute('active'),
])));
$response
->setStatusCode(Response::STATUS_CODE_CREATED)
@@ -11,6 +11,7 @@ use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\DateTime;
use Utopia\Database\Document;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
@@ -86,7 +87,8 @@ class Delete extends Base
$dbForProject->deleteDocument('variables', $variable->getId());
$dbForProject->updateDocument('functions', $function->getId(), $function->setAttribute('live', false));
$function->setAttribute('live', false);
$dbForProject->updateDocument('functions', $function->getId(), new Document(['live' => false]));
// Inform scheduler to pull the latest changes
$schedule = $dbForPlatform->getDocument('schedules', $function->getAttribute('scheduleId'));
@@ -94,7 +96,11 @@ class Delete extends Base
->setAttribute('resourceUpdatedAt', DateTime::now())
->setAttribute('schedule', $function->getAttribute('schedule'))
->setAttribute('active', !empty($function->getAttribute('schedule')) && !empty($function->getAttribute('deploymentId')));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), $schedule));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), new Document([
'resourceUpdatedAt' => $schedule->getAttribute('resourceUpdatedAt'),
'schedule' => $schedule->getAttribute('schedule'),
'active' => $schedule->getAttribute('active'),
])));
$response->noContent();
}
@@ -10,6 +10,7 @@ use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\DateTime;
use Utopia\Database\Document;
use Utopia\Database\Exception\Duplicate as DuplicateException;
use Utopia\Database\Validator\Authorization;
use Utopia\Database\Validator\UID;
@@ -99,12 +100,18 @@ class Update extends Base
->setAttribute('search', implode(' ', [$variableId, $function->getId(), $key, 'function']));
try {
$dbForProject->updateDocument('variables', $variable->getId(), $variable);
$dbForProject->updateDocument('variables', $variable->getId(), new Document([
'key' => $key,
'value' => $value ?? $variable->getAttribute('value'),
'secret' => $secret ?? $variable->getAttribute('secret'),
'search' => implode(' ', [$variableId, $function->getId(), $key, 'function']),
]));
} catch (DuplicateException $th) {
throw new Exception(Exception::VARIABLE_ALREADY_EXISTS);
}
$dbForProject->updateDocument('functions', $function->getId(), $function->setAttribute('live', false));
$function->setAttribute('live', false);
$dbForProject->updateDocument('functions', $function->getId(), new Document(['live' => false]));
// Inform scheduler to pull the latest changes
$schedule = $dbForPlatform->getDocument('schedules', $function->getAttribute('scheduleId'));
@@ -112,7 +119,11 @@ class Update extends Base
->setAttribute('resourceUpdatedAt', DateTime::now())
->setAttribute('schedule', $function->getAttribute('schedule'))
->setAttribute('active', !empty($function->getAttribute('schedule')) && !empty($function->getAttribute('deploymentId')));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), $schedule));
$authorization->skip(fn () => $dbForPlatform->updateDocument('schedules', $schedule->getId(), new Document([
'resourceUpdatedAt' => $schedule->getAttribute('resourceUpdatedAt'),
'schedule' => $schedule->getAttribute('schedule'),
'active' => $schedule->getAttribute('active'),
])));
$response->dynamic($variable, Response::MODEL_VARIABLE);
}
@@ -5,11 +5,13 @@ namespace Appwrite\Platform\Modules\Functions\Workers;
use Ahc\Jwt\JWT;
use Appwrite\Event\Event;
use Appwrite\Event\Func;
use Appwrite\Event\Message\Usage as UsageMessage;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Realtime;
use Appwrite\Event\Screenshot;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Webhook;
use Appwrite\Filter\BranchDomain as BranchDomainFilter;
use Appwrite\Usage\Context;
use Appwrite\Utopia\Response\Model\Deployment;
use Appwrite\Vcs\Comment;
use Exception;
@@ -60,7 +62,8 @@ class Builds extends Action
->inject('queueForWebhooks')
->inject('queueForFunctions')
->inject('queueForRealtime')
->inject('queueForStatsUsage')
->inject('usage')
->inject('publisherForUsage')
->inject('cache')
->inject('dbForProject')
->inject('deviceForFunctions')
@@ -74,24 +77,6 @@ class Builds extends Action
}
/**
* @param Message $message
* @param Document $project
* @param Database $dbForPlatform
* @param Event $queueForEvents
* @param Screenshot $queueForScreenshots
* @param Webhook $queueForWebhooks
* @param Func $queueForFunctions
* @param Realtime $queueForRealtime
* @param StatsUsage $queueForStatsUsage
* @param Cache $cache
* @param Database $dbForProject
* @param Device $deviceForFunctions
* @param Device $deviceForSites
* @param Device $deviceForFiles
* @param Log $log
* @param Executor $executor
* @param array $plan
* @return void
* @throws \Utopia\Database\Exception
*/
public function action(
@@ -103,7 +88,8 @@ class Builds extends Action
Webhook $queueForWebhooks,
Func $queueForFunctions,
Realtime $queueForRealtime,
StatsUsage $queueForStatsUsage,
Context $usage,
UsagePublisher $publisherForUsage,
Cache $cache,
Database $dbForProject,
Device $deviceForFunctions,
@@ -145,7 +131,8 @@ class Builds extends Action
$queueForFunctions,
$queueForRealtime,
$queueForEvents,
$queueForStatsUsage,
$usage,
$publisherForUsage,
$dbForPlatform,
$dbForProject,
$github,
@@ -167,28 +154,7 @@ class Builds extends Action
}
/**
* @param Device $deviceForFunctions
* @param Device $deviceForSites
* @param Device $deviceForFiles
* @param Screenshot $queueForScreenshots
* @param Webhook $queueForWebhooks
* @param Func $queueForFunctions
* @param Realtime $queueForRealtime
* @param Event $queueForEvents
* @param StatsUsage $queueForStatsUsage
* @param Database $dbForPlatform
* @param Database $dbForProject
* @param GitHub $github
* @param Document $project
* @param Document $resource
* @param Document $deployment
* @param Document $template
* @param Log $log
* @param Executor $executor
* @param array $plan
* @return void
* @throws \Utopia\Database\Exception
*
* @throws Exception
*/
protected function buildDeployment(
@@ -200,7 +166,8 @@ class Builds extends Action
Func $queueForFunctions,
Realtime $queueForRealtime,
Event $queueForEvents,
StatsUsage $queueForStatsUsage,
Context $usage,
UsagePublisher $publisherForUsage,
Database $dbForPlatform,
Database $dbForProject,
GitHub $github,
@@ -255,7 +222,7 @@ class Builds extends Action
$version = $this->getVersion($resource);
$runtime = $this->getRuntime($resource, $version);
$spec = Config::getParam('specifications')[$resource->getAttribute('specification', APP_COMPUTE_SPECIFICATION_DEFAULT)];
$spec = Config::getParam('specifications')[$resource->getAttribute('buildSpecification', APP_COMPUTE_SPECIFICATION_DEFAULT)];
if ($resource->getCollection() === 'functions' && \is_null($runtime)) {
throw new \Exception('Runtime "' . $resource->getAttribute('runtime', '') . '" is not supported');
@@ -272,6 +239,7 @@ class Builds extends Action
if ($deployment->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -279,7 +247,10 @@ class Builds extends Action
$deployment->setAttribute('buildStartedAt', $startTime);
$deployment->setAttribute('status', 'processing');
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildStartedAt' => $startTime,
'status' => 'processing',
]));
if ($deployment->getSequence() === $resource->getAttribute('latestDeploymentInternalId', '')) {
$resource = $dbForProject->updateDocument($resource->getCollection(), $resource->getId(), new Document(['latestDeploymentStatus' => $deployment->getAttribute('status', '')]));
@@ -295,7 +266,7 @@ class Builds extends Action
$installationId = $deployment->getAttribute('installationId', '');
$providerRepositoryId = $deployment->getAttribute('providerRepositoryId', '');
$providerCommitHash = $deployment->getAttribute('providerCommitHash', '');
$isVcsEnabled = !empty($providerRepositoryId);
$isVcsEnabled = ! empty($providerRepositoryId);
$owner = '';
$repositoryName = '';
@@ -309,7 +280,7 @@ class Builds extends Action
}
try {
if (!$isVcsEnabled) {
if (! $isVcsEnabled) {
// Non-VCS + Template
$templateRepositoryName = $template->getAttribute('repositoryName', '');
$templateOwnerName = $template->getAttribute('ownerName', '');
@@ -321,7 +292,7 @@ class Builds extends Action
$templateRootDirectory = \ltrim($templateRootDirectory, '.');
$templateRootDirectory = \ltrim($templateRootDirectory, '/');
if (!empty($templateRepositoryName) && !empty($templateOwnerName) && !empty($templateReferenceType) && !empty($templateReferenceValue)) {
if (! empty($templateRepositoryName) && ! empty($templateOwnerName) && ! empty($templateReferenceType) && ! empty($templateReferenceValue)) {
$stdout = '';
$stderr = '';
@@ -355,8 +326,8 @@ class Builds extends Action
$source = $device->getPath($deployment->getId() . '.' . \pathinfo('code.tar.gz', PATHINFO_EXTENSION));
$result = $localDevice->transfer($tmpPathFile, $source, $device);
if (!$result) {
throw new \Exception("Unable to move file");
if (! $result) {
throw new \Exception('Unable to move file');
}
Console::execute('rm -rf ' . \escapeshellarg($tmpTemplateDirectory), '', $stdout, $stderr);
@@ -366,7 +337,11 @@ class Builds extends Action
->setAttribute('sourcePath', $source)
->setAttribute('sourceSize', $directorySize)
->setAttribute('totalSize', $directorySize);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'sourcePath' => $deployment->getAttribute('sourcePath'),
'sourceSize' => $deployment->getAttribute('sourceSize'),
'totalSize' => $deployment->getAttribute('totalSize'),
]));
$queueForRealtime
->setPayload($deployment->getArrayCopy())
@@ -393,7 +368,7 @@ class Builds extends Action
$cloneVersion = $branchName;
$cloneType = GitHub::CLONE_TYPE_BRANCH;
if (!empty($commitHash)) {
if (! empty($commitHash)) {
$cloneVersion = $commitHash;
$cloneType = GitHub::CLONE_TYPE_COMMIT;
}
@@ -406,6 +381,7 @@ class Builds extends Action
if ($dbForProject->getDocument('deployments', $deploymentId)->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -422,7 +398,7 @@ class Builds extends Action
$rootDirectoryWithoutSpaces = str_replace(' ', '', $rootDirectory);
$from = $tmpDirectory . '/' . $rootDirectory;
$to = $tmpDirectory . '/' . $rootDirectoryWithoutSpaces;
$exit = Console::execute('mv "' . \escapeshellarg($from) . '" "' . \escapeshellarg($to) . '"', '', $stdout, $stderr);
$exit = Console::execute('mv ' . \escapeshellarg($from) . ' ' . \escapeshellarg($to), '', $stdout, $stderr);
if ($exit !== 0) {
throw new \Exception('Unable to move function with spaces' . $stderr);
@@ -430,7 +406,6 @@ class Builds extends Action
$rootDirectory = $rootDirectoryWithoutSpaces;
}
// Build from template
$templateRepositoryName = $template->getAttribute('repositoryName', '');
$templateOwnerName = $template->getAttribute('ownerName', '');
@@ -442,7 +417,7 @@ class Builds extends Action
$templateRootDirectory = \ltrim($templateRootDirectory, '.');
$templateRootDirectory = \ltrim($templateRootDirectory, '/');
if (!empty($templateRepositoryName) && !empty($templateOwnerName) && !empty($templateReferenceType) && !empty($templateReferenceValue)) {
if (! empty($templateRepositoryName) && ! empty($templateOwnerName) && ! empty($templateReferenceType) && ! empty($templateReferenceValue)) {
// Clone template repo
$tmpTemplateDirectory = '/tmp/builds/' . $deploymentId . '/template';
@@ -461,7 +436,7 @@ class Builds extends Action
Console::execute('rsync -av --exclude \'.git\' ' . \escapeshellarg($tmpTemplateDirectory . '/' . $templateRootDirectory . '/') . ' ' . \escapeshellarg($tmpDirectory . '/' . $rootDirectory), '', $stdout, $stderr);
// Commit and push
$exit = Console::execute('git config --global user.email '. \escapeshellarg(APP_VCS_GITHUB_EMAIL) .' && git config --global user.name '. \escapeshellarg(APP_VCS_GITHUB_USERNAME) .' && cd ' . \escapeshellarg($tmpDirectory) . ' && git checkout -b ' . \escapeshellarg($branchName) . ' && git add . && git commit -m "Create ' . \escapeshellarg($resource->getAttribute('name', '')) . ' function" && git push origin ' . \escapeshellarg($branchName), '', $stdout, $stderr);
$exit = Console::execute('git config --global user.email ' . \escapeshellarg(APP_VCS_GITHUB_EMAIL) . ' && git config --global user.name ' . \escapeshellarg(APP_VCS_GITHUB_USERNAME) . ' && cd ' . \escapeshellarg($tmpDirectory) . ' && git checkout -b ' . \escapeshellarg($branchName) . ' && git add . && git commit -m "Create ' . \escapeshellarg($resource->getAttribute('name', '')) . ' function" && git push origin ' . \escapeshellarg($branchName), '', $stdout, $stderr);
if ($exit !== 0) {
throw new \Exception('Unable to push code repository: ' . $stderr);
@@ -480,7 +455,13 @@ class Builds extends Action
$deployment->setAttribute('providerCommitAuthor', APP_VCS_GITHUB_USERNAME);
$deployment->setAttribute('providerCommitMessage', "Create '" . $resource->getAttribute('name', '') . "' function");
$deployment->setAttribute('providerCommitUrl', "https://github.com/$cloneOwner/$cloneRepository/commit/$providerCommitHash");
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'providerCommitHash' => $deployment->getAttribute('providerCommitHash'),
'providerCommitAuthorUrl' => $deployment->getAttribute('providerCommitAuthorUrl'),
'providerCommitAuthor' => $deployment->getAttribute('providerCommitAuthor'),
'providerCommitMessage' => $deployment->getAttribute('providerCommitMessage'),
'providerCommitUrl' => $deployment->getAttribute('providerCommitUrl'),
]));
$queueForRealtime
->setPayload($deployment->getArrayCopy())
@@ -498,7 +479,7 @@ class Builds extends Action
}
$directorySize = $localDevice->getDirectorySize($tmpDirectory);
$sizeLimit = (int)System::getEnv('_APP_COMPUTE_SIZE_LIMIT', '30000000');
$sizeLimit = (int) System::getEnv('_APP_COMPUTE_SIZE_LIMIT', '30000000');
if (isset($plan['deploymentSize'])) {
$sizeLimit = (int) $plan['deploymentSize'] * 1000 * 1000;
@@ -516,8 +497,8 @@ class Builds extends Action
$source = $device->getPath($deployment->getId() . '.' . \pathinfo('code.tar.gz', PATHINFO_EXTENSION));
$result = $localDevice->transfer($tmpPathFile, $source, $device);
if (!$result) {
throw new \Exception("Unable to move file");
if (! $result) {
throw new \Exception('Unable to move file');
}
Console::execute('rm -rf ' . \escapeshellarg($tmpPath), '', $stdout, $stderr);
@@ -528,7 +509,11 @@ class Builds extends Action
->setAttribute('sourcePath', $source)
->setAttribute('sourceSize', $directorySize)
->setAttribute('totalSize', $directorySize);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'sourcePath' => $deployment->getAttribute('sourcePath'),
'sourceSize' => $deployment->getAttribute('sourceSize'),
'totalSize' => $deployment->getAttribute('totalSize'),
]));
$queueForRealtime
->setPayload($deployment->getArrayCopy())
@@ -543,7 +528,9 @@ class Builds extends Action
/** Request the executor to build the code... */
$deployment->setAttribute('status', 'building');
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'status' => 'building',
]));
if ($deployment->getSequence() === $resource->getAttribute('latestDeploymentInternalId', '')) {
$resource = $dbForProject->updateDocument($resource->getCollection(), $resource->getId(), new Document(['latestDeploymentStatus' => $deployment->getAttribute('status', '')]));
@@ -604,16 +591,15 @@ class Builds extends Action
}
$cpus = $spec['cpus'] ?? APP_COMPUTE_CPUS_DEFAULT;
$memory = max($spec['memory'] ?? APP_COMPUTE_MEMORY_DEFAULT, $minMemory);
$memory = max($spec['memory'] ?? APP_COMPUTE_MEMORY_DEFAULT, $minMemory);
$timeout = (int) System::getEnv('_APP_COMPUTE_BUILD_TIMEOUT', 900);
$jwtExpiry = (int)System::getEnv('_APP_COMPUTE_BUILD_TIMEOUT', 900);
$jwtExpiry = (int) System::getEnv('_APP_COMPUTE_BUILD_TIMEOUT', 900);
$jwtObj = new JWT(System::getEnv('_APP_OPENSSL_KEY_V1'), 'HS256', $jwtExpiry, 0);
$apiKey = $jwtObj->encode([
'projectId' => $project->getId(),
'scopes' => $resource->getAttribute('scopes', [])
'scopes' => $resource->getAttribute('scopes', []),
]);
// Appwrite vars
@@ -681,6 +667,7 @@ class Builds extends Action
if ($dbForProject->getDocument('deployments', $deploymentId)->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -702,7 +689,7 @@ class Builds extends Action
$listFilesCommand .= 'echo "{APPWRITE_DETECTION_SEPARATOR_START}" && cd /usr/local/build';
// Enter output directory, if set
if (!empty($outputDirectory)) {
if (! empty($outputDirectory)) {
$listFilesCommand .= ' && cd ' . \escapeshellarg($outputDirectory);
}
@@ -729,7 +716,7 @@ class Builds extends Action
cpus: $cpus,
memory: $memory,
timeout: $timeout,
remove: true,
remove: true,
entrypoint: $deployment->getAttribute('entrypoint', ''),
destination: APP_STORAGE_BUILDS . "/app-{$project->getId()}",
variables: $vars,
@@ -763,6 +750,7 @@ class Builds extends Action
if ($deployment->getAttribute('status') === 'canceled') {
$isCanceled = true;
Console::info('Ignoring realtime logs because build has been canceled');
return;
}
@@ -770,7 +758,7 @@ class Builds extends Action
$logs = \mb_substr($logs, 0, null, 'UTF-8');
// Do not stream logs added for SSR detection
if (!$insideSeparation) {
if (! $insideSeparation) {
$separator = \strpos($logs, '{APPWRITE_DETECTION_SEPARATOR_START}');
if ($separator !== false) {
$logs = \substr($logs, 0, $separator);
@@ -800,26 +788,28 @@ class Builds extends Action
$currentLogs = $deployment->getAttribute('buildLogs', '');
$affected = false;
$streamLogs = \str_replace("\\n", "{APPWRITE_LINEBREAK_PLACEHOLDER}", $logs);
$streamLogs = \str_replace('\\n', '{APPWRITE_LINEBREAK_PLACEHOLDER}', $logs);
foreach (\explode("\n", $streamLogs) as $streamLog) {
if (empty($streamLog)) {
continue;
}
$streamLog = \str_replace("{APPWRITE_LINEBREAK_PLACEHOLDER}", "\n", $streamLog);
$streamParts = \explode(" ", $streamLog, 2);
$streamLog = \str_replace('{APPWRITE_LINEBREAK_PLACEHOLDER}', "\n", $streamLog);
$streamParts = \explode(' ', $streamLog, 2);
// TODO: use part[0] as timestamp when switching to dbForLogs for build logs
$currentLogs .= $streamParts[1];
if (!empty($streamParts[1])) {
if (! empty($streamParts[1])) {
$affected = true;
}
}
if ($affected) {
$deployment = $deployment->setAttribute('buildLogs', $currentLogs);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildLogs' => $currentLogs,
]));
$queueForRealtime
->setPayload($deployment->getArrayCopy())
@@ -842,6 +832,7 @@ class Builds extends Action
if ($dbForProject->getDocument('deployments', $deploymentId)->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -849,7 +840,7 @@ class Builds extends Action
throw $err;
}
$buildSizeLimit = (int)System::getEnv('_APP_COMPUTE_BUILD_SIZE_LIMIT', '2000000000');
$buildSizeLimit = (int) System::getEnv('_APP_COMPUTE_BUILD_SIZE_LIMIT', '2000000000');
if (isset($plan['buildSize'])) {
$buildSizeLimit = $plan['buildSize'] * 1000 * 1000;
}
@@ -877,7 +868,7 @@ class Builds extends Action
$deployment->setAttribute('buildLogs', $logs);
$adapter = null;
if ($resource->getCollection() === 'sites' && !empty($detectionLogs)) {
if ($resource->getCollection() === 'sites' && ! empty($detectionLogs)) {
$files = \explode("\n", $detectionLogs); // Parse output
$files = \array_filter($files); // Remove empty
$files = \array_map(fn ($file) => \trim($file), $files); // Remove whitepsaces
@@ -905,7 +896,14 @@ class Builds extends Action
}
}
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildPath' => $deployment->getAttribute('buildPath'),
'buildSize' => $deployment->getAttribute('buildSize'),
'totalSize' => $deployment->getAttribute('totalSize'),
'buildLogs' => $deployment->getAttribute('buildLogs'),
'adapter' => $deployment->getAttribute('adapter'),
'fallbackFile' => $deployment->getAttribute('fallbackFile'),
]));
$queueForRealtime
->setPayload($deployment->getArrayCopy())
->trigger();
@@ -916,12 +914,15 @@ class Builds extends Action
$logs = $deployment->getAttribute('buildLogs', '');
$date = \date('H:i:s');
$logs .= "[$date] [appwrite] Deployment finished. \n";
$logs .= "\033[90m[$date] \033[90m[\033[0mappwrite\033[90m]\033[32m Deployment finished. \033[0m\n";
$deployment->setAttribute('buildLogs', $logs);
/** Update the status */
$deployment->setAttribute('status', 'ready');
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'buildLogs' => $deployment->getAttribute('buildLogs'),
'status' => 'ready',
]));
Console::log('Status marked as ready');
@@ -939,9 +940,9 @@ class Builds extends Action
// Check if current active deployment started later than this deployment
$resource = $dbForProject->getDocument($resource->getCollection(), $resource->getId());
$currentActiveDeploymentId = $resource->getAttribute('deploymentId', '');
if (!empty($currentActiveDeploymentId)) {
if (! empty($currentActiveDeploymentId)) {
$currentActiveDeployment = $dbForProject->getDocument('deployments', $currentActiveDeploymentId);
if (!$currentActiveDeployment->isEmpty()) {
if (! $currentActiveDeployment->isEmpty()) {
$currentActiveStartTime = $currentActiveDeployment->getCreatedAt();
$deploymentStartTime = $deployment->getCreatedAt();
@@ -1027,7 +1028,7 @@ class Builds extends Action
if ($resource->getCollection() === 'sites') {
// VCS branch
$branchName = $deployment->getAttribute('providerBranch');
if (!empty($branchName)) {
if (! empty($branchName)) {
$domain = (new BranchDomainFilter())->apply([
'branch' => $branchName,
'resourceId' => $resource->getId(),
@@ -1054,7 +1055,7 @@ class Builds extends Action
'certificateId' => '',
'search' => implode(' ', [$ruleId, $domain]),
'owner' => 'Appwrite',
'region' => $project->getAttribute('region')
'region' => $project->getAttribute('region'),
]));
} catch (Duplicate $err) {
$rule = $dbForPlatform->updateDocument('rules', $ruleId, new Document([
@@ -1095,6 +1096,7 @@ class Builds extends Action
if ($dbForProject->getDocument('deployments', $deploymentId)->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
@@ -1108,8 +1110,12 @@ class Builds extends Action
$schedule
->setAttribute('resourceUpdatedAt', DateTime::now())
->setAttribute('schedule', $resource->getAttribute('schedule'))
->setAttribute('active', !empty($resource->getAttribute('schedule')) && !empty($resource->getAttribute('deploymentId')));
$dbForPlatform->updateDocument('schedules', $schedule->getId(), $schedule);
->setAttribute('active', ! empty($resource->getAttribute('schedule')) && ! empty($resource->getAttribute('deploymentId')));
$dbForPlatform->updateDocument('schedules', $schedule->getId(), new Document([
'resourceUpdatedAt' => $schedule->getAttribute('resourceUpdatedAt'),
'schedule' => $schedule->getAttribute('schedule'),
'active' => $schedule->getAttribute('active'),
]));
}
/** Screenshot site */
@@ -1132,13 +1138,14 @@ class Builds extends Action
if ($dbForProject->getDocument('deployments', $deploymentId)->getAttribute('status') === 'canceled') {
$this->cancelDeployment($deployment->getId(), $dbForProject, $queueForRealtime);
return;
}
// Color message red
$message = $th->getMessage();
if (!\str_contains($message, '')) {
$message = "" . $message;
if (! \str_contains($message, '')) {
$message = '' . $message;
}
$message = \str_replace('{APPWRITE_DETECTION_SEPARATOR_START}', '', $message);
@@ -1146,9 +1153,9 @@ class Builds extends Action
// Combine with previous logs if deployment got past build process
$previousLogs = '';
if (!is_null($deployment->getAttribute('buildSize', null))) {
if (! is_null($deployment->getAttribute('buildSize', null))) {
$previousLogs = $deployment->getAttribute('buildLogs', '');
if (!empty($previousLogs)) {
if (! empty($previousLogs)) {
$message = $previousLogs . "\n" . $message;
}
}
@@ -1160,7 +1167,12 @@ class Builds extends Action
$deployment->setAttribute('status', 'failed');
$deployment->setAttribute('buildLogs', $message);
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deploymentId, new Document([
'buildEndedAt' => $deployment->getAttribute('buildEndedAt'),
'buildDuration' => $deployment->getAttribute('buildDuration'),
'status' => 'failed',
'buildLogs' => $message,
]));
if ($deployment->getSequence() === $resource->getAttribute('latestDeploymentInternalId', '')) {
$resource = $dbForProject->updateDocument($resource->getCollection(), $resource->getId(), new Document(['latestDeploymentStatus' => $deployment->getAttribute('status', '')]));
@@ -1179,102 +1191,102 @@ class Builds extends Action
->trigger();
$this->sendUsage(
resource:$resource,
resource: $resource,
deployment: $deployment,
project: $project,
queue: $queueForStatsUsage
usage: $usage,
publisherForUsage: $publisherForUsage
);
}
}
protected function sendUsage(Document $resource, Document $deployment, Document $project, StatsUsage $queue): void
protected function sendUsage(Document $resource, Document $deployment, Document $project, Context $usage, UsagePublisher $publisherForUsage): void
{
$spec = Config::getParam('specifications')[$resource->getAttribute('specification', APP_COMPUTE_SPECIFICATION_DEFAULT)];
$spec = Config::getParam('specifications')[$resource->getAttribute('buildSpecification', APP_COMPUTE_SPECIFICATION_DEFAULT)];
switch ($deployment->getAttribute('status')) {
case 'ready':
$queue
$usage
->addMetric(METRIC_BUILDS_SUCCESS, 1) // per project
->addMetric(METRIC_BUILDS_COMPUTE_SUCCESS, (int)$deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(METRIC_BUILDS_COMPUTE_SUCCESS, (int) $deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_SUCCESS), 1) // per function
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_COMPUTE_SUCCESS), (int)$deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_COMPUTE_SUCCESS), (int) $deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_SUCCESS), 1) // per function
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_COMPUTE_SUCCESS), (int)$deployment->getAttribute('buildDuration', 0) * 1000);
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_COMPUTE_SUCCESS), (int) $deployment->getAttribute('buildDuration', 0) * 1000);
break;
case 'failed':
$queue
$usage
->addMetric(METRIC_BUILDS_FAILED, 1) // per project
->addMetric(METRIC_BUILDS_COMPUTE_FAILED, (int)$deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(METRIC_BUILDS_COMPUTE_FAILED, (int) $deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_FAILED), 1) // per function
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_COMPUTE_FAILED), (int)$deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_COMPUTE_FAILED), (int) $deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_FAILED), 1) // per function
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_COMPUTE_FAILED), (int)$deployment->getAttribute('buildDuration', 0) * 1000);
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_COMPUTE_FAILED), (int) $deployment->getAttribute('buildDuration', 0) * 1000);
break;
}
$queue
$usage
->addMetric(METRIC_BUILDS, 1) // per project
->addMetric(METRIC_BUILDS_STORAGE, $deployment->getAttribute('buildSize', 0))
->addMetric(METRIC_BUILDS_COMPUTE, (int)$deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(METRIC_BUILDS_MB_SECONDS, (int)(($spec['memory'] ?? APP_COMPUTE_MEMORY_DEFAULT) * $deployment->getAttribute('buildDuration', 0) * ($spec['cpus'] ?? APP_COMPUTE_CPUS_DEFAULT)))
->addMetric(METRIC_BUILDS_COMPUTE, (int) $deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(METRIC_BUILDS_MB_SECONDS, (int) ($memory * $deployment->getAttribute('buildDuration', 0) * $cpus))
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS), 1) // per function
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_STORAGE), $deployment->getAttribute('buildSize', 0))
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_COMPUTE), (int)$deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_MB_SECONDS), (int)(($spec['memory'] ?? APP_COMPUTE_MEMORY_DEFAULT) * $deployment->getAttribute('buildDuration', 0) * ($spec['cpus'] ?? APP_COMPUTE_CPUS_DEFAULT)))
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_COMPUTE), (int) $deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}'], [$deployment->getAttribute('resourceType')], METRIC_RESOURCE_TYPE_BUILDS_MB_SECONDS), (int) ($memory * $deployment->getAttribute('buildDuration', 0) * $cpus))
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS), 1) // per function
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_STORAGE), $deployment->getAttribute('buildSize', 0))
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_COMPUTE), (int)$deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_MB_SECONDS), (int)(($spec['memory'] ?? APP_COMPUTE_MEMORY_DEFAULT) * $deployment->getAttribute('buildDuration', 0) * ($spec['cpus'] ?? APP_COMPUTE_CPUS_DEFAULT)))
->setProject($project)
->trigger();
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_COMPUTE), (int) $deployment->getAttribute('buildDuration', 0) * 1000)
->addMetric(str_replace(['{resourceType}', '{resourceInternalId}'], [$deployment->getAttribute('resourceType'), $resource->getSequence()], METRIC_RESOURCE_TYPE_ID_BUILDS_MB_SECONDS), (int) ($memory * $deployment->getAttribute('buildDuration', 0) * $cpus));
// Publish usage metrics
if (! $usage->isEmpty()) {
$message = new UsageMessage(
project: $project,
metrics: $usage->getMetrics(),
reduce: $usage->getReduce()
);
$publisherForUsage->enqueue($message);
$usage->reset();
}
}
/**
* Hook to run after build success
*
* @param Realtime $queueForRealtime
* @param Database $dbForProject
* @param Document $deployment
* @param array $runtime
* @param string|null $adapter
* @return void
* @throws Exception
*/
protected function afterBuildSuccess(Realtime $queueForRealtime, Database $dbForProject, Document &$deployment, array $runtime, ?string $adapter): void
{
if (!($queueForRealtime instanceof Realtime)) {
if (! ($queueForRealtime instanceof Realtime)) {
throw new Exception('queueForRealtime must be an instance of Realtime');
}
if (!($dbForProject instanceof Database)) {
if (! ($dbForProject instanceof Database)) {
throw new Exception('dbForProject must be an instance of Database');
}
if (!($deployment instanceof Document)) {
if (! ($deployment instanceof Document)) {
throw new Exception('deployment must be an instance of Document');
}
if (!is_array($runtime)) {
if (! is_array($runtime)) {
throw new Exception('runtime must be an array');
}
if (!is_string($adapter) && !is_null($adapter)) {
if (! is_string($adapter) && ! is_null($adapter)) {
throw new Exception('adapter must be a string or null');
}
}
/**
* Hook to run after deployment is activated
*
* @param Document $project
* @param Document $deployment
* @return void
*/
protected function afterDeploymentSuccess(
Document $project,
Document $deployment,
): void {
if (!($project instanceof Document)) {
if (! ($project instanceof Document)) {
throw new Exception('project must be an instance of Document');
}
if (!($deployment instanceof Document)) {
if (! ($deployment instanceof Document)) {
throw new Exception('deployment must be an instance of Document');
}
}
@@ -1282,7 +1294,7 @@ class Builds extends Action
protected function getRuntime(Document $resource, string $version): array
{
$runtimes = Config::getParam($version === 'v2' ? 'runtimes-v2' : 'runtimes', []);
$key = $resource->getAttribute('runtime');
$key = $resource->getAttribute('runtime');
$runtime = match ($resource->getCollection()) {
'functions' => $runtimes[$resource->getAttribute('runtime')] ?? null,
'sites' => $runtimes[$resource->getAttribute('buildRuntime')] ?? null,
@@ -1315,7 +1327,7 @@ class Builds extends Action
$envCommand = '';
$bundleCommand = '';
if (!is_null($framework)) {
if (! is_null($framework)) {
$envCommand = $framework['envCommand'] ?? '';
$bundleCommand = $framework['bundleCommand'] ?? '';
}
@@ -1324,7 +1336,7 @@ class Builds extends Action
$commands[] = $deployment->getAttribute('buildCommands', '');
$commands[] = $bundleCommand;
$commands = array_filter($commands, fn ($command) => !empty($command));
$commands = array_filter($commands, fn ($command) => ! empty($command));
return implode(' && ', $commands);
}
@@ -1333,19 +1345,6 @@ class Builds extends Action
}
/**
* @param string $status
* @param GitHub $github
* @param string $providerCommitHash
* @param string $owner
* @param string $repositoryName
* @param Document $project
* @param Document $resource
* @param string $deploymentId
* @param Database $dbForProject
* @param Database $dbForPlatform
* @param Realtime $queueForRealtime
* @param array $platform
* @return void
* @throws Structure
* @throws \Utopia\Database\Exception
* @throws Conflict
@@ -1373,7 +1372,7 @@ class Builds extends Action
$deployment = $dbForProject->getDocument('deployments', $deploymentId);
$commentId = $deployment->getAttribute('providerCommentId', '');
if (!empty($providerCommitHash)) {
if (! empty($providerCommitHash)) {
$message = match ($status) {
'ready' => 'Build succeeded.',
'failed' => 'Build failed.',
@@ -1408,7 +1407,7 @@ class Builds extends Action
$github->updateCommitStatus($repositoryName, $providerCommitHash, $owner, $state, $message, $providerTargetUrl, $name);
}
if (!empty($commentId)) {
if (! empty($commentId)) {
$retries = 0;
while (true) {
@@ -1416,7 +1415,7 @@ class Builds extends Action
try {
$dbForPlatform->createDocument('vcsCommentLocks', new Document([
'$id' => $commentId
'$id' => $commentId,
]));
break;
} catch (\Throwable $err) {
@@ -1430,22 +1429,22 @@ class Builds extends Action
// Wrap in try/finally to ensure lock file gets deleted
try {
$resourceType = match($resource->getCollection()) {
$resourceType = match ($resource->getCollection()) {
'functions' => 'function',
'sites' => 'site',
default => throw new \Exception('Invalid resource type')
};
$rule = $dbForPlatform->findOne('rules', [
Query::equal("projectInternalId", [$project->getSequence()]),
Query::equal("type", ["deployment"]),
Query::equal("deploymentInternalId", [$deployment->getSequence()]),
Query::equal('projectInternalId', [$project->getSequence()]),
Query::equal('type', ['deployment']),
Query::equal('deploymentInternalId', [$deployment->getSequence()]),
]);
$protocol = System::getEnv('_APP_OPTIONS_FORCE_HTTPS') == 'disabled' ? 'http' : 'https';
$previewUrl = match($resource->getCollection()) {
$previewUrl = match ($resource->getCollection()) {
'functions' => '',
'sites' => !empty($rule) ? ("{$protocol}://" . $rule->getAttribute('domain', '')) : '',
'sites' => ! empty($rule) ? ("{$protocol}://" . $rule->getAttribute('domain', '')) : '',
default => throw new \Exception('Invalid resource type')
};
@@ -1458,7 +1457,7 @@ class Builds extends Action
}
}
} catch (\Throwable $th) {
Console::warning("Git action failed:");
Console::warning('Git action failed:');
Console::warning($th->getMessage());
Console::warning($th->getTraceAsString());
@@ -1467,7 +1466,9 @@ class Builds extends Action
$logs .= "[$date] [appwrite] Git action failed. Deployment will continue. \n";
$deployment->setAttribute('buildLogs', $logs);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildLogs' => $deployment->getAttribute('buildLogs'),
]));
$queueForRealtime
->setPayload($deployment->getArrayCopy())
@@ -1483,10 +1484,12 @@ class Builds extends Action
$logs = $deployment->getAttribute('buildLogs', '');
$date = \date('H:i:s');
$logs .= "[$date] [appwrite] Build has been canceled. \n";
$logs .= "\033[90m[$date] \033[90m[\033[0mappwrite\033[90m]\033[33m Build has been canceled. \033[0m\n";
$deployment->setAttribute('buildLogs', $logs);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), $deployment);
$deployment = $dbForProject->updateDocument('deployments', $deployment->getId(), new Document([
'buildLogs' => $deployment->getAttribute('buildLogs'),
]));
$queueForRealtime
->setPayload($deployment->getArrayCopy())
@@ -12,9 +12,9 @@ use Appwrite\Event\Func;
use Appwrite\Event\Mail;
use Appwrite\Event\Messaging;
use Appwrite\Event\Migration;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Event\Screenshot;
use Appwrite\Event\StatsResources;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Webhook;
use Appwrite\Platform\Modules\Health\Http\Health\Queue\Base;
use Appwrite\SDK\AuthType;
@@ -79,7 +79,7 @@ class Get extends Base
->inject('queueForMails')
->inject('queueForFunctions')
->inject('queueForStatsResources')
->inject('queueForStatsUsage')
->inject('publisherForUsage')
->inject('queueForWebhooks')
->inject('queueForCertificates')
->inject('queueForBuilds')
@@ -99,7 +99,7 @@ class Get extends Base
Mail $queueForMails,
Func $queueForFunctions,
StatsResources $queueForStatsResources,
StatsUsage $queueForStatsUsage,
UsagePublisher $publisherForUsage,
Webhook $queueForWebhooks,
Certificate $queueForCertificates,
Build $queueForBuilds,
@@ -116,7 +116,7 @@ class Get extends Base
System::getEnv('_APP_MAILS_QUEUE_NAME', Event::MAILS_QUEUE_NAME) => $queueForMails,
System::getEnv('_APP_FUNCTIONS_QUEUE_NAME', Event::FUNCTIONS_QUEUE_NAME) => $queueForFunctions,
System::getEnv('_APP_STATS_RESOURCES_QUEUE_NAME', Event::STATS_RESOURCES_QUEUE_NAME) => $queueForStatsResources,
System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME) => $queueForStatsUsage,
System::getEnv('_APP_STATS_USAGE_QUEUE_NAME', Event::STATS_USAGE_QUEUE_NAME) => $publisherForUsage,
System::getEnv('_APP_WEBHOOK_QUEUE_NAME', Event::WEBHOOK_QUEUE_NAME) => $queueForWebhooks,
System::getEnv('_APP_CERTIFICATES_QUEUE_NAME', Event::CERTIFICATES_QUEUE_NAME) => $queueForCertificates,
System::getEnv('_APP_BUILDS_QUEUE_NAME', Event::BUILDS_QUEUE_NAME) => $queueForBuilds,
@@ -2,7 +2,7 @@
namespace Appwrite\Platform\Modules\Health\Http\Health\Queue\StatsUsage;
use Appwrite\Event\StatsUsage;
use Appwrite\Event\Publisher\Usage as UsagePublisher;
use Appwrite\Platform\Modules\Health\Http\Health\Queue\Base;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
@@ -42,16 +42,16 @@ class Get extends Base
contentType: ContentType::JSON
))
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queueForStatsUsage')
->inject('publisherForUsage')
->inject('response')
->callback($this->action(...));
}
public function action(int|string $threshold, StatsUsage $queueForStatsUsage, Response $response): void
public function action(int|string $threshold, UsagePublisher $publisherForUsage, Response $response): void
{
$threshold = (int) $threshold;
$size = $queueForStatsUsage->getSize();
$size = $publisherForUsage->getSize();
$this->assertQueueThreshold($size, $threshold);
@@ -2,7 +2,6 @@
namespace Appwrite\Platform\Modules\Health\Http\Health\Storage;
use Appwrite\Extend\Exception;
use Appwrite\SDK\AuthType;
use Appwrite\SDK\ContentType;
use Appwrite\SDK\Method;
@@ -58,33 +57,21 @@ class Get extends Action
$checkStart = \microtime(true);
foreach ($devices as $device) {
$uniqueFileName = \uniqid('health', true);
$filePath = $device->getPath($uniqueFileName);
$path = $device->getPath(\uniqid('health', true));
if (!$device->write($filePath, 'test', 'text/plain')) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed writing test file to ' . $device->getRoot());
}
$readError = null;
try {
if ($device->read($filePath) !== 'test') {
$readError = new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed reading test file from ' . $device->getRoot());
if (!$device->write($path, 'test', 'text/plain')) {
throw new \Exception("Failed writing test file to {$device->getRoot()}");
}
$content = $device->read($path);
if ($content !== 'test') {
throw new \Exception("Failed reading test file from {$device->getRoot()}: content mismatch");
}
} catch (\Throwable $e) {
$readError = $e;
} finally {
// Always attempt to clean up test file
if (!$device->delete($filePath)) {
if ($readError !== null) {
// If read already failed, wrap delete error but preserve original
\error_log('Failed deleting test file from ' . $device->getRoot() . ' during read error recovery');
} else {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed deleting test file from ' . $device->getRoot());
}
}
// Re-throw read error if it occurred
if ($readError !== null) {
throw $readError;
try {
$device->delete($path);
} catch (\Throwable) {
}
}
}
@@ -9,6 +9,7 @@ use Appwrite\SDK\Method;
use Appwrite\SDK\Response as SDKResponse;
use Appwrite\Utopia\Response;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Validator\Datetime as DatetimeValidator;
use Utopia\Database\Validator\UID;
use Utopia\Platform\Action;
@@ -73,7 +74,7 @@ class Update extends Action
->setAttribute('name', $name)
->setAttribute('expire', $expire);
$dbForPlatform->updateDocument('devKeys', $key->getId(), $key);
$dbForPlatform->updateDocument('devKeys', $key->getId(), new Document(['name' => $name, 'expire' => $expire]));
$dbForPlatform->purgeCachedDocument('projects', $project->getId());

Some files were not shown because too many files have changed in this diff Show More