Appwrite/appwrite
1
0
Fork 0
mirror of https://github.com/appwrite/appwrite.git synced 2026-05-26 13:51:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch '1.4.x' of github.com:appwrite/appwrite into refactor-usage-sn

Browse Source 
 Conflicts:
	.gitmodules
	CHANGES.md
	Dockerfile
	app/controllers/api/storage.php
	app/init.php
	composer.lock
	src/Appwrite/Migration/Migration.php
	src/Appwrite/Platform/Services/Tasks.php
	src/Appwrite/Usage/Calculators/TimeSeries.php
This commit is contained in:
shimon
2023-11-26 10:28:17 +02:00
parent 91fed7a255 540ff6ee2e
commit 21895e3ac0
50 changed files with 1317 additions and 799 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/linter.yml
+4
View File
@@ -18,6 +18,10 @@ jobs:
- run: git checkout HEAD^2
- name: Validate composer.json and composer.lock
run: |
docker run --rm -v $PWD:/app composer sh -c \
"composer validate"
- name: Run Linter
run: |
docker run --rm -v $PWD:/app composer sh -c \
CHANGES.md
+54 -1
View File
@@ -1,4 +1,4 @@
# Version 1.4.10
# Version 1.4.14
## Changes
- New usage metrics collection flow [#7005](https://github.com/appwrite/appwrite/pull/7005)
@@ -36,6 +36,59 @@
- renamed
- executions to executionsTotal, builds to buildsTotal, requests to requestsTotal, storage to filesStorage, buckets to bucketsTotal, users to usersTotal, documents to documentsTotal, collections to collectionsTotal, databases to databasesTotal.
# Version 1.4.13
## Notable changes
* Change enum size validation in update controller [#7164](https://github.com/appwrite/appwrite/pull/7164)
* Bump console to version 3.2.8 in [#7167](https://github.com/appwrite/appwrite/pull/7167)
## Bug fixes
* Fix error after adding bigger enum [#7162](https://github.com/appwrite/appwrite/pull/7162)
* Add chunkId to abuse key to prevent rate limit for SDKs [#7154](https://github.com/appwrite/appwrite/pull/7154)
## Miscellaneous
* Fix enum test case [#7163](https://github.com/appwrite/appwrite/pull/7163)
* Add flag to send logs to logger [#7155](https://github.com/appwrite/appwrite/pull/7155)
* Add a CI task to validate composer file and lock [#7142](https://github.com/appwrite/appwrite/pull/7142)
# Version 1.4.12
## Miscellaneous
* Bump console to version 3.2.7 [#7148](https://github.com/appwrite/appwrite/pull/7148)
* Chore update database to 0.45.2 [#7138](https://github.com/appwrite/appwrite/pull/7138)
* Implement queue thresholds for the health API [#7123](https://github.com/appwrite/appwrite/pull/7123)
* Add Authorization::skip to the usage worker [#7124](https://github.com/appwrite/appwrite/pull/7124)
## Bug fixes
* fix: use queueForDeletes in git installation delete endpoint [#7140](https://github.com/appwrite/appwrite/pull/7140)
* fix: patch script, make errors silent [#7134](https://github.com/appwrite/appwrite/pull/7134)
* fix: repositories recreation script [#7133](https://github.com/appwrite/appwrite/pull/7133)
* fix: Only delete repositories linked to the particular project [#7131](https://github.com/appwrite/appwrite/pull/7131)
# Version 1.4.11
## Miscellaneous
* Update database by @abnegate in [#7111](https://github.com/appwrite/appwrite/pull/7111)
# Version 1.4.10
## Bug fixes
* Handle cases where password history could contain NULLs [#7092](https://github.com/appwrite/appwrite/pull/7092)
* Missing functionId error on create execution [#7091](https://github.com/appwrite/appwrite/pull/7091)
* Ensure usage endpoints don't throw 500 when usage is disabled [#7087](https://github.com/appwrite/appwrite/pull/7087)
* Missing sessionId error when deleting all user sessions [#7085](https://github.com/appwrite/appwrite/pull/7085)
* Domain validation in Create Proxy rule results in 500 error [#7084](https://github.com/appwrite/appwrite/pull/7084)
* Fix optional services [#7078](https://github.com/appwrite/appwrite/pull/7078)
* Fix regression from worker refactor [#7074](https://github.com/appwrite/appwrite/pull/7074)
* Use getQueueSize() in the Health service's get X queue endpoints [#7073](https://github.com/appwrite/appwrite/pull/7073)
* Delete linked VCS repos and comments [#7066](https://github.com/appwrite/appwrite/pull/7066)
# Version 1.4.9
## Bug fixes
Dockerfile
+6 -1
View File
@@ -99,9 +99,14 @@ RUN chmod +x /usr/local/bin/doctor && \
# Cloud Executabless
RUN chmod +x /usr/local/bin/hamster && \
chmod +x /usr/local/bin/volume-sync && \
chmod +x /usr/local/bin/patch-delete-schedule-updated-at-attribute && \
chmod +x /usr/local/bin/patch-recreate-repositories-documents && \
chmod +x /usr/local/bin/patch-delete-project-collections && \
chmod +x /usr/local/bin/delete-orphaned-projects && \
chmod +x /usr/local/bin/clear-card-cache && \
chmod +x /usr/local/bin/calc-users-stats && \
chmod +x /usr/local/bin/calc-tier-stats && \
chmod +x /usr/local/bin/create-inf-metric
chmod +x /usr/local/bin/get-migration-stats
# Letsencrypt Permissions
RUN mkdir -p /etc/letsencrypt/live/ && chmod -Rf 755 /etc/letsencrypt/live/
README-CN.md
+4 -4
View File
@@ -55,7 +55,7 @@ Appwrite 可以提供给开发者用户验证，外部授权，用户数据读
## 安装
Appwrite 的容器化服务器只需要一行指令就可以运行。您可以使用 docker-compose 在本地主机上运行 Appwrite，也可以在任何其他容器化工具（如 Kubernetes、Docker Swarm 或 Rancher）上运行 Appwrite。
Appwrite 的容器化服务器只需要一行指令就可以运行。您可以使用 docker-compose 在本地主机上运行 Appwrite，也可以在任何其他容器化工具（如 [Kubernetes](https://kubernetes.io/docs/home/)、[Docker Swarm](https://docs.docker.com/engine/swarm/) 或 [Rancher](https://rancher.com/docs/)）上运行 Appwrite。
启动 Appwrite 服务器的最简单方法是运行我们的 docker-compose 文件。在运行安装命令之前，请确保您的机器上安装了 [Docker](https://dockerdocs.cn/get-docker/index.html)：
@@ -66,7 +66,7 @@ docker run -it --rm \
--volume /var/run/docker.sock:/var/run/docker.sock \
--volume "$(pwd)"/appwrite:/usr/src/code/appwrite:rw \
--entrypoint="install" \
appwrite/appwrite:1.4.9
appwrite/appwrite:1.4.13
```
### Windows
@@ -78,7 +78,7 @@ docker run -it --rm ^
--volume //var/run/docker.sock:/var/run/docker.sock ^
--volume "%cd%"/appwrite:/usr/src/code/appwrite:rw ^
--entrypoint="install" ^
appwrite/appwrite:1.4.9
appwrite/appwrite:1.4.13
```
#### PowerShell
@@ -88,7 +88,7 @@ docker run -it --rm `
--volume /var/run/docker.sock:/var/run/docker.sock `
--volume ${pwd}/appwrite:/usr/src/code/appwrite:rw `
--entrypoint="install" `
appwrite/appwrite:1.4.9
appwrite/appwrite:1.4.13
```
运行后，可以在浏览器上访问 http://localhost 找到 Appwrite 控制台。在非 Linux 的本机主机上完成安装后，服务器可能需要几分钟才能启动。
README.md
+3 -3
View File
@@ -76,7 +76,7 @@ docker run -it --rm \
--volume /var/run/docker.sock:/var/run/docker.sock \
--volume "$(pwd)"/appwrite:/usr/src/code/appwrite:rw \
--entrypoint="install" \
appwrite/appwrite:1.4.9
appwrite/appwrite:1.4.13
```
### Windows
@@ -88,7 +88,7 @@ docker run -it --rm ^
--volume //var/run/docker.sock:/var/run/docker.sock ^
--volume "%cd%"/appwrite:/usr/src/code/appwrite:rw ^
--entrypoint="install" ^
appwrite/appwrite:1.4.9
appwrite/appwrite:1.4.13
```
#### PowerShell
@@ -98,7 +98,7 @@ docker run -it --rm `
--volume /var/run/docker.sock:/var/run/docker.sock `
--volume ${pwd}/appwrite:/usr/src/code/appwrite:rw `
--entrypoint="install" `
appwrite/appwrite:1.4.9
appwrite/appwrite:1.4.13
```
Once the Docker installation is complete, go to http://localhost to access the Appwrite console from your browser. Please note that on non-Linux native hosts, the server might take a few minutes to start after completing the installation.
app/config/errors.php
+24
View File
@@ -209,6 +209,7 @@ return [
'name' => Exception::USER_AUTH_METHOD_UNSUPPORTED,
'description' => 'The requested authentication method is either disabled or unsupported. Please check the supported authentication methods in the Appwrite console.',
'code' => 501,
'publish' => false,
],
Exception::USER_PHONE_ALREADY_EXISTS => [
'name' => Exception::USER_PHONE_ALREADY_EXISTS,
@@ -240,6 +241,16 @@ return [
'description' => 'OAuth2 provider returned some error.',
'code' => 424,
],
Exception::USER_EMAIL_ALREADY_VERIFIED => [
'name' => Exception::USER_EMAIL_ALREADY_VERIFIED,
'description' => 'User email is already verified',
'code' => 409,
],
Exception::USER_PHONE_ALREADY_VERIFIED => [
'name' => Exception::USER_PHONE_ALREADY_VERIFIED,
'description' => 'User phone is already verified',
'code' => 409
],
/** Teams */
Exception::TEAM_NOT_FOUND => [
@@ -749,4 +760,17 @@ return [
'description' => 'Too many messages.',
'code' => 1013,
],
Exception::MIGRATION_PROVIDER_ERROR => [
'name' => Exception::MIGRATION_PROVIDER_ERROR,
'description' => 'An error occurred on the provider\'s side. Please try again later.',
'code' => 400,
],
/** Health */
Exception::QUEUE_SIZE_EXCEEDED => [
'name' => Exception::QUEUE_SIZE_EXCEEDED,
'description' => 'Queue size threshold hit.',
'code' => 503,
'publish' => false
],
];
app/config/services.php
+3 -3
View File
@@ -170,7 +170,7 @@ return [
'docs' => false,
'docsUrl' => '',
'tests' => false,
'optional' => true,
'optional' => false,
'icon' => '',
],
'functions' => [
@@ -196,7 +196,7 @@ return [
'docs' => true,
'docsUrl' => 'https://appwrite.io/docs/proxy',
'tests' => false,
'optional' => true,
'optional' => false,
'icon' => '/images/services/proxy.png',
],
'mock' => [
@@ -248,7 +248,7 @@ return [
'docs' => true,
'docsUrl' => 'https://appwrite.io/docs/migrations',
'tests' => true,
'optional' => true,
'optional' => false,
'icon' => '/images/services/migrations.png',
],
];
app/config/specs/open-api3-latest-client.json
+1 -1
View File
File diff suppressed because one or more lines are too long
app/config/specs/open-api3-latest-console.json
+1 -1
View File
File diff suppressed because one or more lines are too long
app/config/specs/open-api3-latest-server.json
+1 -1
View File
File diff suppressed because one or more lines are too long
app/config/specs/swagger2-latest-client.json
+1 -1
View File
File diff suppressed because one or more lines are too long
app/config/specs/swagger2-latest-console.json
+1 -1
View File
File diff suppressed because one or more lines are too long
app/config/specs/swagger2-latest-server.json
+1 -1
View File
File diff suppressed because one or more lines are too long
app/console
+1 -1
Submodule app/console updated: 9810ce8581...212b742992
app/controllers/api/account.php
+8
View File
@@ -2634,6 +2634,10 @@ App::post('/v1/account/verification')
throw new Exception(Exception::GENERAL_SMTP_DISABLED, 'SMTP Disabled');
}
if ($user->getAttribute('emailVerification')) {
throw new Exception(Exception::USER_EMAIL_ALREADY_VERIFIED);
}
$roles = Authorization::getRoles();
$isPrivilegedUser = Auth::isPrivilegedUser($roles);
$isAppUser = Auth::isAppUser($roles);
@@ -2853,6 +2857,10 @@ App::post('/v1/account/verification/phone')
throw new Exception(Exception::USER_PHONE_NOT_FOUND);
}
if ($user->getAttribute('phoneVerification')) {
throw new Exception(Exception::USER_PHONE_ALREADY_VERIFIED);
}
$roles = Authorization::getRoles();
$isPrivilegedUser = Auth::isPrivilegedUser($roles);
$isAppUser = Auth::isAppUser($roles);
app/controllers/api/databases.php
+6 -16
View File
@@ -1208,7 +1208,7 @@ App::post('/v1/databases/:databaseId/collections/:collectionId/attributes/enum')
->param('databaseId', '', new UID(), 'Database ID.')
->param('collectionId', '', new UID(), 'Collection ID. You can create a new collection using the Database service [server integration](https://appwrite.io/docs/server/databases#databasesCreateCollection).')
->param('key', '', new Key(), 'Attribute Key.')
->param('elements', [], new ArrayList(new Text(APP_LIMIT_ARRAY_ELEMENT_SIZE, min: 0), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Array of elements in enumerated type. Uses length of longest element to determine size. Maximum of ' . APP_LIMIT_ARRAY_PARAMS_SIZE . ' elements are allowed, each ' . APP_LIMIT_ARRAY_ELEMENT_SIZE . ' characters long.')
->param('elements', [], new ArrayList(new Text(DATABASE::LENGTH_KEY), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Array of elements in enumerated type. Uses length of longest element to determine size. Maximum of ' . APP_LIMIT_ARRAY_PARAMS_SIZE . ' elements are allowed, each ' . DATABASE::LENGTH_KEY . ' characters long.')
->param('required', null, new Boolean(), 'Is attribute required?')
->param('default', null, new Text(0), 'Default value for attribute when not provided. Cannot be set when attribute is required.', true)
->param('array', false, new Boolean(), 'Is attribute an array?', true)
@@ -1217,16 +1217,6 @@ App::post('/v1/databases/:databaseId/collections/:collectionId/attributes/enum')
->inject('queueForDatabase')
->inject('queueForEvents')
->action(function (string $databaseId, string $collectionId, string $key, array $elements, ?bool $required, ?string $default, bool $array, Response $response, Database $dbForProject, EventDatabase $queueForDatabase, Event $queueForEvents) {
// use length of longest string as attribute size
$size = 0;
foreach ($elements as $element) {
$length = \strlen($element);
if ($length === 0) {
throw new Exception(Exception::ATTRIBUTE_VALUE_INVALID, 'Each enum element must not be empty');
}
$size = ($length > $size) ? $length : $size;
}
if (!is_null($default) && !in_array($default, $elements)) {
throw new Exception(Exception::ATTRIBUTE_VALUE_INVALID, 'Default value not found in elements');
}
@@ -1234,7 +1224,7 @@ App::post('/v1/databases/:databaseId/collections/:collectionId/attributes/enum')
$attribute = createAttribute($databaseId, $collectionId, new Document([
'key' => $key,
'type' => Database::VAR_STRING,
'size' => $size,
'size' => Database::LENGTH_KEY,
'required' => $required,
'default' => $default,
'array' => $array,
@@ -1883,7 +1873,7 @@ App::patch('/v1/databases/:databaseId/collections/:collectionId/attributes/enum/
->param('databaseId', '', new UID(), 'Database ID.')
->param('collectionId', '', new UID(), 'Collection ID. You can create a new collection using the Database service [server integration](https://appwrite.io/docs/server/databases#databasesCreateCollection).')
->param('key', '', new Key(), 'Attribute Key.')
->param('elements', null, new ArrayList(new Text(APP_LIMIT_ARRAY_ELEMENT_SIZE), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Array of elements in enumerated type. Uses length of longest element to determine size. Maximum of ' . APP_LIMIT_ARRAY_PARAMS_SIZE . ' elements are allowed, each ' . APP_LIMIT_ARRAY_ELEMENT_SIZE . ' characters long.')
->param('elements', null, new ArrayList(new Text(DATABASE::LENGTH_KEY), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Array of elements in enumerated type. Uses length of longest element to determine size. Maximum of ' . APP_LIMIT_ARRAY_PARAMS_SIZE . ' elements are allowed, each ' . DATABASE::LENGTH_KEY . ' characters long.')
->param('required', null, new Boolean(), 'Is attribute required?')
->param('default', null, new Nullable(new Text(0)), 'Default value for attribute when not provided. Cannot be set when attribute is required.')
->inject('response')
@@ -3549,7 +3539,7 @@ App::delete('/v1/databases/:databaseId/collections/:collectionId/documents/:docu
App::get('/v1/databases/usage')
->desc('Get usage stats for the database')
->groups(['api', 'database'])
->groups(['api', 'database', 'usage'])
->label('scope', 'collections.read')
->label('sdk.auth', [APP_AUTH_TYPE_ADMIN])
->label('sdk.namespace', 'databases')
@@ -3627,7 +3617,7 @@ App::get('/v1/databases/usage')
App::get('/v1/databases/:databaseId/usage')
->desc('Get usage stats for the database')
->groups(['api', 'database'])
->groups(['api', 'database', 'usage'])
->label('scope', 'collections.read')
->label('sdk.auth', [APP_AUTH_TYPE_ADMIN])
->label('sdk.namespace', 'databases')
@@ -3711,7 +3701,7 @@ App::get('/v1/databases/:databaseId/usage')
App::get('/v1/databases/:databaseId/collections/:collectionId/usage')
->alias('/v1/database/:collectionId/usage', ['databaseId' => 'default'])
->desc('Get usage stats for a collection')
->groups(['api', 'database'])
->groups(['api', 'database', 'usage'])
->label('scope', 'collections.read')
->label('sdk.auth', [APP_AUTH_TYPE_ADMIN])
->label('sdk.namespace', 'databases')
app/controllers/api/functions.php
+9 -4
View File
@@ -242,12 +242,16 @@ App::post('/v1/functions')
// Git connect logic
if (!empty($providerRepositoryId)) {
$teamId = $project->getAttribute('teamId', '');
$repository = $dbForConsole->createDocument('repositories', new Document([
'$id' => ID::unique(),
'$permissions' => [
Permission::read(Role::any()),
Permission::update(Role::any()),
Permission::delete(Role::any()),
Permission::read(Role::team(ID::custom($teamId))),
Permission::update(Role::team(ID::custom($teamId), 'owner')),
Permission::update(Role::team(ID::custom($teamId), 'developer')),
Permission::delete(Role::team(ID::custom($teamId), 'owner')),
Permission::delete(Role::team(ID::custom($teamId), 'developer')),
],
'installationId' => $installation->getId(),
'installationInternalId' => $installation->getInternalId(),
@@ -883,7 +887,6 @@ App::get('/v1/functions/:functionId/deployments/:deploymentId/download')
}
if ($size > APP_STORAGE_READ_BUFFER) {
$response->addHeader('Content-Length', $deviceFunctions->getFileSize($path));
for ($i = 0; $i < ceil($size / MAX_OUTPUT_CHUNK_SIZE); $i++) {
$response->chunk(
$deviceFunctions->read(
@@ -1637,6 +1640,8 @@ App::post('/v1/functions/:functionId/executions')
->setJWT($jwt)
->setProject($project)
->setUser($user)
->setParam('functionId', $function->getId())
->setParam('executionId', $execution->getId())
->trigger();
return $response
app/controllers/api/health.php
+122 -35
View File
@@ -14,6 +14,7 @@ use Utopia\Registry\Registry;
use Utopia\Storage\Device;
use Utopia\Storage\Device\Local;
use Utopia\Storage\Storage;
use Utopia\Validator\Integer;
use Utopia\Validator\Text;
App::get('/v1/health')
@@ -87,22 +88,18 @@ App::get('/v1/health/db')
'ping' => \round((\microtime(true) - $checkStart) / 1000)
]);
} else {
$output[] = new Document([
'name' => $key . " ($database)",
'status' => 'fail',
'ping' => \round((\microtime(true) - $checkStart) / 1000)
]);
$failure[] = $database;
}
} catch (\Throwable $th) {
$output[] = new Document([
'name' => $key . " ($database)",
'status' => 'fail',
'ping' => \round((\microtime(true) - $checkStart) / 1000)
]);
$failure[] = $database;
}
}
}
if (!empty($failure)) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'DB failure on: ' . implode(", ", $failure));
}
$response->dynamic(new Document([
'statuses' => $output,
'total' => count($output),
@@ -348,11 +345,20 @@ App::get('/v1/health/queue/webhooks')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (Connection $queue, Response $response) {
->action(function (int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client(Event::WEBHOOK_QUEUE_NAME, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/queue/logs')
@@ -366,11 +372,20 @@ App::get('/v1/health/queue/logs')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (Connection $queue, Response $response) {
->action(function (int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client(Event::AUDITS_QUEUE_NAME, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/queue/certificates')
@@ -384,11 +399,20 @@ App::get('/v1/health/queue/certificates')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (Connection $queue, Response $response) {
->action(function (int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client(Event::CERTIFICATES_QUEUE_NAME, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/queue/builds')
@@ -402,11 +426,20 @@ App::get('/v1/health/queue/builds')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (Connection $queue, Response $response) {
->action(function (int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client(Event::BUILDS_QUEUE_NAME, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/queue/databases')
@@ -421,11 +454,20 @@ App::get('/v1/health/queue/databases')
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('name', 'database_db_main', new Text(256), 'Queue name for which to check the queue size', true)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (string $name, Connection $queue, Response $response) {
->action(function (string $name, int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client($name, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/queue/deletes')
@@ -439,11 +481,20 @@ App::get('/v1/health/queue/deletes')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (Connection $queue, Response $response) {
->action(function (int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client(Event::DELETE_QUEUE_NAME, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/queue/mails')
@@ -457,11 +508,20 @@ App::get('/v1/health/queue/mails')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (Connection $queue, Response $response) {
->action(function (int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client(Event::MAILS_QUEUE_NAME, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/queue/messaging')
@@ -475,11 +535,20 @@ App::get('/v1/health/queue/messaging')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (Connection $queue, Response $response) {
->action(function (int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client(Event::MESSAGING_QUEUE_NAME, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/queue/migrations')
@@ -493,11 +562,20 @@ App::get('/v1/health/queue/migrations')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (Connection $queue, Response $response) {
->action(function (int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client(Event::MIGRATIONS_QUEUE_NAME, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/queue/functions')
@@ -511,11 +589,20 @@ App::get('/v1/health/queue/functions')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_HEALTH_QUEUE)
->param('threshold', 5000, new Integer(true), 'Queue size threshold. When hit (equal or higher), endpoint returns server error. Default value is 5000.', true)
->inject('queue')
->inject('response')
->action(function (Connection $queue, Response $response) {
->action(function (int|string $threshold, Connection $queue, Response $response) {
$threshold = \intval($threshold);
$client = new Client(Event::FUNCTIONS_QUEUE_NAME, $queue);
$response->dynamic(new Document([ 'size' => $client->getQueueSize() ]), Response::MODEL_HEALTH_QUEUE);
$size = $client->getQueueSize();
if ($size >= $threshold) {
throw new Exception(Exception::QUEUE_SIZE_EXCEEDED, "Queue size threshold hit. Current size is {$size} and threshold is {$threshold}.");
}
$response->dynamic(new Document([ 'size' => $size ]), Response::MODEL_HEALTH_QUEUE);
}, ['response']);
App::get('/v1/health/storage/local')
@@ -536,10 +623,10 @@ App::get('/v1/health/storage/local')
foreach (
[
'Uploads' => APP_STORAGE_UPLOADS,
'Cache' => APP_STORAGE_CACHE,
'Config' => APP_STORAGE_CONFIG,
'Certs' => APP_STORAGE_CERTIFICATES
'Uploads' => APP_STORAGE_UPLOADS,
'Cache' => APP_STORAGE_CACHE,
'Config' => APP_STORAGE_CONFIG,
'Certs' => APP_STORAGE_CERTIFICATES
] as $key => $volume
) {
$device = new Local($volume);
@@ -601,7 +688,7 @@ App::get('/v1/health/anti-virus')
});
App::get('/v1/health/stats') // Currently only used internally
->desc('Get system stats')
->desc('Get system stats')
->groups(['api', 'health'])
->label('scope', 'root')
// ->label('sdk.auth', [APP_AUTH_TYPE_KEY])
app/controllers/api/migrations.php
+91 -27
View File
@@ -208,6 +208,16 @@ App::post('/v1/migrations/firebase')
->inject('queueForEvents')
->inject('queueForMigrations')
->action(function (array $resources, string $serviceAccount, Response $response, Database $dbForProject, Document $project, Document $user, Event $queueForEvents, Migration $queueForMigrations) {
$serviceAccountData = json_decode($serviceAccount, true);
if (empty($serviceAccountData)) {
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Invalid Service Account JSON');
}
if (!isset($serviceAccountData['project_id']) || !isset($serviceAccountData['client_email']) || !isset($serviceAccountData['private_key'])) {
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Invalid Service Account JSON');
}
$migration = $dbForProject->createDocument('migrations', new Document([
'$id' => ID::unique(),
'status' => 'pending',
@@ -449,15 +459,26 @@ App::get('/v1/migrations/appwrite/report')
->inject('project')
->inject('user')
->action(function (array $resources, string $endpoint, string $projectID, string $key, Response $response) {
try {
$appwrite = new Appwrite($projectID, $endpoint, $key);
$appwrite = new Appwrite($projectID, $endpoint, $key);
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($appwrite->report($resources)), Response::MODEL_MIGRATION_REPORT);
try {
$report = $appwrite->report($resources);
} catch (\Throwable $e) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Source Error: ' . $e->getMessage());
switch ($e->getCode()) {
case 401:
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE, 'Source Error: ' . $e->getMessage());
case 429:
throw new Exception(Exception::GENERAL_RATE_LIMIT_EXCEEDED, 'Source Error: Rate Limit Exceeded, Is your Cloud Provider blocking Appwrite\'s IP?');
case 500:
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Source Error: ' . $e->getMessage());
}
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Source Error: ' . $e->getMessage());
}
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($report), Response::MODEL_MIGRATION_REPORT);
});
App::get('/v1/migrations/firebase/report')
@@ -475,15 +496,36 @@ App::get('/v1/migrations/firebase/report')
->param('serviceAccount', '', new Text(65536), 'JSON of the Firebase service account credentials')
->inject('response')
->action(function (array $resources, string $serviceAccount, Response $response) {
try {
$firebase = new Firebase(json_decode($serviceAccount, true));
$serviceAccount = json_decode($serviceAccount, true);
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($firebase->report($resources)), Response::MODEL_MIGRATION_REPORT);
} catch (\Exception $e) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Source Error: ' . $e->getMessage());
if (empty($serviceAccount)) {
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Invalid Service Account JSON');
}
if (!isset($serviceAccount['project_id']) || !isset($serviceAccount['client_email']) || !isset($serviceAccount['private_key'])) {
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Invalid Service Account JSON');
}
$firebase = new Firebase($serviceAccount);
try {
$report = $firebase->report($resources);
} catch (\Throwable $e) {
switch ($e->getCode()) {
case 401:
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE, 'Source Error: ' . $e->getMessage());
case 429:
throw new Exception(Exception::GENERAL_RATE_LIMIT_EXCEEDED, 'Source Error: Rate Limit Exceeded, Is your Cloud Provider blocking Appwrite\'s IP?');
case 500:
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Source Error: ' . $e->getMessage());
}
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Source Error: ' . $e->getMessage());
}
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($report), Response::MODEL_MIGRATION_REPORT);
});
App::get('/v1/migrations/firebase/report/oauth')
@@ -869,15 +911,26 @@ App::get('/v1/migrations/supabase/report')
->inject('response')
->inject('dbForProject')
->action(function (array $resources, string $endpoint, string $apiKey, string $databaseHost, string $username, string $password, int $port, Response $response) {
try {
$supabase = new Supabase($endpoint, $apiKey, $databaseHost, 'postgres', $username, $password, $port);
$supabase = new Supabase($endpoint, $apiKey, $databaseHost, 'postgres', $username, $password, $port);
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($supabase->report($resources)), Response::MODEL_MIGRATION_REPORT);
} catch (\Exception $e) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Source Error: ' . $e->getMessage());
try {
$report = $supabase->report($resources);
} catch (\Throwable $e) {
switch ($e->getCode()) {
case 401:
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE, 'Source Error: ' . $e->getMessage());
case 429:
throw new Exception(Exception::GENERAL_RATE_LIMIT_EXCEEDED, 'Source Error: Rate Limit Exceeded, Is your Cloud Provider blocking Appwrite\'s IP?');
case 500:
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Source Error: ' . $e->getMessage());
}
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Source Error: ' . $e->getMessage());
}
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($report), Response::MODEL_MIGRATION_REPORT);
});
App::get('/v1/migrations/nhost/report')
@@ -901,15 +954,26 @@ App::get('/v1/migrations/nhost/report')
->param('port', 5432, new Integer(true), 'Source\'s Database Port.', true)
->inject('response')
->action(function (array $resources, string $subdomain, string $region, string $adminSecret, string $database, string $username, string $password, int $port, Response $response) {
try {
$nhost = new NHost($subdomain, $region, $adminSecret, $database, $username, $password, $port);
$nhost = new NHost($subdomain, $region, $adminSecret, $database, $username, $password, $port);
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($nhost->report($resources)), Response::MODEL_MIGRATION_REPORT);
} catch (\Exception $e) {
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Source Error: ' . $e->getMessage());
try {
$report = $nhost->report($resources);
} catch (\Throwable $e) {
switch ($e->getCode()) {
case 401:
throw new Exception(Exception::GENERAL_UNAUTHORIZED_SCOPE, 'Source Error: ' . $e->getMessage());
case 429:
throw new Exception(Exception::GENERAL_RATE_LIMIT_EXCEEDED, 'Source Error: Rate Limit Exceeded, Is your Cloud Provider blocking Appwrite\'s IP?');
case 500:
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Source Error: ' . $e->getMessage());
}
throw new Exception(Exception::MIGRATION_PROVIDER_ERROR, 'Source Error: ' . $e->getMessage());
}
$response
->setStatusCode(Response::STATUS_CODE_OK)
->dynamic(new Document($report), Response::MODEL_MIGRATION_REPORT);
});
App::patch('/v1/migrations/:migrationId')
app/controllers/api/projects.php
-15
View File
@@ -100,11 +100,6 @@ App::post('/v1/projects')
$backups['database_db_fra1_v14x_07'] = ['from' => '00:00', 'to' => '02:00'];
$databases = Config::getParam('pools-database', []);
$databaseSelfHosted = 'database_db_fra1_self_hosted_0_0';
$selfHostedIndex = array_search($databaseSelfHosted, $databases);
if ($selfHostedIndex !== false) {
unset($databases[$selfHostedIndex]);
}
/**
* Remove databases from the list that are currently undergoing an backup
@@ -175,16 +170,6 @@ App::post('/v1/projects')
throw new Exception(Exception::PROJECT_ALREADY_EXISTS);
}
/**
* Update database with self-managed db every $mod projects
*/
$mod = 20;
if ($project->getInternalId() % $mod === 0 && $selfHostedIndex !== false) {
$database = $databaseSelfHosted;
$project->setAttribute('database', $database);
$dbForConsole->updateDocument('projects', $project->getId(), $project);
}
$dbForProject = new Database($pools->get($database)->pop()->getResource(), $cache);
$dbForProject->setNamespace("_{$project->getInternalId()}");
$dbForProject->create();
app/controllers/api/proxy.php
+5 -1
View File
@@ -87,7 +87,11 @@ App::post('/v1/proxy/rules')
$resourceInternalId = $function->getInternalId();
}
$domain = new Domain($domain);
try {
$domain = new Domain($domain);
} catch (\Exception) {
throw new Exception(Exception::GENERAL_ARGUMENT_INVALID, 'Domain may not start with http:// or https://.');
}
$ruleId = ID::unique();
$rule = new Document([
app/controllers/api/storage.php
-2
View File
@@ -1094,7 +1094,6 @@ App::get('/v1/storage/buckets/:bucketId/files/:fileId/download')
}
if ($size > APP_STORAGE_READ_BUFFER) {
$response->addHeader('Content-Length', $deviceFiles->getFileSize($path));
for ($i = 0; $i < ceil($size / MAX_OUTPUT_CHUNK_SIZE); $i++) {
$response->chunk(
$deviceFiles->read(
@@ -1245,7 +1244,6 @@ App::get('/v1/storage/buckets/:bucketId/files/:fileId/view')
$size = $deviceFiles->getFileSize($path);
if ($size > APP_STORAGE_READ_BUFFER) {
$response->addHeader('Content-Length', $deviceFiles->getFileSize($path));
for ($i = 0; $i < ceil($size / MAX_OUTPUT_CHUNK_SIZE); $i++) {
$response->chunk(
$deviceFiles->read(
app/controllers/api/users.php
+2 -2
View File
@@ -85,7 +85,7 @@ function createUser(string $hash, mixed $hashOptions, string $userId, ?string $e
'status' => true,
'labels' => [],
'password' => $password,
'passwordHistory' => is_null($password) && $passwordHistory === 0 ? [] : [$password],
'passwordHistory' => is_null($password) || $passwordHistory === 0 ? [] : [$password],
'passwordUpdate' => (!empty($password)) ? DateTime::now() : null,
'hash' => $hash === 'plaintext' ? Auth::DEFAULT_ALGO : $hash,
'hashOptions' => $hash === 'plaintext' ? Auth::DEFAULT_ALGO_OPTIONS : $hashOptionsObject + ['type' => $hash],
@@ -1102,7 +1102,7 @@ App::delete('/v1/users/:userId/sessions/:sessionId')
App::delete('/v1/users/:userId/sessions')
->desc('Delete user sessions')
->groups(['api', 'users'])
->label('event', 'users.[userId].sessions.[sessionId].delete')
->label('event', 'users.[userId].sessions.delete')
->label('scope', 'users.write')
->label('audits.event', 'session.delete')
->label('audits.resource', 'user/{user.$id}')
app/controllers/api/vcs.php
+15 -15
View File
@@ -857,10 +857,10 @@ App::post('/v1/vcs/github/events')
$github->initializeVariables($providerInstallationId, $privateKey, $githubAppId);
//find functionId from functions table
$repositories = $dbForConsole->find('repositories', [
$repositories = Authorization::skip(fn () => $dbForConsole->find('repositories', [
Query::equal('providerRepositoryId', [$providerRepositoryId]),
Query::limit(100),
]);
]));
// create new deployment only on push and not when branch is created
if (!$providerBranchCreated) {
@@ -877,13 +877,13 @@ App::post('/v1/vcs/github/events')
]);
foreach ($installations as $installation) {
$repositories = $dbForConsole->find('repositories', [
$repositories = Authorization::skip(fn () => $dbForConsole->find('repositories', [
Query::equal('installationInternalId', [$installation->getInternalId()]),
Query::limit(1000)
]);
]));
foreach ($repositories as $repository) {
$dbForConsole->deleteDocument('repositories', $repository->getId());
Authorization::skip(fn () => $dbForConsole->deleteDocument('repositories', $repository->getId()));
}
$dbForConsole->deleteDocument('installations', $installation->getId());
@@ -915,10 +915,10 @@ App::post('/v1/vcs/github/events')
$providerCommitAuthor = $commitDetails["commitAuthor"] ?? '';
$providerCommitMessage = $commitDetails["commitMessage"] ?? '';
$repositories = $dbForConsole->find('repositories', [
$repositories = Authorization::skip(fn () => $dbForConsole->find('repositories', [
Query::equal('providerRepositoryId', [$providerRepositoryId]),
Query::orderDesc('$createdAt')
]);
]));
$createGitDeployments($github, $providerInstallationId, $repositories, $providerBranch, $providerBranchUrl, $providerRepositoryName, $providerRepositoryUrl, $providerRepositoryOwner, $providerCommitHash, $providerCommitAuthor, $providerCommitAuthorUrl, $providerCommitMessage, $providerCommitUrl, $providerPullRequestId, $external, $dbForConsole, $queueForBuilds, $getProjectDB, $request);
} elseif ($parsedPayload["action"] == "closed") {
@@ -929,10 +929,10 @@ App::post('/v1/vcs/github/events')
$external = $parsedPayload["external"] ?? true;
if ($external) {
$repositories = $dbForConsole->find('repositories', [
$repositories = Authorization::skip(fn () => $dbForConsole->find('repositories', [
Query::equal('providerRepositoryId', [$providerRepositoryId]),
Query::orderDesc('$createdAt')
]);
]));
foreach ($repositories as $repository) {
$providerPullRequestIds = $repository->getAttribute('providerPullRequestIds', []);
@@ -1046,8 +1046,8 @@ App::delete('/v1/vcs/installations/:installationId')
->inject('response')
->inject('project')
->inject('dbForConsole')
->inject('deletes')
->action(function (string $installationId, Response $response, Document $project, Database $dbForConsole, Delete $deletes) {
->inject('queueForDeletes')
->action(function (string $installationId, Response $response, Document $project, Database $dbForConsole, Delete $queueForDeletes) {
$installation = $dbForConsole->getDocument('installations', $installationId);
if ($installation->isEmpty()) {
@@ -1058,7 +1058,7 @@ App::delete('/v1/vcs/installations/:installationId')
throw new Exception(Exception::GENERAL_SERVER_ERROR, 'Failed to remove installation from DB');
}
$deletes
$queueForDeletes
->setType(DELETE_TYPE_DOCUMENT)
->setDocument($installation);
@@ -1092,9 +1092,9 @@ App::patch('/v1/vcs/github/installations/:installationId/repositories/:repositor
throw new Exception(Exception::INSTALLATION_NOT_FOUND);
}
$repository = $dbForConsole->getDocument('repositories', $repositoryId, [
$repository = Authorization::skip(fn () => $dbForConsole->getDocument('repositories', $repositoryId, [
Query::equal('projectInternalId', [$project->getInternalId()])
]);
]));
if ($repository->isEmpty()) {
throw new Exception(Exception::REPOSITORY_NOT_FOUND);
@@ -1109,7 +1109,7 @@ App::patch('/v1/vcs/github/installations/:installationId/repositories/:repositor
// TODO: Delete from array when PR is closed
$repository = $dbForConsole->updateDocument('repositories', $repository->getId(), $repository);
$repository = Authorization::skip(fn () => $dbForConsole->updateDocument('repositories', $repository->getId(), $repository));
$privateKey = App::getEnv('_APP_VCS_GITHUB_PRIVATE_KEY');
$githubAppId = App::getEnv('_APP_VCS_GITHUB_APP_ID');
app/controllers/general.php
+6 -1
View File
@@ -608,8 +608,13 @@ App::error()
$version = App::getEnv('_APP_VERSION', 'UNKNOWN');
$route = $utopia->getRoute();
$publish = true;
if ($logger) {
if ($error instanceof AppwriteException) {
$publish = $error->isPublishable();
}
if ($logger && $publish) {
if ($error->getCode() >= 500 || $error->getCode() === 0) {
try {
/** @var Utopia\Database\Document $user */
app/controllers/mock.php
-494
View File
@@ -9,508 +9,14 @@ use Appwrite\Utopia\Request;
use Appwrite\Utopia\Response;
use Utopia\App;
use Utopia\Database\Database;
use Utopia\Validator\ArrayList;
use Utopia\Validator\Integer;
use Utopia\Validator\Text;
use Utopia\Storage\Validator\File;
use Utopia\Validator\WhiteList;
use Utopia\Database\Helpers\ID;
use Utopia\Database\Validator\UID;
use Utopia\Validator\Nullable;
use Utopia\VCS\Adapter\Git\GitHub;
use Utopia\Database\Helpers\Permission;
use Utopia\Database\Helpers\Role;
App::get('/v1/mock/tests/foo')
->desc('Get Foo')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'foo')
->label('sdk.method', 'get')
->label('sdk.description', 'Mock a get request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('x', '', new Text(100), 'Sample string param')
->param('y', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($x, $y, $z) {
});
App::post('/v1/mock/tests/foo')
->desc('Post Foo')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'foo')
->label('sdk.method', 'post')
->label('sdk.description', 'Mock a post request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('x', '', new Text(100), 'Sample string param')
->param('y', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($x, $y, $z) {
});
App::patch('/v1/mock/tests/foo')
->desc('Patch Foo')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'foo')
->label('sdk.method', 'patch')
->label('sdk.description', 'Mock a patch request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('x', '', new Text(100), 'Sample string param')
->param('y', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($x, $y, $z) {
});
App::put('/v1/mock/tests/foo')
->desc('Put Foo')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'foo')
->label('sdk.method', 'put')
->label('sdk.description', 'Mock a put request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('x', '', new Text(100), 'Sample string param')
->param('y', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($x, $y, $z) {
});
App::delete('/v1/mock/tests/foo')
->desc('Delete Foo')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'foo')
->label('sdk.method', 'delete')
->label('sdk.description', 'Mock a delete request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('x', '', new Text(100), 'Sample string param')
->param('y', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($x, $y, $z) {
});
App::get('/v1/mock/tests/bar')
->desc('Get Bar')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'bar')
->label('sdk.method', 'get')
->label('sdk.description', 'Mock a get request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('required', '', new Text(100), 'Sample string param')
->param('default', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($required, $default, $z) {
});
App::post('/v1/mock/tests/bar')
->desc('Post Bar')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'bar')
->label('sdk.method', 'post')
->label('sdk.description', 'Mock a post request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.offline.model', '/mock/tests/bar')
->label('sdk.offline.key', '{required}')
->label('sdk.mock', true)
->param('required', '', new Text(100), 'Sample string param')
->param('default', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($required, $default, $z) {
});
App::patch('/v1/mock/tests/bar')
->desc('Patch Bar')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'bar')
->label('sdk.method', 'patch')
->label('sdk.description', 'Mock a patch request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('required', '', new Text(100), 'Sample string param')
->param('default', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($required, $default, $z) {
});
App::put('/v1/mock/tests/bar')
->desc('Put Bar')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'bar')
->label('sdk.method', 'put')
->label('sdk.description', 'Mock a put request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('required', '', new Text(100), 'Sample string param')
->param('default', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($required, $default, $z) {
});
App::delete('/v1/mock/tests/bar')
->desc('Delete Bar')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'bar')
->label('sdk.method', 'delete')
->label('sdk.description', 'Mock a delete request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('required', '', new Text(100), 'Sample string param')
->param('default', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->action(function ($required, $default, $z) {
});
App::get('/v1/mock/tests/general/headers')
->desc('Get headers')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'headers')
->label('sdk.description', 'Return headers from the request')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->inject('request')
->inject('response')
->action(function (Request $request, Response $response) {
$res = [
'x-sdk-name' => $request->getHeader('x-sdk-name'),
'x-sdk-platform' => $request->getHeader('x-sdk-platform'),
'x-sdk-language' => $request->getHeader('x-sdk-language'),
'x-sdk-version' => $request->getHeader('x-sdk-version'),
];
$res = array_map(function ($key, $value) {
return $key . ': ' . $value;
}, array_keys($res), $res);
$res = implode("; ", $res);
$response->dynamic(new Document(['result' => $res]), Response::MODEL_MOCK);
});
App::get('/v1/mock/tests/general/download')
->desc('Download File')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'download')
->label('sdk.methodType', 'location')
->label('sdk.description', 'Mock a file download request.')
->label('sdk.response.type', '*/*')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.mock', true)
->inject('response')
->action(function (Response $response) {
$response
->setContentType('text/plain')
->addHeader('Content-Disposition', 'attachment; filename="test.txt"')
->addHeader('Expires', \date('D, d M Y H:i:s', \time() + (60 * 60 * 24 * 45)) . ' GMT') // 45 days cache
->addHeader('X-Peak', \memory_get_peak_usage())
->send("GET:/v1/mock/tests/general/download:passed");
});
App::post('/v1/mock/tests/general/upload')
->desc('Upload File')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'upload')
->label('sdk.description', 'Mock a file upload request.')
->label('sdk.request.type', 'multipart/form-data')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->param('x', '', new Text(100), 'Sample string param')
->param('y', '', new Integer(true), 'Sample numeric param')
->param('z', null, new ArrayList(new Text(256), APP_LIMIT_ARRAY_PARAMS_SIZE), 'Sample array param')
->param('file', [], new File(), 'Sample file param', skipValidation: true)
->inject('request')
->inject('response')
->action(function (string $x, int $y, array $z, mixed $file, Request $request, Response $response) {
$file = $request->getFiles('file');
$contentRange = $request->getHeader('content-range');
$chunkSize = 5 * 1024 * 1024; // 5MB
if (!empty($contentRange)) {
$start = $request->getContentRangeStart();
$end = $request->getContentRangeEnd();
$size = $request->getContentRangeSize();
$id = $request->getHeader('x-appwrite-id', '');
$file['size'] = (\is_array($file['size'])) ? $file['size'][0] : $file['size'];
if (is_null($start) || is_null($end) || is_null($size) || $end >= $size) {
throw new Exception(Exception::GENERAL_MOCK, 'Invalid content-range header');
}
if ($start > $end || $end > $size) {
throw new Exception(Exception::GENERAL_MOCK, 'Invalid content-range header');
}
if ($start === 0 && !empty($id)) {
throw new Exception(Exception::GENERAL_MOCK, 'First chunked request cannot have id header');
}
if ($start !== 0 && $id !== 'newfileid') {
throw new Exception(Exception::GENERAL_MOCK, 'All chunked request must have id header (except first)');
}
if ($end !== $size - 1 && $end - $start + 1 !== $chunkSize) {
throw new Exception(Exception::GENERAL_MOCK, 'Chunk size must be 5MB (except last chunk)');
}
if ($end !== $size - 1 && $file['size'] !== $chunkSize) {
throw new Exception(Exception::GENERAL_MOCK, 'Wrong chunk size');
}
if ($file['size'] > $chunkSize) {
throw new Exception(Exception::GENERAL_MOCK, 'Chunk size must be 5MB or less');
}
if ($end !== $size - 1) {
$response->json([
'$id' => ID::custom('newfileid'),
'chunksTotal' => (int) ceil($size / ($end + 1 - $start)),
'chunksUploaded' => ceil($start / $chunkSize) + 1
]);
}
} else {
$file['tmp_name'] = (\is_array($file['tmp_name'])) ? $file['tmp_name'][0] : $file['tmp_name'];
$file['name'] = (\is_array($file['name'])) ? $file['name'][0] : $file['name'];
$file['size'] = (\is_array($file['size'])) ? $file['size'][0] : $file['size'];
if ($file['name'] !== 'file.png') {
throw new Exception(Exception::GENERAL_MOCK, 'Wrong file name');
}
if ($file['size'] !== 38756) {
throw new Exception(Exception::GENERAL_MOCK, 'Wrong file size');
}
if (\md5(\file_get_contents($file['tmp_name'])) !== 'd80e7e6999a3eb2ae0d631a96fe135a4') {
throw new Exception(Exception::GENERAL_MOCK, 'Wrong file uploaded');
}
}
});
App::get('/v1/mock/tests/general/redirect')
->desc('Redirect')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'redirect')
->label('sdk.description', 'Mock a redirect request.')
->label('sdk.response.code', Response::STATUS_CODE_MOVED_PERMANENTLY)
->label('sdk.response.type', Response::CONTENT_TYPE_HTML)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->inject('response')
->action(function (Response $response) {
$response->redirect('/v1/mock/tests/general/redirect/done');
});
App::get('/v1/mock/tests/general/redirect/done')
->desc('Redirected')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'redirected')
->label('sdk.description', 'Mock a redirected request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->action(function () {
});
App::get('/v1/mock/tests/general/set-cookie')
->desc('Set Cookie')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'setCookie')
->label('sdk.description', 'Mock a set cookie request.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->inject('response')
->inject('request')
->action(function (Response $response, Request $request) {
$response->addCookie('cookieName', 'cookieValue', \time() + 31536000, '/', $request->getHostname(), true, true);
});
App::get('/v1/mock/tests/general/get-cookie')
->desc('Get Cookie')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'getCookie')
->label('sdk.description', 'Mock a cookie response.')
->label('sdk.response.code', Response::STATUS_CODE_OK)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_MOCK)
->label('sdk.mock', true)
->inject('request')
->action(function (Request $request) {
if ($request->getCookie('cookieName', '') !== 'cookieValue') {
throw new Exception(Exception::GENERAL_MOCK, 'Missing cookie value');
}
});
App::get('/v1/mock/tests/general/empty')
->desc('Empty Response')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'empty')
->label('sdk.description', 'Mock an empty response.')
->label('sdk.response.code', Response::STATUS_CODE_NOCONTENT)
->label('sdk.response.model', Response::MODEL_NONE)
->label('sdk.mock', true)
->inject('response')
->action(function (Response $response) {
$response->noContent();
});
App::post('/v1/mock/tests/general/nullable')
->desc('Nullable Test')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'nullable')
->label('sdk.description', 'Mock a nullable parameter.')
->label('sdk.mock', true)
->param('required', '', new Text(100), 'Sample string param')
->param('nullable', '', new Nullable(new Text(100)), 'Sample string param')
->param('optional', '', new Text(100), 'Sample string param', true)
->action(function (string $required, string $nullable, ?string $optional) {
});
App::post('/v1/mock/tests/general/enum')
->desc('Enum Test')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'enum')
->label('sdk.description', 'Mock an enum parameter.')
->label('sdk.mock', true)
->param('mockType', '', new WhiteList(['first', 'second', 'third']), 'Sample enum param')
->action(function (string $mockType) {
});
App::get('/v1/mock/tests/general/400-error')
->desc('400 Error')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'error400')
->label('sdk.description', 'Mock a 400 failed request.')
->label('sdk.response.code', Response::STATUS_CODE_BAD_REQUEST)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_ERROR)
->label('sdk.mock', true)
->action(function () {
throw new Exception(Exception::GENERAL_MOCK, 'Mock 400 error');
});
App::get('/v1/mock/tests/general/500-error')
->desc('500 Error')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.auth', [APP_AUTH_TYPE_SESSION, APP_AUTH_TYPE_KEY, APP_AUTH_TYPE_JWT])
->label('sdk.namespace', 'general')
->label('sdk.method', 'error500')
->label('sdk.description', 'Mock a 500 failed request.')
->label('sdk.response.code', Response::STATUS_CODE_INTERNAL_SERVER_ERROR)
->label('sdk.response.type', Response::CONTENT_TYPE_JSON)
->label('sdk.response.model', Response::MODEL_ERROR)
->label('sdk.mock', true)
->action(function () {
throw new Exception(Exception::GENERAL_MOCK, 'Mock 500 error', 500);
});
App::get('/v1/mock/tests/general/502-error')
->desc('502 Error')
->groups(['mock'])
->label('scope', 'public')
->label('sdk.platform', [APP_PLATFORM_CLIENT, APP_PLATFORM_SERVER])
->label('sdk.namespace', 'general')
->label('sdk.method', 'error502')
->label('sdk.description', 'Mock a 502 bad gateway.')
->label('sdk.response.code', Response::STATUS_CODE_BAD_GATEWAY)
->label('sdk.response.type', Response::CONTENT_TYPE_TEXT)
->label('sdk.response.model', Response::MODEL_ANY)
->label('sdk.mock', true)
->inject('response')
->action(function (Response $response) {
$response
->setStatusCode(502)
->text('This is a text error');
});
App::get('/v1/mock/tests/general/oauth2')
->desc('OAuth Login')
->groups(['mock'])
app/controllers/shared/api.php
+12 -1
View File
@@ -173,13 +173,16 @@ App::init()
$abuseKeyLabel = (!is_array($abuseKeyLabel)) ? [$abuseKeyLabel] : $abuseKeyLabel;
foreach ($abuseKeyLabel as $abuseKey) {
$start = $request->getContentRangeStart();
$end = $request->getContentRangeEnd();
$timeLimit = new TimeLimit($abuseKey, $route->getLabel('abuse-limit', 0), $route->getLabel('abuse-time', 3600), $dbForProject);
$timeLimit
->setParam('{userId}', $user->getId())
->setParam('{userAgent}', $request->getUserAgent(''))
->setParam('{ip}', $request->getIP())
->setParam('{url}', $request->getHostname() . $route->getPath())
->setParam('{method}', $request->getMethod());
->setParam('{method}', $request->getMethod())
->setParam('{chunkId}', (int) ($start / ($end + 1 - $start)));
$timeLimitArray[] = $timeLimit;
}
@@ -590,3 +593,11 @@ App::shutdown()
->trigger();
}
});
App::init()
->groups(['usage'])
->action(function () {
if (App::getEnv('_APP_USAGE_STATS', 'enabled') !== 'enabled') {
throw new Exception(Exception::GENERAL_USAGE_DISABLED);
}
});
app/init.php
+22 -2
View File
@@ -109,8 +109,8 @@ const APP_LIMIT_LIST_DEFAULT = 25; // Default maximum number of items to return
const APP_KEY_ACCCESS = 24 * 60 * 60; // 24 hours
const APP_USER_ACCCESS = 24 * 60 * 60; // 24 hours
const APP_CACHE_UPDATE = 24 * 60 * 60; // 24 hours
const APP_CACHE_BUSTER = 515;
const APP_VERSION_STABLE = '1.4.9';
const APP_CACHE_BUSTER = 328;
const APP_VERSION_STABLE = '1.4.13';
const APP_DATABASE_ATTRIBUTE_EMAIL = 'email';
const APP_DATABASE_ATTRIBUTE_ENUM = 'enum';
const APP_DATABASE_ATTRIBUTE_IP = 'ip';
@@ -768,6 +768,26 @@ $register->set('pools', function () {
return $group;
});
$register->set('db', function () {
// This is usually for our workers or CLI commands scope
$dbHost = App::getEnv('_APP_DB_HOST', '');
$dbPort = App::getEnv('_APP_DB_PORT', '');
$dbUser = App::getEnv('_APP_DB_USER', '');
$dbPass = App::getEnv('_APP_DB_PASS', '');
$dbScheme = App::getEnv('_APP_DB_SCHEMA', '');
$pdo = new PDO("mysql:host={$dbHost};port={$dbPort};dbname={$dbScheme};charset=utf8mb4", $dbUser, $dbPass, array(
PDO::ATTR_TIMEOUT => 3, // Seconds
PDO::ATTR_PERSISTENT => true,
PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_EMULATE_PREPARES => true,
PDO::ATTR_STRINGIFY_FETCHES => true,
));
return $pdo;
});
$register->set('smtp', function () {
$mail = new PHPMailer(true);
bin/get-migration-stats
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
php /usr/src/code/app/cli.php get-migration-stats $@
bin/patch-recreate-repositories-documents
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
php /usr/src/code/app/cli.php patch-recreate-repositories-documents $@
composer.json
+3 -3
View File
@@ -43,13 +43,13 @@
"ext-sockets": "*",
"appwrite/php-runtimes": "0.13.*",
"appwrite/php-clamav": "2.0.*",
"utopia-php/abuse": "0.32.*",
"utopia-php/abuse": "0.33.*",
"utopia-php/analytics": "0.10.*",
"utopia-php/audit": "0.34.*",
"utopia-php/audit": "0.35.*",
"utopia-php/cache": "0.8.*",
"utopia-php/cli": "0.15.*",
"utopia-php/config": "0.2.*",
"utopia-php/database": "0.44.*",
"utopia-php/database": "0.45.*",
"utopia-php/domains": "0.3.*",
"utopia-php/dsn": "0.1.*",
"utopia-php/framework": "0.31.0",
src/Appwrite/Auth/Validator/PasswordHistory.php
+1 -1
View File
@@ -44,7 +44,7 @@ class PasswordHistory extends Password
public function isValid($value): bool
{
foreach ($this->history as $hash) {
if (Auth::passwordVerify($value, $hash, $this->algo, $this->algoOptions)) {
if (!empty($hash) && Auth::passwordVerify($value, $hash, $this->algo, $this->algoOptions)) {
return false;
}
}
src/Appwrite/Extend/Exception.php
+21 -3
View File
@@ -84,6 +84,8 @@ class Exception extends \Exception
public const USER_OAUTH2_BAD_REQUEST = 'user_oauth2_bad_request';
public const USER_OAUTH2_UNAUTHORIZED = 'user_oauth2_unauthorized';
public const USER_OAUTH2_PROVIDER_ERROR = 'user_oauth2_provider_error';
public const USER_EMAIL_ALREADY_VERIFIED = 'user_email_alread_verified';
public const USER_PHONE_ALREADY_VERIFIED = 'user_phone_already_verified';
/** Teams */
public const TEAM_NOT_FOUND = 'team_not_found';
@@ -224,14 +226,19 @@ class Exception extends \Exception
public const MIGRATION_NOT_FOUND = 'migration_not_found';
public const MIGRATION_ALREADY_EXISTS = 'migration_already_exists';
public const MIGRATION_IN_PROGRESS = 'migration_in_progress';
public const MIGRATION_PROVIDER_ERROR = 'migration_provider_error';
/** Realtime */
public const REALTIME_MESSAGE_FORMAT_INVALID = 'realtime_message_format_invalid';
public const REALTIME_TOO_MANY_MESSAGES = 'realtime_too_many_messages';
public const REALTIME_POLICY_VIOLATION = 'realtime_policy_violation';
public const REALTIME_MESSAGE_FORMAT_INVALID = 'realtime_message_format_invalid';
public const REALTIME_TOO_MANY_MESSAGES = 'realtime_too_many_messages';
public const REALTIME_POLICY_VIOLATION = 'realtime_policy_violation';
/** Health */
public const QUEUE_SIZE_EXCEEDED = 'queue_size_exceeded';
protected string $type = '';
protected array $errors = [];
protected bool $publish = true;
public function __construct(string $type = Exception::GENERAL_UNKNOWN, string $message = null, int $code = null, \Throwable $previous = null)
{
@@ -241,6 +248,7 @@ class Exception extends \Exception
if (isset($this->errors[$type])) {
$this->code = $this->errors[$type]['code'];
$this->message = $this->errors[$type]['description'];
$this->publish = $this->errors[$type]['publish'] ?? true;
}
$this->message = $message ?? $this->message;
@@ -270,4 +278,14 @@ class Exception extends \Exception
{
$this->type = $type;
}
/**
* Check whether the log is publishable for the exception.
*
* @return bool
*/
public function isPublishable(): bool
{
return $this->publish;
}
}
src/Appwrite/Migration/Migration.php
+15 -5
View File
@@ -72,10 +72,13 @@ abstract class Migration
'1.4.5' => 'V19',
'1.4.6' => 'V19',
'1.4.7' => 'V19',
'1.4.8' => 'V20',
'1.4.8' => 'V19',
'1.4.9' => 'V19',
'1.4.10' => 'V20',
'1.4.10' => 'V19',
'1.4.11' => 'V19',
'1.4.12' => 'V19',
'1.4.13' => 'V19'
'1.4.14' => 'V20'
];
/**
@@ -193,17 +196,24 @@ abstract class Migration
* @return iterable<Document>
* @throws \Exception
*/
public function documentsIterator(string $collectionId): iterable
public function documentsIterator(string $collectionId, $queries = []): iterable
{
$sum = 0;
$nextDocument = null;
$collectionCount = $this->projectDB->count($collectionId);
$queries[] = Query::limit($this->limit);
do {
$queries = [Query::limit($this->limit)];
if ($nextDocument !== null) {
$queries[] = Query::cursorAfter($nextDocument);
$cursorQueryIndex = \array_search('cursorAfter', \array_map(fn (Query $query) => $query->getMethod(), $queries));
if ($cursorQueryIndex !== false) {
$queries[$cursorQueryIndex] = Query::cursorAfter($nextDocument);
} else {
$queries[] = Query::cursorAfter($nextDocument);
}
}
$documents = $this->projectDB->find($collectionId, $queries);
$count = count($documents);
$sum += $count;
src/Appwrite/Migration/Version/V19.php
+22
View File
@@ -10,6 +10,7 @@ use Utopia\Database\Database;
use Utopia\Database\DateTime;
use Utopia\Database\Document;
use Utopia\Database\Exception;
use Utopia\Database\Query;
class V19 extends Migration
{
@@ -41,6 +42,11 @@ class V19 extends Migration
Console::info('Migrating Buckets');
$this->migrateBuckets();
if ($this->project->getId() !== 'console') {
Console::info('Migrating Enum Attribute Size');
$this->migrateEnumAttributeSize();
}
Console::info('Migrating Documents');
$this->forEachDocument([$this, 'fixDocument']);
@@ -640,6 +646,22 @@ class V19 extends Migration
return $commands;
}
private function migrateEnumAttributeSize(): void
{
foreach (
$this->documentsIterator('attributes', [
Query::equal('format', ['enum']),
Query::lessThan('size', Database::LENGTH_KEY)
]) as $attribute
) {
$attribute->setAttribute('size', Database::LENGTH_KEY);
$this->projectDB->updateDocument('attributes', $attribute->getId(), $attribute);
$databaseInternalId = $attribute->getAttribute('databaseInternalId');
$collectionInternalId = $attribute->getAttribute('collectionInternalId');
$this->projectDB->updateAttribute('database_' . $databaseInternalId . '_collection_' . $collectionInternalId, $attribute->getAttribute('key'), size: 255);
}
}
/**
* Fix run on each document
*
src/Appwrite/Platform/Services/Tasks.php
+6 -1
View File
@@ -2,7 +2,6 @@
namespace Appwrite\Platform\Services;
use Appwrite\Platform\Tasks\CreateInfMetric;
use Utopia\Platform\Service;
use Appwrite\Platform\Tasks\Doctor;
use Appwrite\Platform\Tasks\Install;
@@ -19,6 +18,10 @@ use Appwrite\Platform\Tasks\VolumeSync;
use Appwrite\Platform\Tasks\CalcTierStats;
use Appwrite\Platform\Tasks\Upgrade;
use Appwrite\Platform\Tasks\DeleteOrphanedProjects;
use Appwrite\Platform\Tasks\GetMigrationStats;
use Appwrite\Platform\Tasks\PatchRecreateRepositoriesDocuments;
use Appwrite\Platform\Tasks\CreateInfMetric;
class Tasks extends Service
{
@@ -41,6 +44,8 @@ class Tasks extends Service
->addAction(Specs::getName(), new Specs())
->addAction(CalcTierStats::getName(), new CalcTierStats())
->addAction(DeleteOrphanedProjects::getName(), new DeleteOrphanedProjects())
->addAction(PatchRecreateRepositoriesDocuments::getName(), new PatchRecreateRepositoriesDocuments())
->addAction(GetMigrationStats::getName(), new GetMigrationStats())
->addAction(CreateInfMetric::getName(), new CreateInfMetric())
;
src/Appwrite/Platform/Tasks/CalcTierStats.php
+11 -3
View File
@@ -41,6 +41,7 @@ class CalcTierStats extends Action
'Functions',
'Deployments',
'Executions',
'Migrations',
];
protected string $directory = '/usr/local';
@@ -99,8 +100,8 @@ class CalcTierStats extends Action
$projects = [$console];
$count = 0;
$limit = 30;
$sum = 30;
$limit = 100;
$sum = 100;
$offset = 0;
while (!empty($projects)) {
foreach ($projects as $project) {
@@ -200,7 +201,7 @@ class CalcTierStats extends Action
try {
/** Get Domains */
$stats['Domains'] = $dbForConsole->count('domains', [
$stats['Domains'] = $dbForConsole->count('rules', [
Query::equal('projectInternalId', [$project->getInternalId()]),
]);
} catch (\Throwable) {
@@ -290,6 +291,13 @@ class CalcTierStats extends Action
$stats['Executions'] = 0;
}
/** Get Total Migrations */
try {
$stats['Migrations'] = $dbForProject->count('migrations', []);
} catch (\Throwable) {
$stats['Migrations'] = 0;
}
$csv->insertOne(array_values($stats));
} catch (\Throwable $th) {
Console::error('Failed on project ("' . $project->getId() . '") version with error on File: ' . $th->getFile() . ' line no: ' . $th->getline() . ' with message: ' . $th->getMessage());
src/Appwrite/Platform/Tasks/DeleteOrphanedProjects.php
+65 -19
View File
@@ -2,17 +2,17 @@
namespace Appwrite\Platform\Tasks;
use PHPMailer\PHPMailer\PHPMailer;
use Utopia\App;
use Utopia\Config\Config;
use Utopia\Database\Helpers\ID;
use Utopia\Database\Query;
use Utopia\Database\Validator\Authorization;
use Utopia\Platform\Action;
use Utopia\Cache\Cache;
use Utopia\CLI\Console;
use Utopia\Database\Database;
use Utopia\Pools\Group;
use Utopia\Registry\Registry;
use Utopia\Validator\Boolean;
class DeleteOrphanedProjects extends Action
{
@@ -25,18 +25,19 @@ class DeleteOrphanedProjects extends Action
{
$this
->desc('Get stats for projects')
->desc('Delete orphaned projects')
->param('commit', false, new Boolean(true), 'Commit project deletion', true)
->inject('pools')
->inject('cache')
->inject('dbForConsole')
->inject('register')
->callback(function (Group $pools, Cache $cache, Database $dbForConsole, Registry $register) {
$this->action($pools, $cache, $dbForConsole, $register);
->callback(function (bool $commit, Group $pools, Cache $cache, Database $dbForConsole, Registry $register) {
$this->action($commit, $pools, $cache, $dbForConsole, $register);
});
}
public function action(Group $pools, Cache $cache, Database $dbForConsole, Registry $register): void
public function action(bool $commit, Group $pools, Cache $cache, Database $dbForConsole, Registry $register): void
{
Console::title('Delete orphaned projects V1');
@@ -45,6 +46,17 @@ class DeleteOrphanedProjects extends Action
/** @var array $collections */
$collectionsConfig = Config::getParam('collections', [])['projects'] ?? [];
$collectionsConfig = array_merge([
'audit' => [
'$id' => ID::custom('audit'),
'$collection' => Database::METADATA
],
'abuse' => [
'$id' => ID::custom('abuse'),
'$collection' => Database::METADATA
]
], $collectionsConfig);
/* Initialise new Utopia app */
$app = new App('UTC');
$console = $app->getResource('console');
@@ -54,7 +66,8 @@ class DeleteOrphanedProjects extends Action
$totalProjects = $dbForConsole->count('projects');
Console::success("Found a total of: {$totalProjects} projects");
$orphans = 0;
$orphans = 1;
$cnt = 0;
$count = 0;
$limit = 30;
$sum = 30;
@@ -79,19 +92,52 @@ class DeleteOrphanedProjects extends Action
$dbForProject = new Database($adapter, $cache);
$dbForProject->setDefaultDatabase('appwrite');
$dbForProject->setNamespace('_' . $project->getInternalId());
$collectionsCreated = $dbForProject->count(Database::METADATA);
$message = ' (' . $collectionsCreated . ') collections where found on project (' . $project->getId() . '))';
if ($collectionsCreated < (count($collectionsConfig) + 2)) {
Console::error($message);
$orphans++;
} else {
Console::log($message);
$collectionsCreated = 0;
$cnt++;
if ($dbForProject->exists($dbForProject->getDefaultDatabase(), Database::METADATA)) {
$collectionsCreated = $dbForProject->count(Database::METADATA);
}
} catch (\Throwable $th) {
//$dbForConsole->deleteDocument('projects', $project->getId());
//Console::success('Deleting project (' . $project->getId() . ')');
Console::error(' (0) collections where found for project (' . $project->getId() . ')');
$msg = '(' . $cnt . ') found (' . $collectionsCreated . ') collections on project (' . $project->getInternalId() . ') , database (' . $project['database'] . ')';
if ($collectionsCreated >= count($collectionsConfig)) {
Console::log($msg . ' ignoring....');
continue;
}
Console::log($msg);
if ($collectionsCreated > 0) {
$collections = $dbForProject->find(Database::METADATA, []);
foreach ($collections as $collection) {
if ($commit) {
$dbForProject->deleteCollection($collection->getId());
$dbForConsole->deleteCachedCollection($collection->getId());
}
Console::info('--Deleting collection (' . $collection->getId() . ') project no (' . $project->getInternalId() . ')');
}
}
if ($commit) {
$dbForConsole->deleteDocument('projects', $project->getId());
$dbForConsole->deleteCachedDocument('projects', $project->getId());
if ($dbForProject->exists($dbForProject->getDefaultDatabase(), Database::METADATA)) {
try {
$dbForProject->deleteCollection(Database::METADATA);
$dbForProject->deleteCachedCollection(Database::METADATA);
} catch (\Throwable $th) {
Console::warning('Metadata collection does not exist');
}
}
}
Console::info('--Deleting project no (' . $project->getInternalId() . ')');
$orphans++;
} catch (\Throwable $th) {
Console::error('Error: ' . $th->getMessage() . ' ' . $th->getTraceAsString());
} finally {
$pools
->get($db)
@@ -110,6 +156,6 @@ class DeleteOrphanedProjects extends Action
$count = $count + $sum;
}
Console::log('Iterated through ' . $count - 1 . '/' . $totalProjects . ' projects found ' . $orphans . ' orphans');
Console::log('Iterated through ' . $count - 1 . '/' . $totalProjects . ' projects found ' . $orphans - 1 . ' orphans');
}
}
src/Appwrite/Platform/Tasks/GetMigrationStats.php
+187
View File
@@ -0,0 +1,187 @@
<?php
namespace Appwrite\Platform\Tasks;
use Exception;
use League\Csv\CannotInsertRecord;
use Utopia\App;
use Utopia\Platform\Action;
use Utopia\Cache\Cache;
use Utopia\CLI\Console;
use Utopia\Database\Database;
use Utopia\Database\Query;
use League\Csv\Writer;
use PHPMailer\PHPMailer\PHPMailer;
use Utopia\Pools\Group;
use Utopia\Registry\Registry;
class GetMigrationStats extends Action
{
/*
* Csv cols headers
*/
private array $columns = [
'Project ID',
'$id',
'$createdAt',
'status',
'stage',
'source'
];
protected string $directory = '/usr/local';
protected string $path;
protected string $date;
public static function getName(): string
{
return 'get-migration-stats';
}
public function __construct()
{
$this
->desc('Get stats for projects')
->inject('pools')
->inject('cache')
->inject('dbForConsole')
->inject('register')
->callback(function (Group $pools, Cache $cache, Database $dbForConsole, Registry $register) {
$this->action($pools, $cache, $dbForConsole, $register);
});
}
/**
* @throws \Utopia\Exception
* @throws CannotInsertRecord
*/
public function action(Group $pools, Cache $cache, Database $dbForConsole, Registry $register): void
{
//docker compose exec -t appwrite get-migration-stats
Console::title('Migration stats calculation V1');
Console::success(APP_NAME . ' Migration stats calculation has started');
/* Initialise new Utopia app */
$app = new App('UTC');
$console = $app->getResource('console');
/** CSV stuff */
$this->date = date('Y-m-d');
$this->path = "{$this->directory}/migration_stats_{$this->date}.csv";
$csv = Writer::createFromPath($this->path, 'w');
$csv->insertOne($this->columns);
/** Database connections */
$totalProjects = $dbForConsole->count('projects');
Console::success("Found a total of: {$totalProjects} projects");
$projects = [$console];
$count = 0;
$limit = 100;
$sum = 100;
$offset = 0;
while (!empty($projects)) {
foreach ($projects as $project) {
/**
* Skip user projects with id 'console'
*/
if ($project->getId() === 'console') {
continue;
}
Console::info("Getting stats for {$project->getId()}");
try {
$db = $project->getAttribute('database');
$adapter = $pools
->get($db)
->pop()
->getResource();
$dbForProject = new Database($adapter, $cache);
$dbForProject->setDefaultDatabase('appwrite');
$dbForProject->setNamespace('_' . $project->getInternalId());
/** Get Project ID */
$stats['Project ID'] = $project->getId();
/** Get Migration details */
$migrations = $dbForProject->find('migrations', [
Query::limit(500)
]);
$migrations = array_map(function ($migration) use ($project) {
return [
$project->getId(),
$migration->getAttribute('$id'),
$migration->getAttribute('$createdAt'),
$migration->getAttribute('status'),
$migration->getAttribute('stage'),
$migration->getAttribute('source'),
];
}, $migrations);
if (!empty($migrations)) {
$csv->insertAll($migrations);
}
} catch (\Throwable $th) {
Console::error('Failed on project ("' . $project->getId() . '") with error on File: ' . $th->getFile() . ' line no: ' . $th->getline() . ' with message: ' . $th->getMessage());
} finally {
$pools
->get($db)
->reclaim();
}
}
$sum = \count($projects);
$projects = $dbForConsole->find('projects', [
Query::limit($limit),
Query::offset($offset),
]);
$offset = $offset + $limit;
$count = $count + $sum;
}
Console::log('Iterated through ' . $count - 1 . '/' . $totalProjects . ' projects...');
$pools
->get('console')
->reclaim();
/** @var PHPMailer $mail */
$mail = $register->get('smtp');
$mail->clearAddresses();
$mail->clearAllRecipients();
$mail->clearReplyTos();
$mail->clearAttachments();
$mail->clearBCCs();
$mail->clearCCs();
try {
/** Addresses */
$mail->setFrom(App::getEnv('_APP_SYSTEM_EMAIL_ADDRESS', APP_EMAIL_TEAM), 'Appwrite Cloud Hamster');
$recipients = explode(',', App::getEnv('_APP_USERS_STATS_RECIPIENTS', ''));
foreach ($recipients as $recipient) {
$mail->addAddress($recipient);
}
/** Attachments */
$mail->addAttachment($this->path);
/** Content */
$mail->Subject = "Migration Report for {$this->date}";
$mail->Body = "Please find the migration report atttached";
$mail->send();
Console::success('Email has been sent!');
} catch (Exception $e) {
Console::error("Message could not be sent. Mailer Error: {$mail->ErrorInfo}");
}
}
}
src/Appwrite/Platform/Tasks/Hamster.php
+3
View File
@@ -149,6 +149,9 @@ class Hamster extends Action
/** Get Total Teams */
$statsPerProject['custom_teams'] = $dbForProject->count('teams', [], APP_LIMIT_COUNT);
/** Get Total Migrations */
$statsPerProject['custom_migrations'] = $dbForProject->count('migrations', [], APP_LIMIT_COUNT);
/** Get Total Members */
$teamInternalId = $project->getAttribute('teamInternalId', null);
if ($teamInternalId) {
src/Appwrite/Platform/Tasks/Migrate.php
+6 -2
View File
@@ -11,6 +11,7 @@ use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Query;
use Utopia\Database\Validator\Authorization;
use Utopia\Registry\Registry;
use Utopia\Validator\Text;
class Migrate extends Action
@@ -29,7 +30,8 @@ class Migrate extends Action
->inject('cache')
->inject('dbForConsole')
->inject('getProjectDB')
->callback(fn ($version, $cache, $dbForConsole, $getProjectDB) => $this->action($version, $cache, $dbForConsole, $getProjectDB));
->inject('register')
->callback(fn ($version, $cache, $dbForConsole, $getProjectDB, Registry $register) => $this->action($version, $cache, $dbForConsole, $getProjectDB, $register));
}
private function clearProjectsCache(Cache $cache, Document $project)
@@ -41,7 +43,7 @@ class Migrate extends Action
}
}
public function action(string $version, Cache $cache, Database $dbForConsole, callable $getProjectDB)
public function action(string $version, Cache $cache, Database $dbForConsole, callable $getProjectDB, Registry $register)
{
Authorization::disable();
if (!array_key_exists($version, Migration::$versions)) {
@@ -89,9 +91,11 @@ class Migrate extends Action
try {
// TODO: Iterate through all project DBs
/** @var Database $projectDB */
$projectDB = $getProjectDB($project);
$migration
->setProject($project, $projectDB, $dbForConsole)
->setPDO($register->get('db', true))
->execute();
} catch (\Throwable $th) {
Console::error('Failed to update project ("' . $project->getId() . '") version with error: ' . $th->getMessage());
src/Appwrite/Platform/Tasks/PatchRecreateRepositoriesDocuments.php
+169
View File
@@ -0,0 +1,169 @@
<?php
namespace Appwrite\Platform\Tasks;
use Utopia\Platform\Action;
use Utopia\CLI\Console;
use Utopia\Database\Database;
use Utopia\Database\Document;
use Utopia\Database\Helpers\ID;
use Utopia\Database\Helpers\Permission;
use Utopia\Database\Helpers\Role;
use Utopia\Database\Query;
use Utopia\Validator\Text;
class PatchRecreateRepositoriesDocuments extends Action
{
public static function getName(): string
{
return 'patch-recreate-repositories-documents';
}
public function __construct()
{
$this
->desc('Recreate missing repositories in consoleDB from projectDBs. They can be missing if you used Appwrite 1.4.10 or 1.4.11, and deleted a function.')
->param('after', '', new Text(36), 'After cursor', true)
->param('projectId', '', new Text(36), 'Select project to validate', true)
->inject('dbForConsole')
->inject('getProjectDB')
->callback(fn ($after, $projectId, $dbForConsole, $getProjectDB) => $this->action($after, $projectId, $dbForConsole, $getProjectDB));
}
public function action($after, $projectId, Database $dbForConsole, callable $getProjectDB): void
{
Console::info("Starting the patch");
$startTime = microtime(true);
if (!empty($projectId)) {
try {
$project = $dbForConsole->getDocument('projects', $projectId);
$dbForProject = call_user_func($getProjectDB, $project);
$this->recreateRepositories($dbForConsole, $dbForProject, $project);
} catch (\Throwable $th) {
Console::error("Unexpected error occured with Project ID {$projectId}");
Console::error('[Error] Type: ' . get_class($th));
Console::error('[Error] Message: ' . $th->getMessage());
Console::error('[Error] File: ' . $th->getFile());
Console::error('[Error] Line: ' . $th->getLine());
}
} else {
$queries = [];
if (!empty($after)) {
Console::info("Iterating remaining projects after project with ID {$after}");
$project = $dbForConsole->getDocument('projects', $after);
$queries = [Query::cursorAfter($project)];
} else {
Console::info("Iterating all projects");
}
$this->foreachDocument($dbForConsole, 'projects', $queries, function (Document $project) use ($getProjectDB, $dbForConsole) {
$projectId = $project->getId();
try {
$dbForProject = call_user_func($getProjectDB, $project);
$this->recreateRepositories($dbForConsole, $dbForProject, $project);
} catch (\Throwable $th) {
Console::error("Unexpected error occured with Project ID {$projectId}");
Console::error('[Error] Type: ' . get_class($th));
Console::error('[Error] Message: ' . $th->getMessage());
Console::error('[Error] File: ' . $th->getFile());
Console::error('[Error] Line: ' . $th->getLine());
}
});
}
$endTime = microtime(true);
$timeTaken = $endTime - $startTime;
$hours = (int)($timeTaken / 3600);
$timeTaken -= $hours * 3600;
$minutes = (int)($timeTaken / 60);
$timeTaken -= $minutes * 60;
$seconds = (int)$timeTaken;
$milliseconds = ($timeTaken - $seconds) * 1000;
Console::info("Recreate patch completed in $hours h, $minutes m, $seconds s, $milliseconds mis ( total $timeTaken milliseconds)");
}
protected function foreachDocument(Database $database, string $collection, array $queries = [], callable $callback = null): void
{
$limit = 1000;
$results = [];
$sum = $limit;
$latestDocument = null;
while ($sum === $limit) {
$newQueries = $queries;
if ($latestDocument != null) {
array_unshift($newQueries, Query::cursorAfter($latestDocument));
}
$newQueries[] = Query::limit($limit);
$results = $database->find($collection, $newQueries);
if (empty($results)) {
return;
}
$sum = count($results);
foreach ($results as $document) {
if (is_callable($callback)) {
$callback($document);
}
}
$latestDocument = $results[array_key_last($results)];
}
}
public function recreateRepositories(Database $dbForConsole, Database $dbForProject, Document $project): void
{
$projectId = $project->getId();
Console::log("Running patch for project {$projectId}");
$this->foreachDocument($dbForProject, 'functions', [], function (Document $function) use ($dbForProject, $dbForConsole, $project) {
$isConnected = !empty($function->getAttribute('providerRepositoryId', ''));
if ($isConnected) {
$repository = $dbForConsole->getDocument('repositories', $function->getAttribute('repositoryId', ''));
if ($repository->isEmpty()) {
$projectId = $project->getId();
$functionId = $function->getId();
Console::success("Recreating repositories document for project ID {$projectId}, function ID {$functionId}");
$repository = $dbForConsole->createDocument('repositories', new Document([
'$id' => ID::unique(),
'$permissions' => [
Permission::read(Role::any()),
Permission::update(Role::any()),
Permission::delete(Role::any()),
],
'installationId' => $function->getAttribute('installationId', ''),
'installationInternalId' => $function->getAttribute('installationInternalId', ''),
'projectId' => $project->getId(),
'projectInternalId' => $project->getInternalId(),
'providerRepositoryId' => $function->getAttribute('providerRepositoryId', ''),
'resourceId' => $function->getId(),
'resourceInternalId' => $function->getInternalId(),
'resourceType' => 'function',
'providerPullRequestIds' => []
]));
$function = $dbForProject->updateDocument('functions', $function->getId(), $function
->setAttribute('repositoryId', $repository->getId())
->setAttribute('repositoryInternalId', $repository->getInternalId()));
$this->foreachDocument($dbForProject, 'deployments', [
Query::equal('resourceInternalId', [$function->getInternalId()]),
Query::equal('resourceType', ['functions'])
], function (Document $deployment) use ($dbForProject, $repository) {
$dbForProject->updateDocument('deployments', $deployment->getId(), $deployment
->setAttribute('repositoryId', $repository->getId())
->setAttribute('repositoryInternalId', $repository->getInternalId()));
});
}
}
});
}
}
src/Appwrite/Platform/Workers/Deletes.php
+34 -2
View File
@@ -45,7 +45,7 @@ class Deletes extends Action
->inject('getFunctionsDevice')
->inject('getBuildsDevice')
->inject('getCacheDevice')
->callback(fn($message, $dbForConsole, callable $getProjectDB, callable $getFilesDevice, callable $getFunctionsDevice, callable $getBuildsDevice, callable $getCacheDevice) => $this->action($message, $dbForConsole, $getProjectDB, $getFilesDevice, $getFunctionsDevice, $getBuildsDevice, $getCacheDevice));
->callback(fn ($message, $dbForConsole, callable $getProjectDB, callable $getFilesDevice, callable $getFunctionsDevice, callable $getBuildsDevice, callable $getCacheDevice) => $this->action($message, $dbForConsole, $getProjectDB, $getFilesDevice, $getFunctionsDevice, $getBuildsDevice, $getCacheDevice));
}
/**
@@ -452,6 +452,21 @@ class Deletes extends Action
Query::equal('projectInternalId', [$projectInternalId])
], $dbForConsole);
// Delete VCS Installations
$this->deleteByGroup('installations', [
Query::equal('projectInternalId', [$projectInternalId])
], $dbForConsole);
// Delete VCS Repositories
$this->deleteByGroup('repositories', [
Query::equal('projectInternalId', [$projectInternalId]),
], $dbForConsole);
// Delete VCS commments
$this->deleteByGroup('vcsComments', [
Query::equal('projectInternalId', [$projectInternalId]),
], $dbForConsole);
// Delete metadata tables
try {
$dbForProject->deleteCollection('_metadata');
@@ -690,7 +705,7 @@ class Deletes extends Action
$this->deleteDeploymentFiles($functionsStorage, $document);
});
/**
/**
* Delete builds
*/
Console::info("Deleting builds for function " . $functionId);
@@ -711,6 +726,23 @@ class Deletes extends Action
Query::equal('functionInternalId', [$functionInternalId])
], $dbForProject);
/**
* Delete VCS Repositories and VCS Comments
*/
Console::info("Deleting VCS repositories and comments linked to function " . $functionId);
$this->deleteByGroup('repositories', [
Query::equal('projectInternalId', [$project->getInternalId()]),
Query::equal('resourceInternalId', [$functionInternalId]),
Query::equal('resourceType', ['function']),
], $dbForConsole, function (Document $document) use ($dbForConsole) {
$providerRepositoryId = $document->getAttribute('providerRepositoryId', '');
$projectInternalId = $document->getAttribute('projectInternalId', '');
$this->deleteByGroup('vcsComments', [
Query::equal('providerRepositoryId', [$providerRepositoryId]),
Query::equal('projectInternalId', [$projectInternalId]),
], $dbForConsole);
});
/**
* Request executor to delete all deployment containers
*/
src/Appwrite/Platform/Workers/Webhooks.php
+14 -11
View File
@@ -25,7 +25,7 @@ class Webhooks extends Action
$this
->desc('Webhooks worker')
->inject('message')
->callback(fn($message) => $this->action($message));
->callback(fn ($message) => $this->action($message));
}
/**
@@ -48,7 +48,7 @@ class Webhooks extends Action
foreach ($project->getAttribute('webhooks', []) as $webhook) {
if (array_intersect($webhook->getAttribute('events', []), $events)) {
$this->execute($events, $webhookPayload, $webhook, $user, $project);
$this->execute($events, $webhookPayload, $webhook, $user, $project);
}
}
@@ -78,7 +78,9 @@ class Webhooks extends Action
\curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
\curl_setopt($ch, CURLOPT_POSTFIELDS, $payload);
\curl_setopt($ch, CURLOPT_HEADER, 0);
\curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
\curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
\curl_setopt($ch, CURLOPT_TIMEOUT, 15);
\curl_setopt($ch, CURLOPT_MAXFILESIZE, 5242880);
\curl_setopt($ch, CURLOPT_USERAGENT, \sprintf(
APP_USERAGENT,
App::getEnv('_APP_VERSION', 'UNKNOWN'),
@@ -88,16 +90,17 @@ class Webhooks extends Action
$ch,
CURLOPT_HTTPHEADER,
[
'Content-Type: application/json',
'Content-Length: ' . \strlen($payload),
'X-' . APP_NAME . '-Webhook-Id: ' . $webhook->getId(),
'X-' . APP_NAME . '-Webhook-Events: ' . implode(',', $events),
'X-' . APP_NAME . '-Webhook-Name: ' . $webhook->getAttribute('name', ''),
'X-' . APP_NAME . '-Webhook-User-Id: ' . $user->getId(),
'X-' . APP_NAME . '-Webhook-Project-Id: ' . $project->getId(),
'X-' . APP_NAME . '-Webhook-Signature: ' . $signature,
'Content-Type: application/json',
'Content-Length: ' . \strlen($payload),
'X-' . APP_NAME . '-Webhook-Id: ' . $webhook->getId(),
'X-' . APP_NAME . '-Webhook-Events: ' . implode(',', $events),
'X-' . APP_NAME . '-Webhook-Name: ' . $webhook->getAttribute('name', ''),
'X-' . APP_NAME . '-Webhook-User-Id: ' . $user->getId(),
'X-' . APP_NAME . '-Webhook-Project-Id: ' . $project->getId(),
'X-' . APP_NAME . '-Webhook-Signature: ' . $signature,
]
);
curl_setopt($ch, CURLOPT_MAXREDIRS, 5);
if (!$webhook->getAttribute('security', true)) {
\curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
tests/e2e/Services/Account/AccountBase.php
+26
View File
@@ -945,6 +945,32 @@ trait AccountBase
return $data;
}
/**
* @depends testUpdateAccountVerification
*/
public function testCreateAccountVerificationForVerifiedEmail($data): array
{
$email = $data['email'] ?? '';
$name = $data['name'] ?? '';
$session = $data['session'] ?? '';
/**
* Test for SUCCESS
*/
$response = $this->client->call(Client::METHOD_POST, '/account/verification', array_merge([
'origin' => 'http://localhost',
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_' . $this->getProject()['$id'] . '=' . $session,
]), [
'url' => 'http://localhost/verification',
]);
$this->assertEquals(409, $response['headers']['status-code']);
return $data;
}
/**
* @depends testUpdateAccountVerification
*/
tests/e2e/Services/Account/AccountCustomClientTest.php
+23
View File
@@ -1068,4 +1068,27 @@ class AccountCustomClientTest extends Scope
return $data;
}
/**
* @depends testPhoneVerification
*/
#[Retry(count: 1)]
public function testPhoneVerificationForVerifiedPhone(array $data): array
{
$session = $data['session'] ?? '';
/**
* Test for SUCCESS
*/
$response = $this->client->call(Client::METHOD_POST, '/account/verification/phone', array_merge([
'origin' => 'http://localhost',
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_' . $this->getProject()['$id'] . '=' . $session,
]));
$this->assertEquals(409, $response['headers']['status-code']);
return $data;
}
}
tests/e2e/Services/Databases/DatabasesBase.php
+1 -1
View File
@@ -978,7 +978,7 @@ trait DatabasesBase
]);
$this->assertEquals(400, $badEnum['headers']['status-code']);
$this->assertEquals('Each enum element must not be empty', $badEnum['body']['message']);
$this->assertEquals('Invalid `elements` param: Value must a valid array and Value must be a valid string and at least 1 chars and no longer than 255 chars', $badEnum['body']['message']);
return $data;
}
tests/e2e/Services/Health/HealthCustomServerTest.php
+93
View File
@@ -138,6 +138,15 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/webhooks?threshold=0', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), []);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
@@ -155,6 +164,15 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/logs?threshold=0', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), []);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
@@ -172,6 +190,15 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/certificates?threshold=0', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), []);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
@@ -189,6 +216,15 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/functions?threshold=0', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), []);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
@@ -206,6 +242,15 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/builds?threshold=0', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), []);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
@@ -225,6 +270,18 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/databases', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), [
'name' => 'database_db_main',
'threshold' => '0'
]);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
@@ -242,6 +299,15 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/deletes?threshold=0', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), []);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
@@ -259,6 +325,15 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/mails?threshold=0', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), []);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
@@ -276,6 +351,15 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/messaging?threshold=0', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), []);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
@@ -293,6 +377,15 @@ class HealthCustomServerTest extends Scope
$this->assertIsInt($response['body']['size']);
$this->assertLessThan(100, $response['body']['size']);
/**
* Test for FAILURE
*/
$response = $this->client->call(Client::METHOD_GET, '/health/queue/migrations?threshold=0', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
], $this->getHeaders()), []);
$this->assertEquals(503, $response['headers']['status-code']);
return [];
}
tests/e2e/Services/Projects/ProjectsConsoleClientTest.php
+94
View File
@@ -1606,6 +1606,100 @@ class ProjectsConsoleClientTest extends Scope
$this->assertEquals(false, $response['body']['authPersonalDataCheck']);
}
public function testUpdateProjectServicesAll(): void
{
$team = $this->client->call(Client::METHOD_POST, '/teams', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_console=' . $this->getRoot()['session'],
]), [
'teamId' => ID::unique(),
'name' => 'Project Test',
]);
$this->assertEquals(201, $team['headers']['status-code']);
$this->assertNotEmpty($team['body']['$id']);
$project = $this->client->call(Client::METHOD_POST, '/projects', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_console=' . $this->getRoot()['session'],
]), [
'projectId' => ID::unique(),
'name' => 'Project Test',
'teamId' => $team['body']['$id'],
'region' => 'default'
]);
$this->assertEquals(201, $project['headers']['status-code']);
$this->assertNotEmpty($project['body']['$id']);
$id = $project['body']['$id'];
$response = $this->client->call(Client::METHOD_PATCH, '/projects/' . $id . '/service/all', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_console=' . $this->getRoot()['session'],
]), [
'status' => false,
]);
$this->assertEquals(200, $response['headers']['status-code']);
$this->assertNotEmpty($response['body']['$id']);
$response = $this->client->call(Client::METHOD_GET, '/projects/' . $id, array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_console=' . $this->getRoot()['session'],
]));
$this->assertEquals(200, $response['headers']['status-code']);
$this->assertNotEmpty($response['body']['$id']);
$matches = [];
$pattern = '/serviceStatusFor.*/';
foreach ($response['body'] as $key => $value) {
if (\preg_match($pattern, $key)) {
\var_dump('Matched key: ' . $key);
$matches[$key] = $value;
}
}
foreach ($matches as $value) {
$this->assertFalse($value);
}
$response = $this->client->call(Client::METHOD_PATCH, '/projects/' . $id . '/service/all', array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_console=' . $this->getRoot()['session'],
]), [
'status' => true,
]);
$this->assertEquals(200, $response['headers']['status-code']);
$this->assertNotEmpty($response['body']['$id']);
$response = $this->client->call(Client::METHOD_GET, '/projects/' . $id, array_merge([
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_console=' . $this->getRoot()['session'],
]));
$this->assertEquals(200, $response['headers']['status-code']);
$matches = [];
foreach ($response['body'] as $key => $value) {
if (\preg_match($pattern, $key)) {
$matches[$key] = $value;
}
}
foreach ($matches as $value) {
$this->assertTrue($value);
}
}
public function testUpdateProjectServiceStatusAdmin(): array
{
tests/e2e/Services/Webhooks/WebhooksCustomClientTest.php
+114 -114
View File
@@ -636,6 +636,120 @@ class WebhooksCustomClientTest extends Scope
return $data;
}
/**
* @depends testUpdateAccountPrefs
*/
public function testCreateAccountVerification($data): array
{
$id = $data['id'] ?? '';
$email = $data['email'] ?? '';
$session = $data['session'] ?? '';
$verification = $this->client->call(Client::METHOD_POST, '/account/verification', array_merge([
'origin' => 'http://localhost',
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_' . $this->getProject()['$id'] . '=' . $session,
]), [
'url' => 'http://localhost/verification',
]);
$verificationId = $verification['body']['$id'];
$this->assertEquals(201, $verification['headers']['status-code']);
$this->assertIsArray($verification['body']);
$webhook = $this->getLastRequest();
$signatureKey = $this->getProject()['signatureKey'];
$payload = json_encode($webhook['data']);
$url = $webhook['url'];
$signatureExpected = base64_encode(hash_hmac('sha1', $url . $payload, $signatureKey, true));
$this->assertEquals($webhook['method'], 'POST');
$this->assertEquals($webhook['headers']['Content-Type'], 'application/json');
$this->assertEquals($webhook['headers']['User-Agent'], 'Appwrite-Server vdev. Please report abuse at security@appwrite.io');
$this->assertStringContainsString('users.*', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString('users.*.verification.*', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString('users.*.verification.*.create', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.*.verification.{$verificationId}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.*.verification.{$verificationId}.create", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.*", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.*.create", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.{$verificationId}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.{$verificationId}.create", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Signature'], $signatureExpected);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Id'] ?? '', $this->getProject()['webhookId']);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Project-Id'] ?? '', $this->getProject()['$id']);
$this->assertEquals(empty($webhook['headers']['X-Appwrite-Webhook-User-Id'] ?? ''), ('server' === $this->getSide()));
$this->assertNotEmpty($webhook['data']['$id']);
$this->assertNotEmpty($webhook['data']['userId']);
$this->assertNotEmpty($webhook['data']['secret']);
$this->assertEquals(true, (new DatetimeValidator())->isValid($webhook['data']['expire']));
$data['secret'] = $webhook['data']['secret'];
return $data;
}
/**
* @depends testCreateAccountVerification
*/
public function testUpdateAccountVerification($data): array
{
$id = $data['id'] ?? '';
$email = $data['email'] ?? '';
$session = $data['session'] ?? '';
$secret = $data['secret'] ?? '';
$verification = $this->client->call(Client::METHOD_PUT, '/account/verification', array_merge([
'origin' => 'http://localhost',
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_' . $this->getProject()['$id'] . '=' . $session,
]), [
'userId' => $id,
'secret' => $secret,
]);
$verificationId = $verification['body']['$id'];
$this->assertEquals(200, $verification['headers']['status-code']);
$this->assertIsArray($verification['body']);
$webhook = $this->getLastRequest();
$signatureKey = $this->getProject()['signatureKey'];
$payload = json_encode($webhook['data']);
$url = $webhook['url'];
$signatureExpected = base64_encode(hash_hmac('sha1', $url . $payload, $signatureKey, true));
$this->assertEquals($webhook['method'], 'POST');
$this->assertEquals($webhook['headers']['Content-Type'], 'application/json');
$this->assertEquals($webhook['headers']['User-Agent'], 'Appwrite-Server vdev. Please report abuse at security@appwrite.io');
$this->assertStringContainsString('users.*', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString('users.*.verification.*', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString('users.*.verification.*.update', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.*.verification.{$verificationId}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.*.verification.{$verificationId}.update", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.*", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.*.update", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.{$verificationId}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.{$verificationId}.update", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Signature'], $signatureExpected);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Id'] ?? '', $this->getProject()['webhookId']);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Project-Id'] ?? '', $this->getProject()['$id']);
$this->assertEquals(empty($webhook['headers']['X-Appwrite-Webhook-User-Id'] ?? ''), ('server' === $this->getSide()));
$this->assertNotEmpty($webhook['data']['$id']);
$this->assertNotEmpty($webhook['data']['userId']);
$this->assertNotEmpty($webhook['data']['secret']);
$this->assertEquals(true, (new DatetimeValidator())->isValid($webhook['data']['expire']));
$data['secret'] = $webhook['data']['secret'];
return $data;
}
/**
* @depends testUpdateAccountPrefs
*/
@@ -751,120 +865,6 @@ class WebhooksCustomClientTest extends Scope
return $data;
}
/**
* @depends testUpdateAccountPrefs
*/
public function testCreateAccountVerification($data): array
{
$id = $data['id'] ?? '';
$email = $data['email'] ?? '';
$session = $data['session'] ?? '';
$verification = $this->client->call(Client::METHOD_POST, '/account/verification', array_merge([
'origin' => 'http://localhost',
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_' . $this->getProject()['$id'] . '=' . $session,
]), [
'url' => 'http://localhost/verification',
]);
$verificationId = $verification['body']['$id'];
$this->assertEquals(201, $verification['headers']['status-code']);
$this->assertIsArray($verification['body']);
$webhook = $this->getLastRequest();
$signatureKey = $this->getProject()['signatureKey'];
$payload = json_encode($webhook['data']);
$url = $webhook['url'];
$signatureExpected = base64_encode(hash_hmac('sha1', $url . $payload, $signatureKey, true));
$this->assertEquals($webhook['method'], 'POST');
$this->assertEquals($webhook['headers']['Content-Type'], 'application/json');
$this->assertEquals($webhook['headers']['User-Agent'], 'Appwrite-Server vdev. Please report abuse at security@appwrite.io');
$this->assertStringContainsString('users.*', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString('users.*.verification.*', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString('users.*.verification.*.create', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.*.verification.{$verificationId}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.*.verification.{$verificationId}.create", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.*", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.*.create", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.{$verificationId}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.{$verificationId}.create", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Signature'], $signatureExpected);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Id'] ?? '', $this->getProject()['webhookId']);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Project-Id'] ?? '', $this->getProject()['$id']);
$this->assertEquals(empty($webhook['headers']['X-Appwrite-Webhook-User-Id'] ?? ''), ('server' === $this->getSide()));
$this->assertNotEmpty($webhook['data']['$id']);
$this->assertNotEmpty($webhook['data']['userId']);
$this->assertNotEmpty($webhook['data']['secret']);
$this->assertEquals(true, (new DatetimeValidator())->isValid($webhook['data']['expire']));
$data['secret'] = $webhook['data']['secret'];
return $data;
}
/**
* @depends testCreateAccountVerification
*/
public function testUpdateAccountVerification($data): array
{
$id = $data['id'] ?? '';
$email = $data['email'] ?? '';
$session = $data['session'] ?? '';
$secret = $data['secret'] ?? '';
$verification = $this->client->call(Client::METHOD_PUT, '/account/verification', array_merge([
'origin' => 'http://localhost',
'content-type' => 'application/json',
'x-appwrite-project' => $this->getProject()['$id'],
'cookie' => 'a_session_' . $this->getProject()['$id'] . '=' . $session,
]), [
'userId' => $id,
'secret' => $secret,
]);
$verificationId = $verification['body']['$id'];
$this->assertEquals(200, $verification['headers']['status-code']);
$this->assertIsArray($verification['body']);
$webhook = $this->getLastRequest();
$signatureKey = $this->getProject()['signatureKey'];
$payload = json_encode($webhook['data']);
$url = $webhook['url'];
$signatureExpected = base64_encode(hash_hmac('sha1', $url . $payload, $signatureKey, true));
$this->assertEquals($webhook['method'], 'POST');
$this->assertEquals($webhook['headers']['Content-Type'], 'application/json');
$this->assertEquals($webhook['headers']['User-Agent'], 'Appwrite-Server vdev. Please report abuse at security@appwrite.io');
$this->assertStringContainsString('users.*', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString('users.*.verification.*', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString('users.*.verification.*.update', $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.*.verification.{$verificationId}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.*.verification.{$verificationId}.update", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.*", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.*.update", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.{$verificationId}", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertStringContainsString("users.{$id}.verification.{$verificationId}.update", $webhook['headers']['X-Appwrite-Webhook-Events']);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Signature'], $signatureExpected);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Id'] ?? '', $this->getProject()['webhookId']);
$this->assertEquals($webhook['headers']['X-Appwrite-Webhook-Project-Id'] ?? '', $this->getProject()['$id']);
$this->assertEquals(empty($webhook['headers']['X-Appwrite-Webhook-User-Id'] ?? ''), ('server' === $this->getSide()));
$this->assertNotEmpty($webhook['data']['$id']);
$this->assertNotEmpty($webhook['data']['userId']);
$this->assertNotEmpty($webhook['data']['secret']);
$this->assertEquals(true, (new DatetimeValidator())->isValid($webhook['data']['expire']));
$data['secret'] = $webhook['data']['secret'];
return $data;
}
/**
* @depends testCreateTeamMembership
*/