Skip to content

Pin projects to specific database workers, scale to 4 in dev#11386

Draft
Copilot wants to merge 4 commits into1.8.xfrom
copilot/pin-projects-to-database-worker
Draft

Pin projects to specific database workers, scale to 4 in dev#11386
Copilot wants to merge 4 commits into1.8.xfrom
copilot/pin-projects-to-database-worker

Conversation

Copy link
Contributor

Copilot AI commented Feb 24, 2026

Deterministically route each project's database events to a specific worker using crc32(projectId) % numWorkers, preventing all projects from funneling into a single queue. Scale to 4 workers in dev, default to 1 for self-hosted.

Event routing (src/Appwrite/Event/Database.php)

  • Queue name now includes worker index suffix: {dbHost}-{workerIndex}
  • Worker count controlled by _APP_DATABASE_WORKERS (default: 1 for self-hosted safety)
$numWorkers = (int) System::getEnv('_APP_DATABASE_WORKERS', 1);
$workerIndex = \abs(\crc32($project->getId())) % $numWorkers;
$this->queue = $dsn->getHost() . '-' . $workerIndex;

Infrastructure

  • docker-compose.yml: 4 worker services (appwrite-worker-databases-{0..3}), each with _APP_QUEUE_NAME=database_db_main-{i}. Dev .env sets _APP_DATABASE_WORKERS=4.
  • compose.phtml (self-hosted install): 1 worker by default — self-hosted users can scale by setting _APP_DATABASE_WORKERS and adding worker containers.
  • app/worker.php: Default queue name updated to database_db_main-0.

Health check

  • Default queue param in /health/queue/databases updated to database_db_main-0.
  • DatabasesQueueTest updated accordingly.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits February 24, 2026 05:12
- Hash project ID (crc32) modulo _APP_DATABASE_WORKERS to determine
  which database worker queue to use for each project
- Add _APP_DATABASE_WORKERS env var (default 4)
- Replace single appwrite-worker-databases service with 4 partitioned
  workers (appwrite-worker-databases-0 through 3)
- Update health check endpoint default queue name
- Update install template with 4 database worker services
- Update health check test for new queue naming

Co-authored-by: abnegate <5857008+abnegate@users.noreply.github.com>
Co-authored-by: abnegate <5857008+abnegate@users.noreply.github.com>
Copilot AI changed the title [WIP] Pin projects to specific database worker and scale to four workers Pin projects to specific database workers, scale to 4 by default Feb 24, 2026
Co-authored-by: abnegate <5857008+abnegate@users.noreply.github.com>
Copilot AI changed the title Pin projects to specific database workers, scale to 4 by default Pin projects to specific database workers, scale to 4 in dev Feb 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants