chore(backend): upgrade deps and remove redis/rq
This commit is contained in:
@@ -11,9 +11,6 @@ POSTGRES_USER=postgres
|
|||||||
POSTGRES_PASSWORD=postgres
|
POSTGRES_PASSWORD=postgres
|
||||||
POSTGRES_PORT=5432
|
POSTGRES_PORT=5432
|
||||||
|
|
||||||
# --- redis ---
|
|
||||||
REDIS_PORT=6379
|
|
||||||
|
|
||||||
# --- backend settings (see backend/.env.example for full list) ---
|
# --- backend settings (see backend/.env.example for full list) ---
|
||||||
CORS_ORIGINS=http://localhost:3000
|
CORS_ORIGINS=http://localhost:3000
|
||||||
DB_AUTO_MIGRATE=true
|
DB_AUTO_MIGRATE=true
|
||||||
|
|||||||
@@ -16,10 +16,10 @@
|
|||||||
From repo root:
|
From repo root:
|
||||||
- `make setup`: install/sync backend + frontend dependencies.
|
- `make setup`: install/sync backend + frontend dependencies.
|
||||||
- `make check`: CI-equivalent suite (lint, typecheck, tests/coverage, frontend build).
|
- `make check`: CI-equivalent suite (lint, typecheck, tests/coverage, frontend build).
|
||||||
- `docker compose -f compose.yml --env-file .env up -d --build`: run full stack (Postgres + Redis included).
|
- `docker compose -f compose.yml --env-file .env up -d --build`: run full stack.
|
||||||
|
|
||||||
Fast local dev:
|
Fast local dev:
|
||||||
- `docker compose -f compose.yml --env-file .env up -d db redis`
|
- `docker compose -f compose.yml --env-file .env up -d db`
|
||||||
- Backend: `cd backend && uv sync --extra dev && uv run uvicorn app.main:app --reload --port 8000`
|
- Backend: `cd backend && uv sync --extra dev && uv run uvicorn app.main:app --reload --port 8000`
|
||||||
- Frontend: `cd frontend && npm install && npm run dev`
|
- Frontend: `cd frontend && npm install && npm run dev`
|
||||||
- API client: `make api-gen` (backend must be running on `127.0.0.1:8000`).
|
- API client: `make api-gen` (backend must be running on `127.0.0.1:8000`).
|
||||||
|
|||||||
17
README.md
17
README.md
@@ -13,7 +13,7 @@ OpenClaw Mission Control is under active development. Expect breaking changes an
|
|||||||
|
|
||||||
- **Frontend:** Next.js app (default http://localhost:3000)
|
- **Frontend:** Next.js app (default http://localhost:3000)
|
||||||
- **Backend:** FastAPI service (default http://localhost:8000)
|
- **Backend:** FastAPI service (default http://localhost:8000)
|
||||||
- **Data:** Postgres + Redis
|
- **Data:** Postgres
|
||||||
- **Gateway integration:** see [`docs/openclaw_gateway_ws.md`](./docs/openclaw_gateway_ws.md)
|
- **Gateway integration:** see [`docs/openclaw_gateway_ws.md`](./docs/openclaw_gateway_ws.md)
|
||||||
|
|
||||||
> Note on auth (Clerk)
|
> Note on auth (Clerk)
|
||||||
@@ -65,13 +65,13 @@ docker compose -f compose.yml --env-file .env logs -f --tail=200
|
|||||||
# Rebuild a single service
|
# Rebuild a single service
|
||||||
docker compose -f compose.yml --env-file .env up -d --build backend
|
docker compose -f compose.yml --env-file .env up -d --build backend
|
||||||
|
|
||||||
# Reset data (DESTRUCTIVE: deletes Postgres/Redis volumes)
|
# Reset data (DESTRUCTIVE: deletes Postgres volume)
|
||||||
docker compose -f compose.yml --env-file .env down -v
|
docker compose -f compose.yml --env-file .env down -v
|
||||||
```
|
```
|
||||||
|
|
||||||
## Quick start (local development)
|
## Quick start (local development)
|
||||||
|
|
||||||
This is the fastest workflow for contributors: run Postgres/Redis via Docker, and run the backend + frontend in dev mode.
|
This is the fastest workflow for contributors: run Postgres via Docker, and run the backend + frontend in dev mode.
|
||||||
|
|
||||||
### Prerequisites
|
### Prerequisites
|
||||||
|
|
||||||
@@ -79,12 +79,12 @@ This is the fastest workflow for contributors: run Postgres/Redis via Docker, an
|
|||||||
- Python **3.12+** + [`uv`](https://github.com/astral-sh/uv)
|
- Python **3.12+** + [`uv`](https://github.com/astral-sh/uv)
|
||||||
- Node.js (recommend 18+) + npm
|
- Node.js (recommend 18+) + npm
|
||||||
|
|
||||||
### 1) Start Postgres + Redis
|
### 1) Start Postgres
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
|
|
||||||
docker compose -f compose.yml --env-file .env up -d db redis
|
docker compose -f compose.yml --env-file .env up -d db
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2) Backend (FastAPI)
|
### 2) Backend (FastAPI)
|
||||||
@@ -103,7 +103,7 @@ uv run uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
|||||||
|
|
||||||
Notes:
|
Notes:
|
||||||
|
|
||||||
- If you run the DB/Redis containers, the backend should use the defaults in `backend/.env` (`localhost:5432` and `localhost:6379`).
|
- If you run the DB container, the backend should use the default in `backend/.env` (`localhost:5432`).
|
||||||
- Database migrations:
|
- Database migrations:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -137,7 +137,6 @@ When running Cypress (`cd frontend && npm run e2e`), make sure `NEXT_PUBLIC_API_
|
|||||||
- **Mission Control backend** exposes a REST API at `/api/v1/*` and also hosts health endpoints (`/healthz`, `/readyz`).
|
- **Mission Control backend** exposes a REST API at `/api/v1/*` and also hosts health endpoints (`/healthz`, `/readyz`).
|
||||||
- **Mission Control frontend** calls the backend via `NEXT_PUBLIC_API_URL`.
|
- **Mission Control frontend** calls the backend via `NEXT_PUBLIC_API_URL`.
|
||||||
- **Postgres** stores boards/tasks/agents/etc.
|
- **Postgres** stores boards/tasks/agents/etc.
|
||||||
- **Redis** is used for background work (RQ).
|
|
||||||
- **OpenClaw Gateway** connectivity is over WebSockets; protocol details live in [`docs/openclaw_gateway_ws.md`](./docs/openclaw_gateway_ws.md).
|
- **OpenClaw Gateway** connectivity is over WebSockets; protocol details live in [`docs/openclaw_gateway_ws.md`](./docs/openclaw_gateway_ws.md).
|
||||||
|
|
||||||
## Common commands
|
## Common commands
|
||||||
@@ -178,7 +177,7 @@ You likely have `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` set (even to a placeholder).
|
|||||||
|
|
||||||
- Remove the `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` line from `frontend/.env.local`, **or** set it to an empty value.
|
- Remove the `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` line from `frontend/.env.local`, **or** set it to an empty value.
|
||||||
|
|
||||||
### Backend can’t connect to Postgres/Redis
|
### Backend can’t connect to Postgres
|
||||||
|
|
||||||
- Confirm containers are up:
|
- Confirm containers are up:
|
||||||
|
|
||||||
@@ -188,7 +187,6 @@ You likely have `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` set (even to a placeholder).
|
|||||||
|
|
||||||
- If you’re running backend locally (not in compose), make sure `backend/.env` points to `localhost`:
|
- If you’re running backend locally (not in compose), make sure `backend/.env` points to `localhost`:
|
||||||
- `DATABASE_URL=postgresql+psycopg://postgres:postgres@localhost:5432/mission_control`
|
- `DATABASE_URL=postgresql+psycopg://postgres:postgres@localhost:5432/mission_control`
|
||||||
- `REDIS_URL=redis://localhost:6379/0`
|
|
||||||
|
|
||||||
### Port already in use
|
### Port already in use
|
||||||
|
|
||||||
@@ -197,7 +195,6 @@ Adjust ports in `.env` (copied from `.env.example`):
|
|||||||
- `FRONTEND_PORT`
|
- `FRONTEND_PORT`
|
||||||
- `BACKEND_PORT`
|
- `BACKEND_PORT`
|
||||||
- `POSTGRES_PORT`
|
- `POSTGRES_PORT`
|
||||||
- `REDIS_PORT`
|
|
||||||
|
|
||||||
## Star History
|
## Star History
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
ENVIRONMENT=dev
|
ENVIRONMENT=dev
|
||||||
LOG_LEVEL=INFO
|
LOG_LEVEL=INFO
|
||||||
DATABASE_URL=postgresql+psycopg://postgres:postgres@localhost:5432/mission_control
|
DATABASE_URL=postgresql+psycopg://postgres:postgres@localhost:5432/mission_control
|
||||||
REDIS_URL=redis://localhost:6379/0
|
|
||||||
CORS_ORIGINS=http://localhost:3000
|
CORS_ORIGINS=http://localhost:3000
|
||||||
BASE_URL=
|
BASE_URL=
|
||||||
|
|
||||||
|
|||||||
@@ -11,16 +11,15 @@ This directory contains the **Mission Control backend API** (FastAPI + SQLModel)
|
|||||||
- Python **3.12+**
|
- Python **3.12+**
|
||||||
- [`uv`](https://github.com/astral-sh/uv) (recommended; used by this repo)
|
- [`uv`](https://github.com/astral-sh/uv) (recommended; used by this repo)
|
||||||
- Postgres (local or Docker)
|
- Postgres (local or Docker)
|
||||||
- Redis (local or Docker)
|
|
||||||
|
|
||||||
## Quick start (local backend + Docker Postgres/Redis)
|
## Quick start (local backend + Docker Postgres)
|
||||||
|
|
||||||
From the repo root:
|
From the repo root:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# start dependencies
|
# start dependencies
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
docker compose -f compose.yml --env-file .env up -d db redis
|
docker compose -f compose.yml --env-file .env up -d db
|
||||||
|
|
||||||
# run backend
|
# run backend
|
||||||
cd backend
|
cd backend
|
||||||
@@ -56,7 +55,6 @@ A starter file exists at `backend/.env.example`.
|
|||||||
- Default: `postgresql+psycopg://postgres:postgres@localhost:5432/openclaw_agency`
|
- Default: `postgresql+psycopg://postgres:postgres@localhost:5432/openclaw_agency`
|
||||||
- Recommended local/dev default (matches `backend/.env.example`):
|
- Recommended local/dev default (matches `backend/.env.example`):
|
||||||
`postgresql+psycopg://postgres:postgres@localhost:5432/mission_control`
|
`postgresql+psycopg://postgres:postgres@localhost:5432/mission_control`
|
||||||
- `REDIS_URL` (default: `redis://localhost:6379/0`)
|
|
||||||
- `CORS_ORIGINS` (comma-separated)
|
- `CORS_ORIGINS` (comma-separated)
|
||||||
- Example: `http://localhost:3000`
|
- Example: `http://localhost:3000`
|
||||||
- `BASE_URL` (optional)
|
- `BASE_URL` (optional)
|
||||||
@@ -153,16 +151,6 @@ uv run python scripts/export_openapi.py
|
|||||||
|
|
||||||
- If backend runs **locally** (not in compose), `DATABASE_URL` should usually point at `localhost`.
|
- If backend runs **locally** (not in compose), `DATABASE_URL` should usually point at `localhost`.
|
||||||
|
|
||||||
### Backend can’t connect to Redis
|
|
||||||
|
|
||||||
- Ensure the Redis container is up:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f compose.yml --env-file .env logs -f --tail=200 redis
|
|
||||||
```
|
|
||||||
|
|
||||||
- Confirm `REDIS_URL=redis://localhost:6379/0` when running backend locally.
|
|
||||||
|
|
||||||
### CORS issues from the frontend
|
### CORS issues from the frontend
|
||||||
|
|
||||||
- Set `CORS_ORIGINS=http://localhost:3000` (or a comma-separated list) in `backend/.env`.
|
- Set `CORS_ORIGINS=http://localhost:3000` (or a comma-separated list) in `backend/.env`.
|
||||||
|
|||||||
@@ -25,7 +25,6 @@ class Settings(BaseSettings):
|
|||||||
|
|
||||||
environment: str = "dev"
|
environment: str = "dev"
|
||||||
database_url: str = "postgresql+psycopg://postgres:postgres@localhost:5432/openclaw_agency"
|
database_url: str = "postgresql+psycopg://postgres:postgres@localhost:5432/openclaw_agency"
|
||||||
redis_url: str = "redis://localhost:6379/0"
|
|
||||||
|
|
||||||
# Clerk auth (auth only; roles stored in DB)
|
# Clerk auth (auth only; roles stored in DB)
|
||||||
clerk_secret_key: str = Field(min_length=1)
|
clerk_secret_key: str = Field(min_length=1)
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ class GatewayConfig:
|
|||||||
|
|
||||||
|
|
||||||
def _build_gateway_url(config: GatewayConfig) -> str:
|
def _build_gateway_url(config: GatewayConfig) -> str:
|
||||||
base_url = (config.url or "").strip()
|
base_url: str = (config.url or "").strip()
|
||||||
if not base_url:
|
if not base_url:
|
||||||
message = "Gateway URL is not configured for this board."
|
message = "Gateway URL is not configured for this board."
|
||||||
raise OpenClawGatewayError(message)
|
raise OpenClawGatewayError(message)
|
||||||
@@ -44,11 +44,11 @@ def _build_gateway_url(config: GatewayConfig) -> str:
|
|||||||
return base_url
|
return base_url
|
||||||
parsed = urlparse(base_url)
|
parsed = urlparse(base_url)
|
||||||
query = urlencode({"token": token})
|
query = urlencode({"token": token})
|
||||||
return urlunparse(parsed._replace(query=query))
|
return str(urlunparse(parsed._replace(query=query)))
|
||||||
|
|
||||||
|
|
||||||
async def _await_response(
|
async def _await_response(
|
||||||
ws: websockets.WebSocketClientProtocol,
|
ws: websockets.ClientConnection,
|
||||||
request_id: str,
|
request_id: str,
|
||||||
) -> object:
|
) -> object:
|
||||||
while True:
|
while True:
|
||||||
@@ -70,7 +70,7 @@ async def _await_response(
|
|||||||
|
|
||||||
|
|
||||||
async def _send_request(
|
async def _send_request(
|
||||||
ws: websockets.WebSocketClientProtocol,
|
ws: websockets.ClientConnection,
|
||||||
method: str,
|
method: str,
|
||||||
params: dict[str, Any] | None,
|
params: dict[str, Any] | None,
|
||||||
) -> object:
|
) -> object:
|
||||||
@@ -102,7 +102,7 @@ def _build_connect_params(config: GatewayConfig) -> dict[str, Any]:
|
|||||||
|
|
||||||
|
|
||||||
async def _ensure_connected(
|
async def _ensure_connected(
|
||||||
ws: websockets.WebSocketClientProtocol,
|
ws: websockets.ClientConnection,
|
||||||
first_message: str | bytes | None,
|
first_message: str | bytes | None,
|
||||||
config: GatewayConfig,
|
config: GatewayConfig,
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|||||||
@@ -1,18 +0,0 @@
|
|||||||
"""RQ queue and Redis connection helpers for background workers."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from redis import Redis
|
|
||||||
from rq import Queue
|
|
||||||
|
|
||||||
from app.core.config import settings
|
|
||||||
|
|
||||||
|
|
||||||
def get_redis() -> Redis:
|
|
||||||
"""Create a Redis client from configured settings."""
|
|
||||||
return Redis.from_url(settings.redis_url)
|
|
||||||
|
|
||||||
|
|
||||||
def get_queue(name: str) -> Queue:
|
|
||||||
"""Return an RQ queue bound to the configured Redis connection."""
|
|
||||||
return Queue(name, connection=get_redis())
|
|
||||||
@@ -12,36 +12,34 @@ name = "openclaw-agency-backend"
|
|||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
requires-python = ">=3.12"
|
requires-python = ">=3.12"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"fastapi==0.128.0",
|
"alembic==1.18.3",
|
||||||
"uvicorn[standard]==0.30.6",
|
"clerk-backend-api==4.2.0",
|
||||||
"sqlmodel==0.0.22",
|
"fastapi==0.128.6",
|
||||||
"sqlalchemy[asyncio]==2.0.34",
|
"fastapi-pagination==0.15.10",
|
||||||
"alembic==1.13.2",
|
"jinja2==3.1.6",
|
||||||
"psycopg[binary]==3.3.2",
|
"psycopg[binary]==3.3.2",
|
||||||
"pydantic-settings==2.5.2",
|
"pydantic-settings==2.12.0",
|
||||||
"python-dotenv==1.0.1",
|
"python-dotenv==1.2.1",
|
||||||
"websockets==12.0",
|
"sqlalchemy[asyncio]==2.0.46",
|
||||||
"rq==1.16.2",
|
"sqlmodel==0.0.32",
|
||||||
"redis==5.1.1",
|
"sse-starlette==3.2.0",
|
||||||
"sse-starlette==2.1.3",
|
"uvicorn[standard]==0.40.0",
|
||||||
"jinja2==3.1.6",
|
"websockets==16.0",
|
||||||
"fastapi-pagination==0.15.9",
|
|
||||||
"clerk-backend-api==1.4.1",
|
|
||||||
]
|
]
|
||||||
|
|
||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
dev = [
|
dev = [
|
||||||
"black==24.10.0",
|
"aiosqlite==0.22.1",
|
||||||
"flake8==7.1.1",
|
"black==26.1.0",
|
||||||
"httpx==0.27.0",
|
"coverage[toml]==7.13.4",
|
||||||
"isort==5.13.2",
|
"flake8==7.3.0",
|
||||||
"mypy==1.11.2",
|
"httpx==0.28.1",
|
||||||
"pytest==8.3.3",
|
"isort==7.0.0",
|
||||||
"pytest-asyncio==0.24.0",
|
"mypy==1.19.1",
|
||||||
"pytest-cov==6.0.0",
|
"pytest==9.0.2",
|
||||||
"coverage[toml]==7.6.10",
|
"pytest-asyncio==1.3.0",
|
||||||
"ruff==0.6.9",
|
"pytest-cov==7.0.0",
|
||||||
"aiosqlite==0.21.0",
|
"ruff==0.15.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
[tool.mypy]
|
[tool.mypy]
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ async def run() -> None:
|
|||||||
from app.models.boards import Board
|
from app.models.boards import Board
|
||||||
from app.models.gateways import Gateway
|
from app.models.gateways import Gateway
|
||||||
from app.models.users import User
|
from app.models.users import User
|
||||||
from app.services.openclaw import GatewayAgentIdentity
|
from app.services.openclaw.shared import GatewayAgentIdentity
|
||||||
|
|
||||||
await init_db()
|
await init_db()
|
||||||
async with async_session_maker() as session:
|
async with async_session_maker() as session:
|
||||||
|
|||||||
@@ -52,7 +52,10 @@ def _parse_args() -> argparse.Namespace:
|
|||||||
async def _run() -> int:
|
async def _run() -> int:
|
||||||
from app.db.session import async_session_maker
|
from app.db.session import async_session_maker
|
||||||
from app.models.gateways import Gateway
|
from app.models.gateways import Gateway
|
||||||
from app.services.openclaw import GatewayTemplateSyncOptions, sync_gateway_templates
|
from app.services.openclaw.provisioning import (
|
||||||
|
GatewayTemplateSyncOptions,
|
||||||
|
sync_gateway_templates,
|
||||||
|
)
|
||||||
|
|
||||||
args = _parse_args()
|
args = _parse_args()
|
||||||
gateway_id = UUID(args.gateway_id)
|
gateway_id = UUID(args.gateway_id)
|
||||||
|
|||||||
790
backend/uv.lock
generated
790
backend/uv.lock
generated
File diff suppressed because it is too large
Load Diff
16
compose.yml
16
compose.yml
@@ -17,18 +17,6 @@ services:
|
|||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 20
|
retries: 20
|
||||||
|
|
||||||
redis:
|
|
||||||
image: redis:7-alpine
|
|
||||||
volumes:
|
|
||||||
- redis_data:/data
|
|
||||||
ports:
|
|
||||||
- "${REDIS_PORT:-6379}:6379"
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "redis-cli", "ping"]
|
|
||||||
interval: 5s
|
|
||||||
timeout: 3s
|
|
||||||
retries: 20
|
|
||||||
|
|
||||||
backend:
|
backend:
|
||||||
build:
|
build:
|
||||||
# Build from repo root so the backend image can include repo-level assets
|
# Build from repo root so the backend image can include repo-level assets
|
||||||
@@ -40,14 +28,11 @@ services:
|
|||||||
environment:
|
environment:
|
||||||
# Override localhost defaults for container networking
|
# Override localhost defaults for container networking
|
||||||
DATABASE_URL: postgresql+psycopg://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-postgres}@db:5432/${POSTGRES_DB:-mission_control}
|
DATABASE_URL: postgresql+psycopg://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-postgres}@db:5432/${POSTGRES_DB:-mission_control}
|
||||||
REDIS_URL: redis://redis:6379/0
|
|
||||||
CORS_ORIGINS: ${CORS_ORIGINS:-http://localhost:3000}
|
CORS_ORIGINS: ${CORS_ORIGINS:-http://localhost:3000}
|
||||||
DB_AUTO_MIGRATE: ${DB_AUTO_MIGRATE:-true}
|
DB_AUTO_MIGRATE: ${DB_AUTO_MIGRATE:-true}
|
||||||
depends_on:
|
depends_on:
|
||||||
db:
|
db:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
redis:
|
|
||||||
condition: service_healthy
|
|
||||||
ports:
|
ports:
|
||||||
- "${BACKEND_PORT:-8000}:8000"
|
- "${BACKEND_PORT:-8000}:8000"
|
||||||
|
|
||||||
@@ -71,4 +56,3 @@ services:
|
|||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
postgres_data:
|
postgres_data:
|
||||||
redis_data:
|
|
||||||
|
|||||||
@@ -8,7 +8,6 @@ At a high level:
|
|||||||
- The **frontend** is a Next.js app used by humans.
|
- The **frontend** is a Next.js app used by humans.
|
||||||
- The **backend** is a FastAPI service that exposes REST endpoints under `/api/v1/*`.
|
- The **backend** is a FastAPI service that exposes REST endpoints under `/api/v1/*`.
|
||||||
- **Postgres** stores core state (boards/tasks/agents/etc.).
|
- **Postgres** stores core state (boards/tasks/agents/etc.).
|
||||||
- **Redis** supports async/background primitives (RQ queue scaffolding exists).
|
|
||||||
|
|
||||||
## Components
|
## Components
|
||||||
|
|
||||||
@@ -20,7 +19,6 @@ flowchart LR
|
|||||||
FE -->|HTTP /api/v1/*| BE[FastAPI Backend :8000]
|
FE -->|HTTP /api/v1/*| BE[FastAPI Backend :8000]
|
||||||
|
|
||||||
BE -->|SQL| PG[(Postgres :5432)]
|
BE -->|SQL| PG[(Postgres :5432)]
|
||||||
BE -->|Redis protocol| R[(Redis :6379)]
|
|
||||||
|
|
||||||
BE -->|WebSocket (optional integration)| GW[OpenClaw Gateway]
|
BE -->|WebSocket (optional integration)| GW[OpenClaw Gateway]
|
||||||
GW --> OC[OpenClaw runtime]
|
GW --> OC[OpenClaw runtime]
|
||||||
@@ -50,8 +48,6 @@ flowchart LR
|
|||||||
- **Postgres**: persistence for boards/tasks/agents/approvals/etc.
|
- **Postgres**: persistence for boards/tasks/agents/approvals/etc.
|
||||||
- Models: `backend/app/models/*`
|
- Models: `backend/app/models/*`
|
||||||
- Migrations: `backend/migrations/*`
|
- Migrations: `backend/migrations/*`
|
||||||
- **Redis**: used for background primitives.
|
|
||||||
- RQ helper: `backend/app/workers/queue.py`
|
|
||||||
|
|
||||||
### Gateway integration (optional)
|
### Gateway integration (optional)
|
||||||
Mission Control can call into an OpenClaw Gateway over WebSockets.
|
Mission Control can call into an OpenClaw Gateway over WebSockets.
|
||||||
@@ -64,7 +60,7 @@ Mission Control can call into an OpenClaw Gateway over WebSockets.
|
|||||||
### UI → API
|
### UI → API
|
||||||
1. Browser loads the Next.js frontend.
|
1. Browser loads the Next.js frontend.
|
||||||
2. Frontend calls backend endpoints under `/api/v1/*`.
|
2. Frontend calls backend endpoints under `/api/v1/*`.
|
||||||
3. Backend reads/writes Postgres and may use Redis depending on the operation.
|
3. Backend reads/writes Postgres.
|
||||||
|
|
||||||
### Auth (Clerk — required for now)
|
### Auth (Clerk — required for now)
|
||||||
- **Frontend** enables Clerk when a publishable key is present/valid.
|
- **Frontend** enables Clerk when a publishable key is present/valid.
|
||||||
@@ -76,11 +72,8 @@ Automation/agents can use the “agent” API surface:
|
|||||||
- Endpoints under `/api/v1/agent/*` (router: `backend/app/api/agent.py`).
|
- Endpoints under `/api/v1/agent/*` (router: `backend/app/api/agent.py`).
|
||||||
- Auth via `X-Agent-Token` (see `backend/app/core/agent_auth.py`, referenced from `backend/app/api/deps.py`).
|
- Auth via `X-Agent-Token` (see `backend/app/core/agent_auth.py`, referenced from `backend/app/api/deps.py`).
|
||||||
|
|
||||||
### Background jobs (RQ / Redis)
|
### Background jobs
|
||||||
The codebase includes RQ/Redis dependencies and a queue helper (`backend/app/workers/queue.py`).
|
There is currently no queue runtime configured in this repo.
|
||||||
If/when background jobs are added, the expected shape is:
|
|
||||||
- API enqueues work to Redis.
|
|
||||||
- A separate RQ worker process executes queued jobs.
|
|
||||||
|
|
||||||
## Key directories
|
## Key directories
|
||||||
|
|
||||||
|
|||||||
@@ -4,14 +4,13 @@ This guide covers how to self-host **OpenClaw Mission Control** using the reposi
|
|||||||
|
|
||||||
> Scope
|
> Scope
|
||||||
> - This is a **dev-friendly self-host** setup intended for local or single-host deployments.
|
> - This is a **dev-friendly self-host** setup intended for local or single-host deployments.
|
||||||
> - For production hardening (TLS, backups, external Postgres/Redis, observability), see **Production notes** below.
|
> - For production hardening (TLS, backups, external Postgres, observability), see **Production notes** below.
|
||||||
|
|
||||||
## What you get
|
## What you get
|
||||||
|
|
||||||
When running Compose, you get:
|
When running Compose, you get:
|
||||||
|
|
||||||
- **Postgres** database (persistent volume)
|
- **Postgres** database (persistent volume)
|
||||||
- **Redis** (persistent volume)
|
|
||||||
- **Backend API** (FastAPI) on `http://localhost:${BACKEND_PORT:-8000}`
|
- **Backend API** (FastAPI) on `http://localhost:${BACKEND_PORT:-8000}`
|
||||||
- Health check: `GET /healthz`
|
- Health check: `GET /healthz`
|
||||||
- **Frontend UI** (Next.js) on `http://localhost:${FRONTEND_PORT:-3000}`
|
- **Frontend UI** (Next.js) on `http://localhost:${FRONTEND_PORT:-3000}`
|
||||||
@@ -61,7 +60,6 @@ curl -I http://localhost:${FRONTEND_PORT:-3000}/
|
|||||||
`compose.yml` defines:
|
`compose.yml` defines:
|
||||||
|
|
||||||
- `db` (Postgres 16)
|
- `db` (Postgres 16)
|
||||||
- `redis` (Redis 7)
|
|
||||||
- `backend` (FastAPI)
|
- `backend` (FastAPI)
|
||||||
- `frontend` (Next.js)
|
- `frontend` (Next.js)
|
||||||
|
|
||||||
@@ -70,7 +68,6 @@ curl -I http://localhost:${FRONTEND_PORT:-3000}/
|
|||||||
By default:
|
By default:
|
||||||
|
|
||||||
- Postgres: `5432` (`POSTGRES_PORT`)
|
- Postgres: `5432` (`POSTGRES_PORT`)
|
||||||
- Redis: `6379` (`REDIS_PORT`)
|
|
||||||
- Backend: `8000` (`BACKEND_PORT`)
|
- Backend: `8000` (`BACKEND_PORT`)
|
||||||
- Frontend: `3000` (`FRONTEND_PORT`)
|
- Frontend: `3000` (`FRONTEND_PORT`)
|
||||||
|
|
||||||
@@ -81,7 +78,6 @@ Ports are sourced from `.env` (passed via `--env-file .env`) and wired into `com
|
|||||||
Compose creates named volumes:
|
Compose creates named volumes:
|
||||||
|
|
||||||
- `postgres_data` → Postgres data directory
|
- `postgres_data` → Postgres data directory
|
||||||
- `redis_data` → Redis data directory
|
|
||||||
|
|
||||||
These persist across `docker compose down`.
|
These persist across `docker compose down`.
|
||||||
|
|
||||||
@@ -100,7 +96,7 @@ docker compose -f compose.yml --env-file .env ...
|
|||||||
|
|
||||||
### Backend env
|
### Backend env
|
||||||
|
|
||||||
The backend container loads `./backend/.env.example` via `env_file` and then overrides DB/Redis URLs for container networking.
|
The backend container loads `./backend/.env.example` via `env_file` and then overrides the DB URL for container networking.
|
||||||
|
|
||||||
If you need backend customization, prefer creating a real `backend/.env` and updating compose to use it (optional improvement).
|
If you need backend customization, prefer creating a real `backend/.env` and updating compose to use it (optional improvement).
|
||||||
|
|
||||||
@@ -174,8 +170,6 @@ docker compose -f compose.yml --env-file .env logs -f --tail=200
|
|||||||
- If the repo doesn’t have `frontend/public`, the Dockerfile should not `COPY public/`.
|
- If the repo doesn’t have `frontend/public`, the Dockerfile should not `COPY public/`.
|
||||||
- **Backend build fails looking for `uv.lock`**
|
- **Backend build fails looking for `uv.lock`**
|
||||||
- If backend build context is repo root, Dockerfile must copy `backend/uv.lock` not `uv.lock`.
|
- If backend build context is repo root, Dockerfile must copy `backend/uv.lock` not `uv.lock`.
|
||||||
- **Redis warning about `vm.overcommit_memory`**
|
|
||||||
- Usually non-fatal for dev; for stability under load, set `vm.overcommit_memory=1` on the host.
|
|
||||||
|
|
||||||
## Reset / start fresh
|
## Reset / start fresh
|
||||||
|
|
||||||
@@ -185,7 +179,7 @@ Safe (keeps volumes/data):
|
|||||||
docker compose -f compose.yml --env-file .env down
|
docker compose -f compose.yml --env-file .env down
|
||||||
```
|
```
|
||||||
|
|
||||||
Destructive (removes volumes; deletes Postgres/Redis data):
|
Destructive (removes volumes; deletes Postgres data):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker compose -f compose.yml --env-file .env down -v
|
docker compose -f compose.yml --env-file .env down -v
|
||||||
@@ -195,7 +189,7 @@ docker compose -f compose.yml --env-file .env down -v
|
|||||||
|
|
||||||
If you’re running this beyond local dev, consider:
|
If you’re running this beyond local dev, consider:
|
||||||
|
|
||||||
- Run Postgres and Redis as managed services (or on separate hosts)
|
- Run Postgres as a managed service (or on a separate host)
|
||||||
- Add TLS termination (reverse proxy)
|
- Add TLS termination (reverse proxy)
|
||||||
- Configure backups for Postgres volume
|
- Configure backups for Postgres volume
|
||||||
- Set explicit resource limits and healthchecks
|
- Set explicit resource limits and healthchecks
|
||||||
|
|||||||
@@ -2,11 +2,11 @@
|
|||||||
|
|
||||||
This document describes **production-ish** deployment patterns for **OpenClaw Mission Control**.
|
This document describes **production-ish** deployment patterns for **OpenClaw Mission Control**.
|
||||||
|
|
||||||
Mission Control is a web app (frontend) + API (backend) + Postgres + Redis. The simplest reliable
|
Mission Control is a web app (frontend) + API (backend) + Postgres. The simplest reliable
|
||||||
baseline is Docker Compose plus a reverse proxy with TLS.
|
baseline is Docker Compose plus a reverse proxy with TLS.
|
||||||
|
|
||||||
> This repo currently ships a developer-friendly `compose.yml`. For real production, you should:
|
> This repo currently ships a developer-friendly `compose.yml`. For real production, you should:
|
||||||
> - put Postgres/Redis on managed services or dedicated hosts when possible
|
> - put Postgres on a managed service or dedicated host when possible
|
||||||
> - terminate TLS at a reverse proxy
|
> - terminate TLS at a reverse proxy
|
||||||
> - set up backups + upgrades
|
> - set up backups + upgrades
|
||||||
> - restrict network exposure (firewall)
|
> - restrict network exposure (firewall)
|
||||||
@@ -33,7 +33,6 @@ On one VM:
|
|||||||
- frontend container (internal port 3000)
|
- frontend container (internal port 3000)
|
||||||
- backend container (internal port 8000)
|
- backend container (internal port 8000)
|
||||||
- Postgres container (internal 5432)
|
- Postgres container (internal 5432)
|
||||||
- Redis container (internal 6379)
|
|
||||||
|
|
||||||
### Ports / firewall
|
### Ports / firewall
|
||||||
|
|
||||||
@@ -44,7 +43,6 @@ Expose to the internet:
|
|||||||
Do **not** expose:
|
Do **not** expose:
|
||||||
|
|
||||||
- Postgres 5432
|
- Postgres 5432
|
||||||
- Redis 6379
|
|
||||||
- backend 8000
|
- backend 8000
|
||||||
- frontend 3000
|
- frontend 3000
|
||||||
|
|
||||||
@@ -160,13 +158,13 @@ The main reason to split is reliability and blast-radius reduction.
|
|||||||
### Option A: 2 hosts
|
### Option A: 2 hosts
|
||||||
|
|
||||||
- Host 1: reverse proxy + frontend + backend
|
- Host 1: reverse proxy + frontend + backend
|
||||||
- Host 2: Postgres + Redis (or managed)
|
- Host 2: Postgres (or managed)
|
||||||
|
|
||||||
### Option B: 3 hosts
|
### Option B: 3 hosts
|
||||||
|
|
||||||
- Host 1: reverse proxy + frontend
|
- Host 1: reverse proxy + frontend
|
||||||
- Host 2: backend
|
- Host 2: backend
|
||||||
- Host 3: Postgres + Redis (or managed)
|
- Host 3: Postgres (or managed)
|
||||||
|
|
||||||
### Networking / security groups
|
### Networking / security groups
|
||||||
|
|
||||||
@@ -175,14 +173,12 @@ Minimum rules:
|
|||||||
- Public internet → reverse proxy host: `80/443`
|
- Public internet → reverse proxy host: `80/443`
|
||||||
- Reverse proxy host → backend host: `8000` (or whatever you publish internally)
|
- Reverse proxy host → backend host: `8000` (or whatever you publish internally)
|
||||||
- Backend host → DB host: `5432`
|
- Backend host → DB host: `5432`
|
||||||
- Backend host → Redis host: `6379`
|
|
||||||
|
|
||||||
Everything else: deny.
|
Everything else: deny.
|
||||||
|
|
||||||
### Configuration considerations
|
### Configuration considerations
|
||||||
|
|
||||||
- `DATABASE_URL` must point to the DB host (not `localhost`).
|
- `DATABASE_URL` must point to the DB host (not `localhost`).
|
||||||
- `REDIS_URL` must point to the Redis host.
|
|
||||||
- `CORS_ORIGINS` must include the public frontend URL.
|
- `CORS_ORIGINS` must include the public frontend URL.
|
||||||
- `NEXT_PUBLIC_API_URL` should be the public API base URL.
|
- `NEXT_PUBLIC_API_URL` should be the public API base URL.
|
||||||
|
|
||||||
@@ -197,7 +193,7 @@ The backend currently runs Alembic migrations on startup (see logs). In multi-ho
|
|||||||
|
|
||||||
- [ ] TLS is enabled, HTTP redirects to HTTPS
|
- [ ] TLS is enabled, HTTP redirects to HTTPS
|
||||||
- [ ] Only 80/443 exposed publicly
|
- [ ] Only 80/443 exposed publicly
|
||||||
- [ ] Postgres/Redis not publicly accessible
|
- [ ] Postgres not publicly accessible
|
||||||
- [ ] Backups tested (restore drill)
|
- [ ] Backups tested (restore drill)
|
||||||
- [ ] Log retention/rotation configured
|
- [ ] Log retention/rotation configured
|
||||||
- [ ] Regular upgrade process (pull latest, rebuild, restart)
|
- [ ] Regular upgrade process (pull latest, rebuild, restart)
|
||||||
@@ -209,4 +205,3 @@ The backend currently runs Alembic migrations on startup (see logs). In multi-ho
|
|||||||
- `NEXT_PUBLIC_API_URL`
|
- `NEXT_PUBLIC_API_URL`
|
||||||
- backend CORS settings (`CORS_ORIGINS`)
|
- backend CORS settings (`CORS_ORIGINS`)
|
||||||
- firewall rules between proxy ↔ backend
|
- firewall rules between proxy ↔ backend
|
||||||
|
|
||||||
|
|||||||
@@ -128,7 +128,7 @@ npm run api:gen # regenerate typed API client via Orval
|
|||||||
|
|
||||||
There is a `frontend/Dockerfile` used by the root `compose.yml`.
|
There is a `frontend/Dockerfile` used by the root `compose.yml`.
|
||||||
|
|
||||||
If you’re working on self-hosting, prefer running compose from the repo root so the backend/db/redis are aligned with the documented ports/env.
|
If you’re working on self-hosting, prefer running compose from the repo root so the backend/db are aligned with the documented ports/env.
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user