docs: add canonical IA landing + first-pass core pages

2026-02-11 06:15:54 +00:00
parent c3490630a4
commit f1a8ef074d
12 changed files with 368 additions and 0 deletions
--- a/docs/01-overview.md
+++ b/docs/01-overview.md
@@ -0,0 +1,23 @@
+# Overview
+
+Mission Control is the **web UI + HTTP API** for operating OpenClaw. It’s where you manage boards, tasks, agents, approvals, and (optionally) gateway connections.
+
+## Problem statement
+- Provide a single place to coordinate work (boards/tasks) and execute automation (agents) safely.
+
+## Non-goals (first pass)
+- Not a general-purpose project management suite.
+- Not a full observability platform.
+
+## Key concepts (glossary-lite)
+- **Board**: a workspace containing tasks, memory, and agents.
+- **Task**: unit of work on a board; has status and comments.
+- **Agent**: an automated worker that can execute tasks and post evidence.
+- **Gateway**: OpenClaw runtime host that executes tools/skills and runs heartbeats/cron.
+- **Heartbeat**: periodic agent check-in loop for doing incremental work.
+- **Cron job**: scheduled execution (recurring or one-shot) isolated from conversational context.
+
+## Where to go next
+- Want it running? → [Quickstart](02-quickstart.md)
+- Want to contribute? → [Development](03-development.md)
+- Want to understand internals? → [Architecture](05-architecture.md)
--- a/docs/02-quickstart.md
+++ b/docs/02-quickstart.md
@@ -0,0 +1,14 @@
+# Quickstart (self-host with Docker Compose)
+
+This page is a pointer to the canonical quickstart in the repo root README.
+
+- Canonical quickstart: [`README.md#quick-start-self-host-with-docker-compose`](../README.md#quick-start-self-host-with-docker-compose)
+
+## Verify it works
+After `docker compose up`:
+- Backend health: `http://localhost:8000/healthz` returns `{ "ok": true }`
+- Frontend: `http://localhost:3000`
+
+## Common gotchas
+- `NEXT_PUBLIC_API_URL` must be reachable from your browser (host), not just from inside Docker.
+- If you are not configuring Clerk locally, ensure Clerk env vars are **unset/blank** so Clerk stays gated off.
--- a/docs/03-development.md
+++ b/docs/03-development.md
@@ -0,0 +1,23 @@
+# Development
+
+This is the contributor-focused dev workflow for the Mission Control repo.
+
+## Prereqs
+- Docker + Docker Compose
+- Node (for the frontend)
+- Python (for the backend)
+
+## Common commands
+- See `Makefile` for the canonical targets.
+- Self-host stack (dev-ish): follow the [Quickstart](02-quickstart.md).
+
+## Local services
+- Postgres is provided by Compose (`compose.yml`).
+
+## Debugging tips
+- Backend liveness: `GET /healthz`
+- Backend routes live under `/api/v1/*` (see `backend/app/main.py`).
+
+## Next (to flesh out)
+- Document the exact backend/frontend dev commands used by maintainers (once finalized).
+- Add a “how to run tests” pointer to `docs/testing/README.md`.
--- a/docs/04-repo-tour.md
+++ b/docs/04-repo-tour.md
@@ -0,0 +1,35 @@
+# Repo tour
+
+High-level map of the codebase so you can quickly find where to change things.
+
+## Top-level
+- `backend/` — FastAPI backend (API server)
+- `frontend/` — Next.js frontend (web UI)
+- `docs/` — documentation
+- `compose.yml` — local/self-host stack (db + backend + frontend)
+- `scripts/` — helper scripts
+
+## Backend: where to look
+- App entrypoint + router wiring: `backend/app/main.py`
+- Routers: `backend/app/api/*`
+- Settings/config: `backend/app/core/config.py`
+- Auth (Clerk + agent token): `backend/app/core/auth.py`, `backend/app/core/agent_auth.py`
+- Models: `backend/app/models/*`
+- Services/domain logic: `backend/app/services/*`
+
+## Frontend: where to look
+- Routes (App Router): `frontend/src/app/*`
+- Components: `frontend/src/components/*`
+- API utilities: `frontend/src/lib/*` and `frontend/src/api/*`
+- Auth (Clerk gating/wrappers): `frontend/src/auth/*`
+
+## Where to change X
+
+| You want to… | Start here |
+|---|---|
+| Add/modify an API endpoint | `backend/app/api/*` + `backend/app/main.py` |
+| Change auth behavior | `backend/app/core/auth.py` + `frontend/src/auth/*` |
+| Change a UI page | `frontend/src/app/*` |
+| Update Compose topology | `compose.yml` |
+
+Next: see [Architecture](05-architecture.md) for system-level flows.
--- a/docs/05-architecture.md
+++ b/docs/05-architecture.md
@@ -0,0 +1,59 @@
+# Architecture
+
+Mission Control is the **web UI + HTTP API** for operating OpenClaw. It’s where you manage boards, tasks, agents, approvals, and (optionally) gateway connections.
+
+> Auth note: **Clerk is required for now** (current product direction). The codebase includes gating so CI/local can run with placeholders, but real deployments should configure Clerk.
+
+## Components
+
+- **Frontend**: Next.js app used by humans
+  - Location: `frontend/`
+  - Routes/pages: `frontend/src/app/*` (Next.js App Router)
+- **Backend**: FastAPI service exposing REST endpoints
+  - Location: `backend/`
+  - App wiring: `backend/app/main.py`
+  - API prefix: `/api/v1/*`
+- **Database**: Postgres (from `compose.yml`)
+
+## Diagram (conceptual)
+
+```mermaid
+flowchart LR
+  U[User / Browser] -->|HTTP| FE[Next.js Frontend :3000]
+  FE -->|HTTP /api/v1/*| BE[FastAPI Backend :8000]
+
+  BE -->|SQL| PG[(Postgres :5432)]
+
+  BE -->|WebSocket (optional)| GW[OpenClaw Gateway]
+  GW --> OC[OpenClaw runtime]
+```
+
+## Key request/data flows
+
+### UI → API
+1. Browser loads the Next.js frontend.
+2. Frontend calls backend endpoints under `/api/v1/*`.
+3. Backend reads/writes Postgres.
+
+### Auth (Clerk)
+- Frontend enables Clerk when a publishable key is present/valid.
+- Backend verifies Clerk JWTs using **`CLERK_JWKS_URL`**.
+
+See also:
+- Frontend auth gating: `frontend/src/auth/*` (notably `frontend/src/auth/clerkKey.ts`).
+- Backend auth: `backend/app/core/auth.py`.
+
+### Agent access (X-Agent-Token)
+Automation/agents can use the “agent API surface”:
+- Endpoints under `/api/v1/agent/*`.
+- Auth via `X-Agent-Token`.
+
+See: `backend/app/api/agent.py`, `backend/app/core/agent_auth.py`.
+
+## Links to deeper docs
+
+- Existing deep-dive: `docs/architecture/README.md`
+- Deployment: [docs/deployment/README.md](deployment/README.md)
+- Production notes: [docs/production/README.md](production/README.md)
+- Gateway protocol: [docs/openclaw_gateway_ws.md](openclaw_gateway_ws.md)
+
--- a/docs/06-configuration.md
+++ b/docs/06-configuration.md
@@ -0,0 +1,29 @@
+# Configuration
+
+This page documents how Mission Control is configured across local dev, self-host, and production.
+
+## Config sources (first pass)
+
+- Docker Compose uses `compose.yml` plus environment variables.
+- Backend reads env vars (see `backend/app/core/config.py`).
+- Frontend uses Next.js env vars at build/runtime (see `frontend/` plus `compose.yml`).
+
+## Key environment variables
+
+### Frontend
+- `NEXT_PUBLIC_API_URL` — backend base URL reachable from the browser
+- `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` — enables Clerk in the frontend when set
+
+### Backend
+- `DATABASE_URL` — Postgres connection string
+- `CORS_ORIGINS` — comma-separated allowed origins
+- `CLERK_JWKS_URL` — enables Clerk JWT verification on protected routes
+- `DB_AUTO_MIGRATE` — whether to auto-run migrations on startup (see backend docs/config)
+
+## Secrets handling
+- Do not commit secret keys.
+- Prefer `.env` files that are excluded by `.gitignore`.
+
+## Links
+- Deployment notes: [docs/deployment/README.md](deployment/README.md)
+- Production notes: [docs/production/README.md](production/README.md)
--- a/docs/07-api-reference.md
+++ b/docs/07-api-reference.md
@@ -0,0 +1,52 @@
+# API reference (first pass)
+
+This is a **map** of the Mission Control HTTP API surface. It’s intentionally light-weight for the first pass.
+
+## Base
+- Backend service: `backend/app/main.py`
+- API prefix: `/api/v1`
+
+## Auth
+
+### User/browser auth (Clerk)
+- Used for the human UI.
+- Frontend enables Clerk when `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` is set.
+- Backend verifies JWTs when `CLERK_JWKS_URL` is configured.
+
+### Agent auth (X-Agent-Token)
+- Used by automation/agents.
+- Header: `X-Agent-Token: <token>`
+- Agent endpoints live under `/api/v1/agent/*`.
+
+## Endpoint groups (routers)
+Routers are registered in `backend/app/main.py`:
+- `auth_router`
+- `agent_router` (agent surface)
+- `agents_router`
+- `activity_router`
+- `gateway_router`, `gateways_router`
+- `metrics_router`
+- `organizations_router`
+- `souls_directory_router`
+- `board_groups_router`, `board_group_memory_router`
+- `boards_router`, `board_memory_router`, `board_onboarding_router`
+- `approvals_router`
+- `tasks_router`
+- `users_router`
+
+## Examples
+
+### Health
+```bash
+curl -s http://localhost:8000/healthz
+```
+
+### Agent call (example)
+```bash
+curl -s http://localhost:8000/api/v1/agent/boards \
+  -H "X-Agent-Token: $AGENT_TOKEN"
+```
+
+## Next
+- Add a per-router table of key paths once we standardize which endpoints are “public” vs “internal”.
+- If an OpenAPI schema exists/gets added, link it here.
--- a/docs/08-agents-and-skills.md
+++ b/docs/08-agents-and-skills.md
@@ -0,0 +1,27 @@
+# Agents & skills
+
+This page explains the automation model as it appears in Mission Control.
+
+## Agent lifecycle (conceptual)
+- An **agent** checks in to Mission Control (often on a schedule) and posts work results as task comments.
+- In OpenClaw terms, agents can run:
+  - **heartbeats** (periodic loops)
+  - **cron jobs** (scheduled runs; better for exact timing / isolation)
+
+## Heartbeats vs cron
+- Use **heartbeat** for batched checks and context-aware incremental work.
+- Use **cron** for exact timing and isolated, standalone actions.
+
+## Skills (how to think about them)
+- A skill is a packaged workflow/tooling instruction set that agents can follow.
+- Skills typically define:
+  - when to use them
+  - required binaries/services
+  - command patterns
+
+## Where this connects in the repo
+- Gateway protocol: [docs/openclaw_gateway_ws.md](openclaw_gateway_ws.md)
+- Gateway base config: [docs/openclaw_gateway_base_config.md](openclaw_gateway_base_config.md)
+
+## Next
+- Add repo-specific guidance for authoring skills and where they live (once standardized).
--- a/docs/09-ops-runbooks.md
+++ b/docs/09-ops-runbooks.md
@@ -0,0 +1,37 @@
+# Ops / runbooks
+
+This page is the operator/SRE entry point. It intentionally links to existing deeper docs to minimize churn.
+
+## Where to start
+- Deployment: [docs/deployment/README.md](deployment/README.md)
+- Production checklist/notes: [docs/production/README.md](production/README.md)
+- Troubleshooting: [docs/troubleshooting/README.md](troubleshooting/README.md)
+
+## “First 30 minutes” incident checklist
+
+1. **Confirm user impact + scope**
+   - What is broken: UI, API, auth, or gateway integration?
+   - Is it all users or a subset?
+
+2. **Check service health**
+   - Backend: `/healthz` and `/readyz`
+   - Frontend: can it load? does it reach the API?
+
+3. **Check auth (Clerk) configuration**
+   - Frontend: is Clerk enabled unexpectedly? (publishable key set)
+   - Backend: is `CLERK_JWKS_URL` configured correctly?
+
+4. **Check DB connectivity**
+   - Can backend connect to Postgres (`DATABASE_URL`)?
+
+5. **Check logs**
+   - Backend logs for 5xx spikes or auth failures.
+   - Frontend logs for proxy/API URL misconfig.
+
+6. **Stabilize**
+   - Roll back the last change if available.
+   - Temporarily disable optional integrations (gateway) to isolate.
+
+## Backups / restore (placeholder)
+- Define backup cadence and restore steps once production deployment is finalized.
+
--- a/docs/10-troubleshooting.md
+++ b/docs/10-troubleshooting.md
@@ -0,0 +1,22 @@
+# Troubleshooting
+
+This is the high-level troubleshooting entry point (minimal churn).
+
+- Deep-dive troubleshooting: [docs/troubleshooting/README.md](troubleshooting/README.md)
+
+## Quick triage
+
+### Symptom: frontend loads but shows API errors
+- Confirm `NEXT_PUBLIC_API_URL` points to a reachable backend.
+- Check backend `/healthz`.
+
+### Symptom: frontend keeps redirecting / Clerk errors
+- If you are running locally without Clerk, ensure `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` is **unset/blank**.
+- See: [repo README Clerk note](../README.md#note-on-auth-clerk).
+
+### Symptom: backend 5xx
+- Check DB connectivity (`DATABASE_URL`) and migrations.
+- Check backend logs.
+
+## Next
+- Promote the most common issues from `docs/troubleshooting/README.md` into this page once we see repeated incidents.
--- a/docs/11-contributing.md
+++ b/docs/11-contributing.md
@@ -0,0 +1,13 @@
+# Contributing
+
+## How to contribute (first pass)
+- Follow the repo’s existing PR and review conventions.
+- Prefer small PRs.
+- Update docs when behavior changes.
+
+## Adding/changing docs
+- Canonical docs live in `docs/`.
+- Follow the IA in `docs/README.md`.
+
+## Testing expectations
+- See [docs/testing/README.md](testing/README.md).
--- a/docs/README.md
+++ b/docs/README.md
@@ -0,0 +1,34 @@
+# Mission Control docs
+
+This folder is the canonical documentation set for Mission Control.
+
+## Start here (by role)
+
+- **Contributor**: start with [Quickstart](../README.md#quick-start-self-host-with-docker-compose) → [Development](development.md) → [Contributing](contributing.md)
+- **Maintainer**: start with [Architecture](05-architecture.md) → [Repo tour](04-repo-tour.md) → [API reference](07-api-reference.md)
+- **Operator/SRE**: start with [Ops / runbooks](09-ops-runbooks.md) → [Troubleshooting](10-troubleshooting.md)
+
+## Table of contents (IA)
+
+1. [Overview](01-overview.md)
+2. [Quickstart](02-quickstart.md)
+3. [Development](03-development.md)
+4. [Repo tour](04-repo-tour.md)
+5. [Architecture](05-architecture.md)
+6. [Configuration](06-configuration.md)
+7. [API reference](07-api-reference.md)
+8. [Agents & skills](08-agents-and-skills.md)
+9. [Ops / runbooks](09-ops-runbooks.md)
+10. [Troubleshooting](10-troubleshooting.md)
+11. [Contributing](11-contributing.md)
+
+## Existing deep-dive docs
+
+Some deeper docs already exist under `docs/**` directories; the IA pages above link to them where appropriate:
+- `docs/architecture/README.md`
+- `docs/deployment/README.md`
+- `docs/production/README.md`
+- `docs/testing/README.md`
+- `docs/troubleshooting/README.md`
+- `docs/openclaw_gateway_ws.md`
+- `docs/openclaw_gateway_base_config.md`