The Frappe Control-Plane & Data Model

This page is the map of what Frappe stores for Doppio Box / Bench Space. It is the data-model reference; the behaviour that reads and writes these doctypes lives in pages 08 (billing), 09 (auth/GitHub) and 12 (Ansible provisioning). Everything here was read off the actual .json schemas and .py controllers under benchspace/bench_space/doctype/ — architecture.md is stale, this is not.

1. Frappe as the single source of truth

Doppio Box is split into two halves:

• Control-plane — this Frappe app (benchspace). It owns intent and record-keeping: who owns which Dev Box, what plan they're on, what the wallet balance is, which template to build, what the agent should do next.
• Data-plane — the per-server Go agent (/opt/devbox/agent) that actually creates Firecracker/Docker VMs, plus the Caddy proxy. The agent has its own SQLite DB but treats Frappe as authoritative.

The two never share a database. They communicate through two narrow seams, both expressed as doctypes:

1. Agent Job — Frappe writes a row ("create this slug"), the agent polls Frappe every ~10s, executes it, and reports status back. This is the command queue.
2. The agent user + API key (Server.agent_user / agent_api_secret) — credentials the agent uses to call back into Frappe's REST API to push container status, stats, and build results.

So a "Dev Box" is really two things kept in sync: a Dev Box document (the truth) and a container/VM on a server (the reality). Scheduler jobs (sync_all_stats, status polls) continuously reconcile reality back onto the document. When in doubt, the document wins.

2. Doctype catalog by domain

Module: Bench Space (benchspace/modules.txt). Permissions revolve around two roles: System Manager (full admin) and Dev Box User (the end customer; auto-assigned to every new User — see §4).

2.1 Core — Dev Box, Server, Agent Job

Dev Box

Purpose: one developer workspace (a VM/container with a Frappe bench + code-server). The central entity.

autoname: field:slug — the document name is the slug.

Controller logic (dev_box.py):

• before_insert → generate unique slug, compute URLs from the Server's domain, default owner_user to session user (unless pool).
• after_insert → enqueue an Agent Job with action create, set status creating.
• Whitelisted lifecycle actions, each guarded by a status check (and a subscription_end_date gate for non-admins): start_box, stop_box, reset_box (blocked for pool boxes), delete_box, send_to_pool (admin-only; expires linked subscriptions, returns box to pool), destroy (admin-only force teardown), claim_for_user (pool → owned, used by billing).
• sync_status / fetch_stats call the agent over HTTP (/containers, /containers/{slug}/stats). _send_agent_task(action) is the helper that writes the Agent Job; for create/reset it injects the owner's GITHUB_TOKEN into env_vars.
• Module-level STATUS_MAP translates agent container states (running→active, exited→stopped…). _apply_status deliberately skips boxes in resetting so polls don't clobber the transient state.
• Module functions replenish_pool, sync_all_dev_boxes, sync_all_stats are the scheduler entry points (see §3).
• has_permission / get_permission_query_conditions: non-admins see only rows where owner_user == user.

Server

Purpose: a physical/virtual host that runs the agent and hosts Dev Boxes.

autoname: prompt (operator types the name, e.g. fc-01).

Controller logic (server.py):

• ping — SSH echo pong health check.
• prepare_server — generates agent_token, creates the agent User (_create_agent_user, a System Manager API user named agent-{name}@benchspace.dev), then enqueues run_prepare_server (long queue, runs Ansible playbook.yml → see page 12).
• deploy_agent — enqueues run_deploy_agent (Ansible deploy-agent.yml).
• _agent_variables builds the Ansible var dict (Frappe URL, API key/secret, token, backend) so the agent is wired to call home.
• Links (dashboard): Dev Box and Ansible Play both link back via server.

Agent Job

Purpose: the command queue between control-plane and agent. One row = one action for the agent to execute.

autoname: hash. Controller is an empty pass — all behaviour lives on the agent side (it polls, executes, and PATCHes status back).

2.2 Templates — DevBox Template, DevBox Template App, App Source, Server Template Image

DevBox Template

Purpose: a reusable definition of a bench stack (versions + apps) that gets baked into a rootfs image and pool.

autoname: field:title.

Controller logic (devbox_template.py):

• before_insert → derive slug via _make_slug (lowercase, non-alnum → -).
• after_insert → _create_server_template_images: auto-create a Server Template Image row for every Active server except the builder server (the builder only builds rootfs, never hosts boxes).
• validate → _validate_single_default: throws if another template is already is_default.
• build_image (admin) → assembles an APPS JSON from each App Source, writes a build_image Agent Job to the configured builder_server, sets status Building.

DevBox Template App (child table, istable: 1)

Purpose: one app entry inside a template.

App Source

Purpose: a reusable Git source for a Frappe app (so templates reference it instead of repeating URLs).

autoname: field:title.

Controller: empty pass.

Server Template Image

Purpose: join record tracking the build state of one template's image on one server (each host needs its own golden snapshot per template).

autoname: hash.

Controller: empty pass (status driven by the agent). Has a "Built" green state.

2.3 Billing — Dev Box Plan, Dev Box Subscription, Wallet Transaction, Doppio Customer

(Behaviour & money flow: see page 08. Here = the data model only.)

Dev Box Plan

Purpose: a purchasable plan (duration + price), or a trial marker.

autoname: field:plan_name. Controller: empty pass.

Dev Box Subscription

Purpose: a user's purchase of a plan that backs one Dev Box for a time window.

autoname: SUB-.#####.

Controller logic (dev_box_subscription.py): before_insert defaults user to session user. Owner-scoped permissions (read-only for Dev Box User; the lifecycle is driven by api/billing.py).

Wallet Transaction (is_submittable: 1)

Purpose: an immutable ledger entry for a customer's prepaid wallet. Running balance is stored, not computed on the fly.

autoname: WTXN-.#####.

Controller logic (wallet_transaction.py):

• validate → reject zero amount.
• before_submit → ending_balance = prev_balance + amount; throws on negative balance (insufficient funds guard). This is the core ledger invariant.
• on_submit → publish_realtime("wallet_updated", …) to the user; send a top-up confirmation email for Top-up.
• on_cancel → throws — ledger entries can never be cancelled (only amended). Owner-scoped read for Dev Box User.

Doppio Customer

Purpose: the billing identity for a user (one per user).

autoname: field:user (so name == user email).

Controller logic (doppio_customer.py): before_insert sets currency = INR if country == "India" else USD. get_balance() reads the latest submitted Wallet Transaction.ending_balance. Owner-scoped permissions.

2.4 Auth / Integration — Login Request, GitHub Token

(Flow & API: see page 09.)

Login Request

Purpose: holds an email OTP for passwordless login.

autoname: hash. Controller: empty pass.

GitHub Token

Purpose: stores a user's GitHub OAuth access token (so Dev Boxes can clone private repos / push).

autoname: field:user, allow_rename: 0 (so name == user; that's why Dev Box looks it up by owner_user).

Controller: empty pass. Permission: owner can read their own (if_owner for role All).

2.5 Provisioning — Ansible Play, Ansible Task

(Runner internals: see page 12.)

Ansible Play

Purpose: a record of one Ansible playbook run against a server (the parent of its tasks).

autoname: autoincrement.

Controller logic (ansible_play.py): on_trash cascade-deletes its Ansible Task rows. Dashboard link: Ansible Task via play.

Ansible Task

Purpose: one task within a play, with its output/result.

autoname: autoincrement.

Controller logic (ansible_task.py): on_update → publish_realtime("ansible_play_update", {"play": …}) so the Desk form streams progress live.

2.6 Settings — Bench Space Settings

Bench Space Settings (issingle: 1)

Purpose: the one global config document for the whole app.

Controller logic (bench_space_settings.py): load_from_db computes ssh_keys_status; whitelisted restore_ssh_keys (re-writes keys from site config to disk — see setup.py) and get_github_app_manifest (returns the GitHub App creation manifest with redirect/callback URLs).

Stale-types note: server.json defines runtime_backend, agent_user, and agent_api_secret, but the auto-generated TYPE_CHECKING block in server.py (lines 16-21) omits all three. The JSON fields are authoritative; the annotation block is out of date.

3. Relationships

The links, in plain English:

• Dev Box → Server (where it runs), User (owner), DevBox Template (its stack).
• Agent Job → Dev Box, Server, DevBox Template (a command targeting them).
• Server ← back-referenced by Dev Box, Ansible Play; → User (its agent user).
• DevBox Template ⇒ owns child rows DevBox Template App ⇒ each → App Source.
• Server Template Image → Server + DevBox Template (per-host build state; one auto-created per Active non-builder server when a template is added).
• Dev Box Subscription → User, Doppio Customer, Dev Box Plan, Dev Box, DevBox Template.
• Doppio Customer → User, Country; one per user.
• Wallet Transaction → Doppio Customer (+ fetched User), plus a dynamic reference_doctype/reference_name (often a Subscription).
• GitHub Token / Login Request → keyed by user email (token) / email (login).
• Ansible Play ⇒ owns Ansible Task rows; play → Server.
• Bench Space Settings (Single) → default_server, builder_server.

3.1 ER diagram

3.2 Domains at a glance

4. hooks.py — how the data model comes alive

App metadata (benchspace/hooks.py:1): app_name="benchspace", app_title="Bench Space", publisher BWH, license agpl-3.0. Version 0.0.1 (__init__.py).

Document events (doc_events)

(Note: the rest of the per-doctype behaviour is implemented as controller methods on the doctypes themselves, not via doc_events.)

Scheduler events (scheduler_events)

Note: dev_box.py also defines sync_all_dev_boxes(), but it is not wired into scheduler_events (confirmed in hooks.py:151-163) — only sync_all_stats and replenish_pool are. Box-status sync is driven by agent callbacks (update_task) and the on-demand sync_status method instead.

Permissions

hooks.py registers permission_query_conditions + has_permission for Dev Box, Doppio Customer, Wallet Transaction, Dev Box Subscription — all four enforce the same rule: System Managers see everything; everyone else sees only their own rows (owner_user/user/customer.user).

Other hooks

• after_migrate = ["benchspace.setup.after_migrate"] — restores SSH keys from site config to disk (skipped in developer mode), since Frappe Cloud filesystems aren't persistent.
• jinja.methods exposes get_landing_css_hash (cache-busting for the landing CSS).
• website_route_rules maps /dashboard/<path> → the dashboard SPA.
• export_python_type_annotations = True, require_type_annotated_api_methods = True.
• No fixtures hook is defined — plans, templates, app sources, and settings are seeded operationally, not shipped as fixtures.

5. Mental model for a newcomer

1. A customer signs up → gets the Dev Box User role → creates a Doppio Customer (billing identity, currency from country).
2. They buy a Dev Box Plan → a Dev Box Subscription is created; payment is either Razorpay or Wallet Transaction debit (page 08).
3. The subscription either claims a pooled Dev Box (claim_for_user) or triggers a fresh create. Either way the box's subscription_end_date is set from the subscription's end_date.
4. The Dev Box writes an Agent Job; the agent on its Server executes it and reports back, flipping status toward active.
5. Templates (DevBox Template + DevBox Template App + App Source) define what stack gets baked; Server Template Image tracks the per-host build; the warm pool is kept full by replenish_pool.
6. Server bring-up runs Ansible, logged as Ansible Play / Ansible Task.
7. Bench Space Settings (Single) holds the global wiring: default/builder servers, pool size, GitHub App, Cloudflare R2, trial duration.