Test-Agent Pipeline

Navgold's tests are produced and gated by a pipeline of specialised AI sub-agents, each with a disjoint job and (for the read-only reviewers) no ability to edit code. The design is distilled from how mature Frappe apps test — india-compliance, ERPNext, Press, HRMS, LMS and Helpdesk.

The pipeline

flowchart LR
  P[Plan<br/>+ do-NOT-test list] --> W[Write<br/>frappe-tester]
  W --> R[Quality gate<br/>frappe-test-reviewer]
  R -->|request changes| W
  R --> F[Flow / E2E<br/>frappe-flow-tester]
  F --> S[Source review<br/>frappe-reviewer]
  S --> V[Verify + evidence]

The pipeline is tiered to the change size — a one-line fix gets writer + a quick quality pass; a money-path change gets the whole chain, and the flow-tester is mandatory for anything that pushes to an external system.

The test-quality gate (frappe-test-reviewer)

There is no linter in the Frappe ecosystem that rejects a meaningless test, so this agent is net-new value. It judges whether each test earns its place against a fixed rubric (Q1–Q16), rejecting, among others:

• Tautology / constant tests (assert a literal or membership in a constant set).
• Echo assertions (re-reading a field the test just set).
• Circular expected values (the "expected" produced by the same code path under test).
• Mocking the subject (patching the code under test, so the assertion checks the mock).
• External call mocked but the outbound payload never asserted — a tautology for a price-pusher.
• Raw-float money comparison, absolute dates, one-branch-only config forks, order-dependent tests.

It also protects money-path redundancy (a duplicate-looking guard test that asserts the unsafe path is blocked is defense-in-depth — kept, not cut) and returns a ranked coverage-gap list that becomes the next slice of work.

The flow tester (frappe-flow-tester)

Unit tests prove a function; the flow tester proves the assembled chain does the right thing and produces the right side-effects. Its doctrine (F1–F9) includes:

• Run the real chain; neutralise only the outermost boundary — a config flag or the HTTP seam, never both.
• Force enqueued work to the foreground so the submit actually runs under test.
• Assert the outbound payload (right price / SKU / currency), not just "it was called".
• Assert the persisted side-effects (submission log, snapshot) with a fixed order, not the return value.
• Control time with freeze_time; guard against the frappe.db.commit()-in-code-under-test trap that leaks rows past the per-class rollback.

The skill reviewer (skill-reviewer)

A meta-agent that reviews the skills/agents themselves for bloat (B1–B7): restated defaults, generic filler, redundancy, progressive-disclosure violations. It keeps the toolset lean — every instruction that doesn't change behaviour is a token tax paid on every run.

This is a development-process page; the guarantees these agents enforce are described in the Test Coverage & CI page.