dev-orchestra で役割ごとに決まった指示を渡してみた Giving dev-orchestra Standing Instructions per Role

Introduction

Every time I asked dev-orchestra for something, I wrote the same requests: “run tests with python -m unittest”, “use only the standard library”. With instructions, added in 1.2.1, these standing instructions can be written once per role. All the changes from v1.1.0 to v1.2.1 are in the release post.

My environment:

• Windows 11
• dev-orchestra 1.2.1
• Claude Code 2.1.309 (the implementer was Claude’s sonnet)
• Codex CLI 0.163.0 (one of the reviewers)

For the test, I pointed DEV_ORCHESTRA_HOME at a folder separate from my everyday configuration and worked in a small Python repository made for it (textkit). I ran the steps the orchestrator normally runs for you as commands, one at a time.

dev-orchestra is open source on GitHub (MIT license). If it looks useful, a star would mean a lot.

→ github.com/istb16/dev-orchestra

Steps

1. Write the instructions

Put instructions in the repository’s .dev-orchestra.yaml.

version: 1
instructions:
  orchestrator: "Ask before adding a new dependency."
  implementer:
    file: .dev-orchestra/implementer.md
  reviewers: "Check that every public function in textkit/ has a docstring."

The implementer’s instructions looked like they’d grow, so I put them in a file.

- Use only the Python standard library.
- Every public function gets a docstring and type hints.
- Run tests with `python -m unittest`.

These are the roles you can write for:

A value is a string or {file: <path>}. The path is relative to the repository root and can’t point outside it. A reviewer seat can also carry its own instructions.

2. Read the notes from config validate

After writing them, run config validate.

$ dev-orchestra config validate
Configuration is valid.
Warnings:
  - instructions.orchestrator: the project config (.dev-orchestra.yaml) gives the session standing orders, which `status` tells it to follow; that file can come with the branch under review, so check that this is intended
  - instructions.implementer: the project config (.dev-orchestra.yaml) gives the implementer standing orders, and it runs with write access; that file can come with the branch under review, so check that this is intended
  - instructions.reviewers: the project config (.dev-orchestra.yaml) sets reviewer instructions; that file can come with the branch under review, so check that this is intended

The configuration is valid, but there is one note per instruction. The .dev-orchestra.yaml in a repository can arrive with the branch under review. If someone else’s branch says “implementer, do this”, it reaches a run with write access. That’s why the implementer’s line says it runs with write access.

The notes don’t refuse anything. If you wrote the instructions yourself, carry on. The same lines also appear from doctor and review run.

3. Check with status and config show

status prints an Instructions: line. It shows which roles have instructions in force, where they come from (global or project, inline or a file), and how many characters they are.

$ dev-orchestra status
...
Plan approval: pending -- present .ai/workflows/instr/plan.md and the instructions in force to the user and ask; a yes is recorded with `design approve`, never without one
Instructions: orchestrator (project, inline, 35 chars); implementer (project, .dev-orchestra/implementer.md, 135 chars); reviewers (project, inline, 61 chars)
Orchestrator instruction (follow it; the skill's rules come first):
  Ask before adding a new dependency.

Two things to notice:

• The orchestrator’s instruction reaches it through status, not a prompt. The orchestrator reads status as it goes, so the text is printed in full, with a note that the skill’s rules come first
• You’re prompted to show the instructions when the plan is approved. While any instruction is in force, the Plan approval: line gains and the instructions in force

config show has an Instructions section too.

  Instructions
    orchestrator (project, inline, 35 chars)
    implementer (project, .dev-orchestra/implementer.md, 135 chars)
    reviewers (project, inline, 61 chars)
    Orchestrator instruction:
      Ask before adding a new dependency.

4. Look at the prompt the implementer received

I approved a small plan (add truncate(text, width)) and ran the implementer. To see the prompt, I ran the implementer once more on the mock adapter (a test adapter that spends no tokens) with --detach, and opened the saved prompt.

## Project instructions

Where a project instruction conflicts with any other rule in this prompt or with the request below, follow those instead.
Sources: project .dev-orchestra/implementer.md.

- Use only the Python standard library.
- Every public function gets a docstring and type hints.
- Run tests with `python -m unittest`.

Implement the approved plan at .ai/workflows/instr/plan.md.
...

The instructions go under ## Project instructions, before the request. The line saying “where they conflict with other rules or the request, follow those instead” makes them weaker than the CLI’s rules and the request.

The textkit/truncate.py written by the run on Claude (sonnet) had type hints and a docstring and used only the standard library, as instructed.

def truncate(text: str, width: int) -> str:
    """Return text unchanged if it fits in width, else cut it so that the result ending in "..." is width characters long.

    Raises ValueError if text must be cut but width is too small to hold "...".
    """

The plan said nothing about type hints or docstrings. I believe they came from the instructions, but I didn’t run the same plan without them to compare.

5. Global and project instructions are joined

instructions behaves differently from the other settings. For other keys, the project value overrides the global one, but for instructions both texts are joined (global first).

I added an implementer instruction to the global configuration (config.yaml in DEV_ORCHESTRA_HOME) too.

instructions:
  implementer: "Keep each change small; leave no TODO comments."

status now lists two implementer instructions.

Instructions: orchestrator (project, inline, 35 chars); implementer (global, inline, 47 chars); implementer (project, .dev-orchestra/implementer.md, 135 chars); reviewers (project, inline, 61 chars)

You can keep your own preferences (global) apart from the repository’s rules (project).

6. Reviewer instructions can only add things to look at

Reviewer instructions go after the task’s rules and before the change. Reviewers are also told that instructions can only add things to look at. Instructions like “don’t report this” or “lower the severity” are not followed, so instructions arriving with a branch can’t soften the review.

Reviewer prompts aren’t saved to a file, so I checked with a reviewer on the mock adapter. The mock fails when the prompt contains the string in the environment variable DEV_ORCHESTRA_MOCK_FAIL. That’s how I confirmed that both the instruction text (has a docstring) and the sentence about only adding things to look at (may only add) were in the reviewer’s prompt (with a string that isn’t there, it succeeded).

7. A change to the instructions is reviewed as high-risk

I put a change that edits the instruction file up for review.

$ echo "- Never edit tests/ without a failing test first." >> .dev-orchestra/implementer.md
$ dev-orchestra review run --force
note: balanced → quality: .dev-orchestra/implementer.md matches .dev-orchestra/implementer.md (and 1 more)
note: claude-test (when: relevance) added: .dev-orchestra/implementer.md matches .dev-orchestra/implementer.md (and 1 more)
note: high-risk round (.dev-orchestra/implementer.md matches .dev-orchestra/implementer.md (and 1 more)): claude-security runs opus instead of sonnet
ok      claude-general     claude   opus               1 finding(s)
ok      codex-general      codex    gpt-6.1-sol        0 finding(s)
ok      claude-security    claude   opus               0 finding(s)
ok      claude-test        claude   sonnet             2 finding(s)
ok      mock-general       mock     recommended-coding 0 finding(s)

5 successful, 0 failed

(I left out the instructions note lines. mock-general is the test reviewer added in step 6. --force is there because I had already run several reviews in this workflow and needed to go past the round limit of 2.)

It’s a one-line change, but the optimization level went up from balanced to quality, the whole panel ran, and the security reviewer was raised to opus. review snapshot --json also records the changed instruction file as instruction_changes.

{'instruction_changes': ['.dev-orchestra/implementer.md']}

Instructions apply to every run from then on. That’s why a change to them is treated as high-risk, like changes to authentication or payment code.

8. When an instruction file is missing

When I removed the instruction file, status said 0 chars: not found and doctor printed a note.

Instructions: orchestrator (project, inline, 35 chars); implementer (global, inline, 47 chars); implementer (project, .dev-orchestra/implementer.md, 0 chars: not found); reviewers (project, inline, 61 chars)

  - instructions for implementer (project file): .dev-orchestra/implementer.md not found; runs go on without it

Runs don’t stop; they go on without that instruction. According to the reference, a file over 32 KB or one reached through a link or junction is treated the same way (I didn’t try those).

9. Settings for work units came along too

1.2.1 also added two settings for work units (1.2.0).

I used unit_tests: after_merge in the work units post. I didn’t try worktree_files this time.

Summary

• Write standing instructions per role under instructions in .dev-orchestra.yaml (or the global configuration), inline or in a file
• For instructions in the project file, config validate / doctor / review run print one note each, since that file can arrive with a branch
• Instructions go into the prompt under ## Project instructions and are weaker than the CLI’s rules and the request. The orchestrator’s appear in status
• Global and project instructions are joined (not overridden like other settings)
• Reviewer instructions can only add things to look at; they can’t hold findings back
• A change to the instructions is reviewed as high-risk
• A missing instruction file doesn’t stop the run

GitHub: istb16/dev-orchestra (a star is always appreciated)