Finished and passed checks
Good: 80% or more. Poor: under 50%.
Requires a task suite to measure.
No coding agent has been measured on this repository yet.
Static readiness is 84 out of 100. The friction listed below is what a task suite would test.
| What we measured | This repo | What good looks like | Reading |
|---|---|---|---|
| Finished and passed checksAttempts that produced a working change and passed every check | Not measured | Good: 80% or more. Poor: under 50%. | Not measured |
| Needed a person to step inRuns where someone had to correct the agent | Not measured | Good: 10% or less. Poor: over 30%. | Not measured |
| Time to review one resultMedian engineer time to check what the agent produced | Not measured | Good: 5 min or less. Poor: over 15 min. | Not measured |
| Agent cost per finished changeAll agent spend, failures included, divided by changes that were accepted | Not measured | Compare with your engineer cost for the same change. | Not measured |
Then run a representative task suite: agree real tasks, acceptance criteria and one agent configuration, and measure.
Each number shows what good looks like.
Definitions are under each card.
Good: 80% or more. Poor: under 50%.
Requires a task suite to measure.
Good: 10% or less. Poor: over 30%.
Requires a task suite to measure.
Good: 5 min or less. Poor: over 15 min.
Requires a task suite to measure.
Good: 80% or more. Poor: under 50%.
Requires a task suite to measure.
Compare with your engineer cost for the same change.
Requires a task suite to measure.
Compare with your team's usual time for the same change.
Requires a task suite to measure.
What we saw in the repository.
Each item has a change and a retest.
2 guidance and 0 context artifacts found. Presence does not establish actual loading, relevance or consistency. Indexed instruction size: 19 lines, 373 bytes (not a measured context-window load). Referenced paths: 2 existing / 2 referenced.
Evidence AGENTS.md, CLAUDE.md
Action. Check guidance loading in the chosen agent; verify commands, remove contradictions and route task-specific detail.
Retest. Compare the same tasks with current versus corrected context under matched agent settings.
0 planning artifacts found. PLAN.md is optional; acceptance criteria, correctness and freshness are not inferred from presence.
Action. Make task scope, acceptance criteria, constraints and validation explicit in the team's existing task system.
Retest. Review whether the agent identifies required dependencies and asks about genuine ambiguity before changing code.
Language-aware dependency analysis is unavailable. File count and size do not measure semantic coupling.
Action. Investigate boundary crossings and duplicated change sites using a real failure, not import count alone.
Retest. Make a bounded change and inspect unrelated edits, missed dependencies and regression outcomes.
0 architecture documents and 0 contract artifacts found. Business invariants and cross-file understanding remain untested.
Action. Capture non-obvious domain rules and point to authoritative implementations.
Retest. Ask the agent to trace a real business flow with file/symbol citations; have a maintainer verify it.
Static indexing does not establish whether an agent finds the right implementation and tests.
Evidence AGENTS.md, CLAUDE.md
Action. Add concise module entry points where navigation failures are observed.
Retest. Give a representative issue without file hints; record correct localization and wrong-module detours.
Semantic naming clarity and conflicting business meanings have not been assessed. Short identifiers are not automatically defects.
Action. Review ambiguous domain terms, units and identifiers where concrete confusion is observed.
Retest. Test a task crossing the suspected terminology boundary; verify that meanings and units are preserved.
Test commands can be discovered statically; execution, oracle strength and the agent's verification choices are separate evidence.
Evidence package.json, AGENTS.md
Action. Provide reproducible failures and independently reviewed acceptance/regression checks.
Retest. Require the fix to pass protected checks; verify that known incorrect solutions fail them.
0 progress/handoff artifacts found. Successful recovery across sessions has not been tested.
Action. Record remaining work, decisions and verification state in an appropriate durable artifact.
Retest. Resume in a fresh session; check for lost constraints, repeated work and premature completion claims.
A static scan cannot establish whether an agent's first solution makes the next change harder.
Action. Include follow-up requirements in the evaluation task set.
Retest. Extend the first implementation and check retained behaviour, rework and unnecessary duplication.
Repository-visible permission settings do not establish enforced sandboxing or safe runtime behaviour.
Action. Agree isolated execution, data handling and permission boundaries before any task trial.
Retest. Use approved synthetic boundary tests without production credentials; inspect actual enforcement and escalation.
L2 · Controlled. 100% coverage of the versioned static checklist, not coverage of agent performance. Safety and verification remain gates, not offsettable bonus points.
Reproducibility, verification, structure, and repository knowledge.
4/4 dimensions scoredControls that help a change remain reviewable, testable, and bounded.
4/4 dimensions scoredStatic preconditions for supervised coding-agent work.
3/4 dimensions scored21 stable checks scored · 2 not applicable · 0 unavailable. Coverage reports what could be resolved, not how healthy the repository is.
Files, configuration, and available git history.
Setup, build, tests, lint, and type checks.
See the performance section. Static checks do not establish task success.
Deploy, rollback, reliability, and operational controls.
Repository profile. service (operator supplied confidence). Portfolio manifest classification; verify with the repository owner.
Knowledge inventory (presence only). Agent guidance: AGENTS.md, CLAUDE.md. Planning: none found. Documentation: 1 files. Missing PLAN.md is not a failed control; file presence does not establish quality or freshness.
Structure diagnostic. Language-aware dependency analysis not available for this repository; no semantic coupling or naming claim is made.
Decision boundary. Use as a portfolio preflight and remediation baseline, not as proof of autonomous delivery.
Add .devcontainer/devcontainer.json with postCreateCommand running your setup command.
Add the setup file for the agent you use; the evidence names the exact structural issue when one exists.
Convert the bug template to a form with required Steps to reproduce, Expected, Actual, and Version fields.
Next level L3 Prepared. The repository has stronger static preconditions for supervised, bounded coding-agent changes. Requires ENV-03 >= 5 (is 0); GIT-01 >= 5 (is 0).
| Reproducibility | 5 | 10 | ||
|---|---|---|---|---|
| Verification | 10 | 10 | ||
| Delivery controls | 10 | 7.5 | ||
| Security boundaries | 8.5 | 7.5 | ||
| Changeability | 10 | 7.5 | ||
| Agent guidance | 10 | 5 | ||
| System knowledge | 10 | 2.5 | ||
| Work definition | 0 | 2.5 | ||
| Agent controls | no conclusive checks | n/a | 2.5 |
pass partial fail experimental, not counted not applicable
Add .devcontainer/devcontainer.json with postCreateCommand running your setup command.
Why it matters Devcontainers are what Copilot, Cursor cloud, and Claude Code cloud consume; EnvBench shows agents build under 30% of environments without one. [1][2]
Add the setup file for the agent you use; the evidence names the exact structural issue when one exists.
Why it matters Vendor requirement. Copilot silently ignores a mis-named job; Cursor cloud agents need an idempotent install. [1][2]
Convert the bug template to a form with required Steps to reproduce, Expected, Actual, and Version fields.
Why it matters Underspecified issues were 38% of the flaws in the original SWE-bench; human-written requirements tripled agent success in a controlled study. [1][2]
Add to .claude/settings.json: permissions.deny ["Read(./.env)", "Read(./.env.*)"] and a .cursorignore with .env*.
Why it matters Vendor-documented boundary. Reading a secret into context is one tool call away from writing it somewhere. [1][2]
| CI-06 | Git hooks manager configured | 0 | No git hooks manager. |
| CI-07 | Lint or format enforced in CI | 10 | CI runs a linter or a formatter check. |
| DOCS-03 | Architecture decisions written down | 0 | No architecture document or decision records. |
| DOCS-04 | CONTRIBUTING present | 0 | No CONTRIBUTING file. |
| ENV-05 | Runtime version pinned | 0 | No runtime version pin (.nvmrc, .python-version, .tool-versions, rust-toolchain, go directive, engines). |
| GIT-02 | PR template asks for verification | 0 | No PR template. |
| GIT-03 | CODEOWNERS present | 0 | No CODEOWNERS. |
| INSTR-07 | No directory-tree dumps in instruction files | 10 | Instruction files are prose and commands, not tree dumps. |
| STRUCT-08 | No large binaries in the tree | 10 | No tracked files over 5 MiB outside asset directories. |
| TOOL-07 | Build artefacts ignored | 10 | .gitignore covers the ecosystem's build and dependency directories. |
| CI-01 | CI runs tests on pull requests | CI runs a recognised test command on pull requests or pushes. |
| CI-02 | Linter configured for the primary language | A linter is configured for the primary language. |
| CI-03 | Formatter configured | A formatter is configured (or ships with the toolchain). |
| CI-04 | Type checking configured | Strict type checking (or a statically typed toolchain). |
| DOCS-01 | README covers setup, usage, and testing | README has at least two of setup/usage/test sections and a fenced command. |
| ENV-01 | Every manifest has a tracked lockfile | 1 of 1 |
| ENV-02 | One-command setup | An explicit setup entry point exists (make/just target, scripts/setup, devcontainer command, nix). |
| INSTR-01 | Cross-tool instruction file present | AGENTS.md is present; read by 20+ agents. |
| INSTR-02 | Claude and AGENTS.md instructions do not diverge | CLAUDE.md imports or links to AGENTS.md, so the two instruction surfaces cannot silently diverge. |
| INSTR-03 | Instruction files within vendor size bands | Always-on instruction content is under 200 lines and 32 KiB. |
| INSTR-04 | Instruction files state build, test, and lint commands | 3 of 3 |
| INSTR-05 | Referenced paths exist | 2 of 2 |
| SAFE-01 | No committed secrets | No high-confidence secret patterns and no tracked .env with values. |
| STRUCT-01 | Share of source files over 1,000 lines | value 0 (median anchor 0.02, good anchor 0.002); anchors uncalibrated |
| STRUCT-02 | Repository size tier | value 14 (median anchor 500, good anchor 100); anchors uncalibrated |
| VERIFY-01 | Test framework configured with test files | A test framework is configured and test files exist. |
| VERIFY-02 | One documented test command | A test command exists and is written down where an agent reads first. |
| DOCS-02 | API contracts in the repository | not applicable to this repository |
| GIT-04 | Historical change size is small | data not available: |
| GIT-05 | Repository is maintained | data not available: not a git repository |
| GIT-06 | Conventional commit messages | data not available: |
| INSTR-10 | Local-only agent files are untracked | data not available: |
| SAFE-04 | Destructive commands guarded | not applicable to this repository |
| SAFE-05 | Sandbox configured | not applicable to this repository |
| TOOL-01 | Committed agent settings are valid | not applicable to this repository |
| TOOL-02 | Auto-format or lint hook | not applicable to this repository |
| TOOL-03 | Verification hook | not applicable to this repository |
| TOOL-04 | MCP configuration valid and scoped | not applicable to this repository |
| TOOL-05 | Skills and subagents well-formed | not applicable to this repository |
| VERIFY-03 | Test-to-source file ratio | not applicable to this repository |
Agent performance has not been measured. This scan identifies observable codebase friction and the tasks needed to test it; it does not predict success.
How to read the numbers. Every figure comes from recorded runs of one agent configuration on one repository revision. Missing evidence is shown as not measured, never estimated. "What good looks like" bands are decision thresholds set by rubric apr/0.1.0, not industry benchmarks or percentiles; a band never authorises unattended merging, and anything that broke existing behaviour or a safety rule overrides every band.
Only repository files and supported static diagnostics were assessed. Task completion, human effort, cost, session recovery and production performance are unmeasured.
No universal autonomy certification, validated percentile or calibrated success prediction. Small and selected task sets limit generalisation. Missing evidence stays unknown. Imported reviewer and independence claims are supplied evidence, not attested facts. Production delivery is not established.