Finished and passed checks
Good: 80% or more. Poor: under 50%.
Requires a task suite to measure.
No coding agent has been measured on this repository yet.
Static readiness is 24 out of 100. The friction listed below is what a task suite would test.
| What we measured | This repo | What good looks like | Reading |
|---|---|---|---|
| Finished and passed checksAttempts that produced a working change and passed every check | Not measured | Good: 80% or more. Poor: under 50%. | Not measured |
| Needed a person to step inRuns where someone had to correct the agent | Not measured | Good: 10% or less. Poor: over 30%. | Not measured |
| Time to review one resultMedian engineer time to check what the agent produced | Not measured | Good: 5 min or less. Poor: over 15 min. | Not measured |
| Agent cost per finished changeAll agent spend, failures included, divided by changes that were accepted | Not measured | Compare with your engineer cost for the same change. | Not measured |
Then run a representative task suite: agree real tasks, acceptance criteria and one agent configuration, and measure.
Each number shows what good looks like.
Definitions are under each card.
Good: 80% or more. Poor: under 50%.
Requires a task suite to measure.
Good: 10% or less. Poor: over 30%.
Requires a task suite to measure.
Good: 5 min or less. Poor: over 15 min.
Requires a task suite to measure.
Good: 80% or more. Poor: under 50%.
Requires a task suite to measure.
Compare with your engineer cost for the same change.
Requires a task suite to measure.
Compare with your team's usual time for the same change.
Requires a task suite to measure.
What we saw in the repository.
Each item has a change and a retest.
0 guidance and 0 context artifacts found. Presence does not establish actual loading, relevance or consistency. Referenced paths: 0 existing / 0 referenced.
Action. Check guidance loading in the chosen agent; verify commands, remove contradictions and route task-specific detail.
Retest. Compare the same tasks with current versus corrected context under matched agent settings.
0 planning artifacts found. PLAN.md is optional; acceptance criteria, correctness and freshness are not inferred from presence.
Action. Make task scope, acceptance criteria, constraints and validation explicit in the team's existing task system.
Retest. Review whether the agent identifies required dependencies and asks about genuine ambiguity before changing code.
Language-aware dependency analysis is unavailable. File count and size do not measure semantic coupling.
Action. Investigate boundary crossings and duplicated change sites using a real failure, not import count alone.
Retest. Make a bounded change and inspect unrelated edits, missed dependencies and regression outcomes.
0 architecture documents and 0 contract artifacts found. Business invariants and cross-file understanding remain untested.
Action. Capture non-obvious domain rules and point to authoritative implementations.
Retest. Ask the agent to trace a real business flow with file/symbol citations; have a maintainer verify it.
Static indexing does not establish whether an agent finds the right implementation and tests.
Action. Add concise module entry points where navigation failures are observed.
Retest. Give a representative issue without file hints; record correct localization and wrong-module detours.
Semantic naming clarity and conflicting business meanings have not been assessed. Short identifiers are not automatically defects.
Action. Review ambiguous domain terms, units and identifiers where concrete confusion is observed.
Retest. Test a task crossing the suspected terminology boundary; verify that meanings and units are preserved.
Test commands can be discovered statically; execution, oracle strength and the agent's verification choices are separate evidence.
Action. Provide reproducible failures and independently reviewed acceptance/regression checks.
Retest. Require the fix to pass protected checks; verify that known incorrect solutions fail them.
0 progress/handoff artifacts found. Successful recovery across sessions has not been tested.
Action. Record remaining work, decisions and verification state in an appropriate durable artifact.
Retest. Resume in a fresh session; check for lost constraints, repeated work and premature completion claims.
A static scan cannot establish whether an agent's first solution makes the next change harder.
Action. Include follow-up requirements in the evaluation task set.
Retest. Extend the first implementation and check retained behaviour, rework and unnecessary duplication.
Repository-visible permission settings do not establish enforced sandboxing or safe runtime behaviour.
Action. Agree isolated execution, data handling and permission boundaries before any task trial.
Retest. Use approved synthetic boundary tests without production credentials; inspect actual enforcement and escalation.
L0 · Unscored. 100% coverage of the versioned static checklist, not coverage of agent performance. Safety and verification remain gates, not offsettable bonus points.
Reproducibility, verification, structure, and repository knowledge.
2/4 dimensions scoredControls that help a change remain reviewable, testable, and bounded.
3/4 dimensions scoredStatic preconditions for supervised coding-agent work.
3/4 dimensions scored7 stable checks scored · 16 not applicable · 0 unavailable. Coverage reports what could be resolved, not how healthy the repository is.
Files, configuration, and available git history.
Setup, build, tests, lint, and type checks.
See the performance section. Static checks do not establish task success.
Deploy, rollback, reliability, and operational controls.
Repository profile. documentation (operator supplied confidence). Portfolio manifest classification; verify with the repository owner.
Knowledge inventory (presence only). Agent guidance: none found. Planning: none found. Documentation: 2 files. Missing PLAN.md is not a failed control; file presence does not establish quality or freshness.
Structure diagnostic. Language-aware dependency analysis not available for this repository; no semantic coupling or naming claim is made.
Decision boundary. Use as a portfolio preflight and remediation baseline, not as proof of autonomous delivery.
Add a workflow that runs the test command on pull_request and push.
Add `make setup` (or the ecosystem equivalent) that works from a clean clone, and name it in AGENTS.md.
Add .devcontainer/devcontainer.json with postCreateCommand running your setup command.
Next level L1 Discoverable. Static evidence identifies the repository's setup, test, and usage entry points; those commands were not executed. Requires ENV-02 >= 5 (is 0).
| Reproducibility | 0 | 10 | ||
|---|---|---|---|---|
| Verification | no conclusive checks | n/a | 10 | |
| Delivery controls | 0 | 7.5 | ||
| Security boundaries | 10 | 7.5 | ||
| Changeability | no conclusive checks | n/a | 7.5 | |
| Agent guidance | 0 | 5 | ||
| System knowledge | 4 | 2.5 | ||
| Work definition | 0 | 2.5 | ||
| Agent controls | no conclusive checks | n/a | 2.5 |
pass partial fail experimental, not counted not applicable
Add a workflow that runs the test command on pull_request and push.
Why it matters DORA's AI capabilities model names strong version control and fast feedback as the amplifiers of AI gains; without CI an agent's PR is unverified. [1]
Add `make setup` (or the ecosystem equivalent) that works from a clean clone, and name it in AGENTS.md.
Why it matters Every agent vendor's guidance opens with "install, run, test commands"; Codex's own docs say many quality issues are really setup issues. [1][2]
Add .devcontainer/devcontainer.json with postCreateCommand running your setup command.
Why it matters Devcontainers are what Copilot, Cursor cloud, and Claude Code cloud consume; EnvBench shows agents build under 30% of environments without one. [1][2]
Add a root AGENTS.md with commands, layout, conventions, and completion criteria. Bridge CLAUDE.md to it (INSTR-02).
Why it matters Two controlled studies found instruction files do not change pass rates, which is why this dimension is Medium; they do change cost and every vendor reads them. [1][2][3]
Convert the bug template to a form with required Steps to reproduce, Expected, Actual, and Version fields.
Why it matters Underspecified issues were 38% of the flaws in the original SWE-bench; human-written requirements tripled agent success in a controlled study. [1][2]
Add Getting started, Usage, and Development sections with one command each.
Why it matters Copilot's coding agent reads the README before coding; documentation reduces API hallucination when it is specific. [1][2]
| CI-06 | Git hooks manager configured | 0 | No git hooks manager. |
| DOCS-03 | Architecture decisions written down | 0 | No architecture document or decision records. |
| DOCS-04 | CONTRIBUTING present | 0 | No CONTRIBUTING file. |
| ENV-05 | Runtime version pinned | 0 | No runtime version pin (.nvmrc, .python-version, .tool-versions, rust-toolchain, go directive, engines). |
| GIT-02 | PR template asks for verification | 0 | No PR template. |
| GIT-03 | CODEOWNERS present | 0 | No CODEOWNERS. |
| STRUCT-08 | No large binaries in the tree | 10 | No tracked files over 5 MiB outside asset directories. |
| SAFE-01 | No committed secrets | No high-confidence secret patterns and no tracked .env with values. |
| CI-02 | Linter configured for the primary language | not applicable to this repository |
| CI-03 | Formatter configured | not applicable to this repository |
| CI-04 | Type checking configured | not applicable to this repository |
| CI-07 | Lint or format enforced in CI | not applicable to this repository |
| DOCS-02 | API contracts in the repository | not applicable to this repository |
| ENV-01 | Every manifest has a tracked lockfile | not applicable to this repository |
| ENV-04 | Agent cloud setup file present and valid | not applicable to this repository |
| GIT-04 | Historical change size is small | data not available: |
| GIT-05 | Repository is maintained | data not available: not a git repository |
| GIT-06 | Conventional commit messages | data not available: |
| INSTR-02 | Claude and AGENTS.md instructions do not diverge | not applicable to this repository |
| INSTR-03 | Instruction files within vendor size bands | not applicable to this repository |
| INSTR-04 | Instruction files state build, test, and lint commands | not applicable to this repository |
| INSTR-05 | Referenced paths exist | not applicable to this repository |
| INSTR-07 | No directory-tree dumps in instruction files | not applicable to this repository |
| INSTR-10 | Local-only agent files are untracked | data not available: |
| SAFE-02 | Secrets excluded from agent reads | not applicable to this repository |
| SAFE-04 | Destructive commands guarded | not applicable to this repository |
| SAFE-05 | Sandbox configured | not applicable to this repository |
| STRUCT-01 | Share of source files over 1,000 lines | not applicable to this repository |
| STRUCT-02 | Repository size tier | not applicable to this repository |
| TOOL-01 | Committed agent settings are valid | not applicable to this repository |
| TOOL-02 | Auto-format or lint hook | not applicable to this repository |
| TOOL-03 | Verification hook | not applicable to this repository |
| TOOL-04 | MCP configuration valid and scoped | not applicable to this repository |
| TOOL-05 | Skills and subagents well-formed | not applicable to this repository |
| TOOL-07 | Build artefacts ignored | not applicable to this repository |
| VERIFY-01 | Test framework configured with test files | not applicable to this repository |
| VERIFY-02 | One documented test command | not applicable to this repository |
| VERIFY-03 | Test-to-source file ratio | not applicable to this repository |
Agent performance has not been measured. This scan identifies observable codebase friction and the tasks needed to test it; it does not predict success.
How to read the numbers. Every figure comes from recorded runs of one agent configuration on one repository revision. Missing evidence is shown as not measured, never estimated. "What good looks like" bands are decision thresholds set by rubric apr/0.1.0, not industry benchmarks or percentiles; a band never authorises unattended merging, and anything that broke existing behaviour or a safety rule overrides every band.
Only repository files and supported static diagnostics were assessed. Task completion, human effort, cost, session recovery and production performance are unmeasured.
No universal autonomy certification, validated percentile or calibrated success prediction. Small and selected task sets limit generalisation. Missing evidence stays unknown. Imported reviewer and independence claims are supplied evidence, not attested facts. Production delivery is not established.