Assessment scoping / A practical first step
Start with work
that matters.
Bring one important repository, representative tasks and the decision you need to make. Agree scope before access, execution or spend.
Coding-agent performance assessment
Measure how a named agent configuration performs on your work. Identify codebase friction and improvements worth retesting. The readout leads with outcomes, human review and resource use, with static engineering evidence underneath.
Bring to the first conversation
- The decision: which work you want to delegate, and the risk you need to manage.
- Repository context: a system owner, business criticality, revision, languages and known pain points.
- Representative work: bugs, features, refactors and follow-up requirements, with acceptance criteria.
- Agent configuration: chosen agent/model, harness, access, time budget and provider policy.
- Evaluation controls: an approved execution environment, protected checks, review capacity and cost records.
Download the optional repository inventory template ↓
The template is a local JSON starting point, not a form submission. No repository paths or source are collected by this site.
A sequence with measurable checkpoints.
- Scope: agree representative tasks, success conditions, boundaries, owners and budgets.
- Inspect: scan context, documentation, planning artifacts, supported structure and engineering controls.
- Evaluate: run fresh repeated attempts; retain failures; review outcomes independently.
- Diagnose: connect observed failures to testable codebase or workflow hypotheses.
- Improve: change a targeted constraint and retest matched tasks plus held-out work.
- Decide: agree what can be delegated under human oversight, what needs work and what remains unproven.
Task count, duration, access and commercial terms are agreed during scoping. A six-week engagement is an option, not a promised result or fixed prerequisite. Start with a bounded pilot rather than requiring a whole-estate rollout.
What the first brief should answer.
- Which sampled work succeeded, failed or needed rescue?
- How consistent were the outcomes, and how much review did they need?
- What did accepted work cost, where the data exists?
- Which constraints have direct evidence, and which remain hypotheses?
- What should change next, who owns it, and what would count as improvement?
The CLI's local runner is not a security sandbox or an independent grader. Protected evaluation, provider handling and human review must be agreed separately. Imported evidence is labeled reported; a static scan alone never claims task performance.
Repository Health Review
Available independently. Review reproducibility, verification, documentation and maintainability with engineers. Use the supported static checks and a scoped walkthrough to produce an evidence-backed improvement plan.
Change Safety Review
Available independently. Review verification, release controls, ownership and agreed operational evidence. Production and rollback conclusions depend on the evidence supplied, not on repository file presence.