Claude Code Built Most of a Vim Feature. Anthropic Still Reviews Core Logic.

Crafting seamless user experiences with a passion for headless CMS, Vercel deployments, and Cloudflare optimization. I'm a Full Stack Developer with expertise in building modern web applications that are blazing fast, secure, and scalable. Let's connect and discuss how I can help you elevate your next project!
Claude Code works best when a team can define a task and judge the resulting change. Anthropic's 2025 team interviews describe a Vim implementation with roughly 70% coming from Claude's autonomous work. They also describe engineers staying closely involved when changes touch core business logic.
That distinction makes the report useful for developers. Anthropic documents ten teams using Claude Code, its agentic coding environment, to investigate repositories, implement changes and build internal tools. The associated official case-study article was published on July 24, 2025.
The examples are historical, self-reported accounts. They offer practical starting points for a workflow, with no guarantee that another repository will produce the same results. Current documentation supplies the implementation guidance below. Anthropic's official case study
Choose the handoff before choosing the prompt
The Claude Code product team used asynchronous development for a feature at the edge of its product: Vim key bindings. After several iterations, autonomous work accounted for roughly 70% of the final implementation.
For core business logic, the same team described synchronous collaboration. Engineers supplied more detailed instructions and watched architecture and code quality as changes developed. These are two different ways to organize work around the same tool. Report, pages 5–6
The practical choice depends on how easily a change can be contained and evaluated. A disposable prototype can tolerate experimentation. A change to account authorization needs someone who understands the policy behind the implementation.
I would start by naming the artifact the agent must return. An investigation should produce code references and a testable explanation. An implementation should produce a bounded diff with evidence that the intended behavior works. Those deliverables give the reviewer something concrete to inspect.
The reinforcement learning engineering team supplies a useful qualification. It reported first-attempt success only about one-third of the time on its tasks. Further guidance or manual work was often necessary. Frequent Git checkpoints made those attempts easier to revise or discard. Report, pages 19–20
The one-third figure measures a different outcome from the Vim implementation share. Neither establishes an organization-wide productivity rate. Both accounts include work that remains with an engineer.

A repository investigation is a useful first deliverable
Several teams used Claude Code before writing any code. The API Knowledge team asked which files mattered for a task. Security engineers supplied stack traces and documentation, then asked Claude to follow control flow.
Data infrastructure engineers used dashboard screenshots to investigate a Kubernetes failure. They traced unscheduled pods to exhausted pod IP addresses and obtained commands to address the configuration. This was a specific incident, with specific environmental evidence. It does not justify applying generated infrastructure commands without checking their effects. Report, pages 3–4 and 7
For a first project, keep the investigation smaller than the repository. Here is an original task template, developed from the report's patterns. Replace the placeholders with evidence from your own system.
Task: investigate why a report export sometimes omits records.
Do not edit files yet.
Observed behavior: <reproduction steps, expected result, actual result>
Investigation scope: <relevant directory or entry point>
Trace the calls that read, filter and export the records.
Attach a file path and code location to each factual conclusion.
Separate confirmed facts from hypotheses that still need evidence.
Propose the smallest test that distinguishes those hypotheses.
State its expected result, and identify any missing information.
The result should make the next decision easier: approve a small fix, gather more evidence, or reject the explanation. If the agent proposes a redesign before establishing the defect, narrow the task again.
This pattern also helps onboarding. A new contributor can check a referenced function while learning how it fits into the application. A fluent explanation without code references is harder to challenge.
Give the implementation a check it can run
The product team recommends building a loop in which Claude changes code and runs relevant checks. Security engineers described working through test-driven development, with periodic human direction. The inference team used Claude to add unit tests and explore edge cases.
Test-driven development, or TDD, defines expected behavior through tests before completing the implementation. In an agent workflow, a failing test can provide a concrete target that remains visible across revisions. Report, pages 6–10
After the export investigation has established a cause, the implementation request can become much narrower:
Fix only <the confirmed behavior> using the repository's existing patterns.
Do not add unrelated features.
Acceptance criteria:
1. Reproduce the defect with the existing test framework and record the failure.
2. Make the same test pass after the fix; cover <specific boundary condition>.
3. Run the tests and checks required for the affected area.
4. Return the change summary, commands actually run, results and unverified areas.
Do not delete or weaken tests to obtain a passing result.
Explain any need to change a module outside the agreed scope before doing so.
Hand the result to a human reviewer before a merge decision.
This template specifies a workflow, not runnable project code. Its test command must come from the repository. A Python service and a TypeScript application should not inherit each other's test instructions.
Passing tests still leave review work. A generated test may repeat the implementation's mistaken assumptions. A reviewer should check expected behavior against the requirement, inspect the diff and examine the execution evidence. Current Claude Code guidance likewise emphasizes giving the agent a verification method it can execute. Official best practices
Keep project instructions separate from enforced controls
The report's data infrastructure team documented recurring workflows and improved those instructions after use. RL engineers recorded operational corrections, including the right test runner and avoiding unnecessary directory changes.
The current filename is CLAUDE.md. Use it for project-specific conventions that change how work should be done. Keep detailed background elsewhere when it does not belong in every session.
A useful instruction names an observable behavior: which command runs the relevant tests, which convention an existing module follows, or which business rule must remain intact. Generic requests for better code provide little guidance.
CLAUDE.md is behavioral context. Claude Code's documentation explicitly distinguishes it from managed settings that enforce restrictions. Configure tool permissions and sandbox boundaries where an operation needs a technical limit. Project memory and instructions
The report also describes using Model Context Protocol, or MCP, to mediate sensitive data access. MCP connects AI tools to external capabilities. A server can expose narrowly defined operations and maintain audit records if its implementation provides them.
MCP does not automatically make a connection safe. Anthropic states that it does not security-audit every MCP server. Review the server's source or provider, granted permissions and data access. Git checkpoints can recover code changes; they cannot reverse every action taken in an external system. Claude Code security guidance
What the ten teams actually handed over
The report spans more than conventional feature development. The table keeps the evidence attached to its original task. Timing claims remain interview reports, with no common experimental baseline.
| Team | Task handed to Claude Code | Reported detail |
|---|---|---|
| Data infrastructure | Infrastructure investigation and repeatable data workflows | Diagnosed pod IP exhaustion using dashboard evidence; pages 3–4 |
| Product development | Prototypes and peripheral features | About 70% of the Vim implementation came from autonomous work; pages 5–6 |
| Security engineering | Control-flow investigation and Terraform plan review | A manual code-scanning task of 10–15 minutes took about 5 minutes; pages 7–8 |
| Inference | Concept research and unit tests | Some research fell from about an hour to 10–20 minutes; pages 9–10 |
| Data science and visualization | Reusable analytics applications | Built a roughly 5,000-line TypeScript app despite limited language experience; pages 11–12 |
| API Knowledge | Finding relevant files and debugging unfamiliar code | Used repository exploration as the first step of a task; pages 13–14 |
| Growth marketing | Data-driven ad-copy generation | Copy creation fell from about 2 hours to 15 minutes; pages 15–16 |
| Product design | Interface changes and interactive prototypes | One coordinated copy update took two 30-minute calls; pages 17–18 |
| RL engineering | Small-to-medium changes and tests | First-attempt success was about one-third; pages 19–20 |
| Legal | Internal tools and workflow prototypes | Planned in Claude.ai, then implemented incrementally; pages 21–22 |
Source: the supplied Anthropic team-interview PDF. The figures describe individual activities, not department-wide savings.
The visualization team's shift is particularly useful for technical leads. Instead of producing disposable Jupyter notebooks, it built reusable React dashboards for model analysis. That creates a different output from merely finishing an existing task faster.
A reusable dashboard can reduce repeated setup on later evaluations. It also creates maintenance work. A team should decide who owns data changes and dependency updates before treating the prototype as an operational service.
The same applies to tools created by designers, marketers or legal staff. These people often know the task's requirements better than an outside implementer. Claude Code lets them express those requirements through working software sooner. Production ownership still needs an explicit handoff.

Measure the cost of accepted work
The Vim figure counts contribution to an implementation. It does not establish a 70% reduction in engineering time. The report does not supply the full cost of prompting, failed attempts, review and later maintenance.
For a pilot, record elapsed delivery time and human work separately. Include investigation, review and rework. Track how many proposed changes are accepted, and why others are discarded. Add tool charges and maintenance effort when comparing operating costs.
This is a proposed measurement approach, not an ROI calculation supplied by Anthropic. It helps reveal a common constraint: faster generation can increase the queue of changes waiting for review.
Independent evidence supports measuring locally. METR's July 10, 2025 study involved 16 experienced developers and 246 real tasks in familiar open-source repositories. AI access increased completion time by 19% in that setting. Participants primarily used Cursor with Claude 3.5/3.7 Sonnet; this was not a direct Claude Code evaluation. METR's original experiment
In its February 24, 2026 update, METR said newer tools likely provided more acceleration, while selection effects made the new estimates unreliable. Neither result should become a universal productivity claim. METR's follow-up
Start with comparable, bounded changes in your own repository. If implementation gets faster while review and rework grow, inspect those stages before increasing the number of concurrent tasks.
Frequently asked questions
Which tasks are sensible starting points for Claude Code?
Repository investigation, missing tests and small, well-defined defects provide inspectable outputs. Expand the scope after the team can verify those outputs reliably. Core business logic and high-impact operations deserve closer human involvement.
Can non-developers use Claude Code effectively?
Anthropic's design, marketing and legal examples show that they can. Initial repository setup and permissions may still require engineering help. A working prototype also needs an owner before it becomes a maintained internal tool.
Does a longer CLAUDE.md improve results?
Project-specific instructions are useful when they change behavior. Remove stale guidance and move detailed material into separate files where appropriate. A long document cannot substitute for enforced permissions or useful tests.
Do these case studies prove an expected productivity gain?
They identify promising use cases. They do not establish a general causal estimate. Measure accepted changes, review time, failed attempts and maintenance in your own environment.
Sources
- Anthropic: How Anthropic teams use Claude Code, supplied PDF
- Anthropic: Official team case study, July 24, 2025
- Claude Code: Best practices
- Claude Code: Project memory and instructions
- Claude Code: Security
- METR: Early-2025 developer productivity experiment
- METR: February 2026 experiment-design update
Author Insight
Tenten editorial perspective: I would evaluate an agent workflow by how quickly a reviewer can decide whether to accept its output. Clear requirements and readable verification evidence make that decision easier. If every proposed change requires someone to reconstruct the original task, the workflow needs better handoffs before it needs more autonomy.
Terminology
| Term | Meaning in this article |
|---|---|
| Claude Code | An agentic development environment that can inspect a project, edit files and use tools |
| CLAUDE.md | A file containing project conventions and behavioral instructions |
| MCP | Model Context Protocol, used to connect AI tools to external capabilities |
| RL | Reinforcement learning; the report includes a dedicated engineering team |
| TDD | Test-driven development, using expected behavior and tests to guide implementation |
| Git checkpoint | A saved version used to compare or recover code changes |





