When Can AI Agents Merge Their Own Code Safely?

Once coding agents can open pull requests on their own, someone has to decide which changes are safe to merge automatically and which need a human to look first. That decision shouldn't be ad hoc. Codebase invariants define what must always stay true — architectural boundaries, error-handling conventions, security-sensitive paths — and a risk-tiered merge policy uses those invariants to route changes to the right level of review. A shared agent-first control plane helps teams connect that policy to issues, tests, and quality gates, so merge decisions are consistent rather than left to individual judgment calls.

At a Glance

AI coding agents need an operating system for quality, not just better prompts.

• Core problem: Agents can finish well-defined coding tasks while still creating maintainability, consistency, and reliability problems.
• New role: Harness engineers encode engineering expectations as reusable context, deterministic checks, review criteria, and risk policies.
• Key practice: Define codebase invariants that must remain true across every change, regardless of whether a human or agent authored it.
• Feedback loop: Agent logs, pull request comments, CI failures, architecture signals, and test quality reveal where the harness needs improvement.
• Operational goal: Match review and merge requirements to the risk of the change instead of applying one approval process everywhere.
  

Why do AI coding agents need a harness?

AI coding agents need a harness because task completion is not the same as safe, maintainable software delivery. Agents can write working code quickly, but they may miss local conventions, create architectural drift, follow instructions inconsistently, or produce changes that require costly review and repair later.

AI-assisted development is now common enough that the limiting factor is often no longer code generation. It is confidence. A coding agent may produce a plausible implementation, open a pull request, and pass a narrow test suite without preserving the broader properties that keep a system healthy.

This gap appears when teams optimize for benchmark-like tasks. A task with a clear prompt and an objective completion condition is much easier than a sequence of interdependent changes in a long-lived repository. Real software has accumulated constraints: package boundaries, error-handling conventions, security rules, naming standards, design systems, deployment expectations, and tribal knowledge.

Evidence from the Stack Overflow Developer Survey 2024 shows widespread developer use of AI tools alongside persistent concerns about accuracy and trust. The relevant lesson is not that teams should stop using coding agents. It is that adoption must be paired with controls that make agent output inspectable and verifiable.

A harness makes those controls explicit. It turns assumptions that once lived in a senior engineer’s head or emerged during peer review into reusable operating rules for every agent-driven change.

What belongs in an agent harness?

• Structured task context: Clear problem statements, acceptance criteria, relevant repository context, and constraints.
• Skills and instructions: Reusable guidance for architecture, libraries, naming, API usage, migrations, and testing.
• Deterministic checks: Linters, static analysis, test suites, dependency checks, and CI/CD gates.
• Agentic review: Focused review prompts that evaluate a change against specific criteria.
• Risk policies: Rules that determine whether a change can merge automatically, needs review, or requires an owner’s approval.
• Feedback signals: Logs, code review patterns, failures, regressions, and operational outcomes used to refine the system.
  

What are codebase invariants, and why do they matter?

Codebase invariants are properties that should remain true across all changes, such as dependency boundaries, consistent error handling, approved libraries, or required test coverage. They matter because they give agents durable system-level constraints instead of relying on task-specific prompts and human memory alone.

An invariant is broader than a feature requirement. “Add password reset” is a task. “Authentication errors must be handled through the shared error boundary” is an invariant. The first tells an agent what to build. The second tells it how the system must continue to behave.

Useful invariants usually describe consistency across the codebase. They can apply to architecture, interface design, data flow, security, observability, performance, or operational behavior.

Examples of practical invariants

• Architecture: A UI package cannot import directly from the database layer.
• Dependencies: New code must use approved libraries for logging, validation, and authentication.
• Error handling: Service errors must be mapped to typed application errors before reaching API handlers.
• Data flow: All changes to customer data must pass through a designated service boundary.
• Testing: Every acceptance criterion for a high-risk workflow must link to a test case.
• Design: Product interfaces must use the established component system rather than one-off visual patterns.

Writing invariants is uncomfortable because it forces teams to decide what they actually believe should always be true. That discomfort is useful. If a rule cannot be stated clearly, it is difficult to consistently enforce through review, automation, or agent context.

Once a team can name an invariant, it can decide how to enforce it. Some rules belong in a coding agent’s task context. Others should become lint rules, static checks, tests, or merge gates. The strongest approach combines both: explain the intended pattern before implementation, then check for violations afterward.

How do you verify agent-written code at scale?

Verify agent-written code at scale by using layered checks: deterministic CI rules for clear violations, focused agent review for broader criteria, and human review for high-risk or ambiguous decisions. Review should test evidence against explicit requirements, not assume generated changes are correct because they look plausible.

Traditional pull request review does not automatically scale with agent throughput. If code production increases sharply while review capacity stays constant, teams either delay delivery or lower their quality bar. Neither outcome is sustainable.

The answer is not to ask a second agent to blindly approve the first. Automated reviewers can make mistakes, contradict one another, or create long cycles of unproductive changes. A review finding should be treated as evidence to consider, not an unquestionable command.

Use the right verification mechanism for each rule

• Deterministic rules: Use static analysis, linting, dependency checks, schema validation, and tests when the requirement can be stated precisely.
• Narrow semantic checks: Use targeted prompts to inspect a file or change for a specific concern, such as missing authorization or inconsistent error handling.
• Broad review criteria: Use agent-assisted review to identify design, maintainability, or requirement-fit concerns that do not reduce cleanly to a rule.
• Human judgment: Reserve human review for product tradeoffs, high-impact architecture, unclear requirements, and security-sensitive changes.

The GitHub code scanning documentation illustrates the value of automating repeatable checks within the development workflow. The same principle applies to agentic development: move predictable verification into automation so humans can focus on decisions that genuinely require judgment.

A useful review loop asks four questions:

1. What acceptance criteria does this change satisfy?
2. Which invariants could this change violate?
3. What deterministic evidence supports the implementation?
4. Does the risk level require a human to inspect or approve it.
   

Give agents work with context, gates, and proof.

Levr connects coding agents to shared issues, acceptance criteria, tests, workflow states, and attributed activity, so plan, implementation, verification, and review remain connected.

Explore agent-first project workflows

How can teams use analytics to improve coding agents?

Teams improve coding agents by treating their activity as operational data. Agent logs, failed runs, review comments, CI failures, test outcomes, and architecture signals can reveal repeated failure modes, which teams can convert into better context, stronger checks, or clearer risk policies.

Analytics changes the question from “Did this agent make a mistake?” to “What recurring system condition makes this class of mistake likely?” That distinction matters. An individual bad change is an incident. A repeating pattern is a harness design problem.

High-value data sources for agentic development

• Agent execution logs: Identify loops, confused tool use, repeated failed commands, and excessive context gathering.
• Pull request discussions: Find comments that reviewers make repeatedly across generated changes.
• CI/CD history: Track recurring test failures, lint violations, build errors, and deployment blocks.
• Codebase structure: Detect rising complexity, undesirable imports, circular dependencies, and weak module boundaries.
• Test quality signals: Use mutation testing or related analysis to find tests that pass without meaningfully detecting faulty behavior.

Start small. Ask an agent to summarize the last 50 pull requests, focusing on recurring review comments, reverted changes, or failed checks. Then review the summary critically and select one pattern worth addressing. A pattern may indicate a missing instruction, a poorly designed task template, an absent deterministic check, or a tool interface that agents misuse.

Do not turn every observation into a permanent rule. Measure whether the new rule catches a real class of problems and whether it creates noisy false positives. Harness engineering needs iteration, not rule accumulation.

How should teams decide when agents can auto-merge?

Teams should decide auto-merge permissions through a risk ladder that considers the code area, change type, available verification, and potential blast radius. Low-risk changes with strong automated evidence can move faster, while sensitive or high-leverage changes should require increasingly deliberate human approval.

One review policy for every repository path is usually too blunt. A formatting update, a well-tested internal utility change, and a modification to authentication logic do not deserve identical merge requirements.

A practical agent auto-merge ladder

1. Experimental or disposable code: Minimal controls. Fast iteration is the priority.
2. Low-risk internal tooling: Auto-merge after required checks pass and scoped rules are satisfied.
3. Standard product changes: Automated checks plus human review by the engineer accountable for the area.
4. High-impact systems: Require domain-owner review, stronger test evidence, and potentially staged rollout controls.
5. Security, access, or irreversible changes: Require explicit approvals and additional verification before merge or release.

Risk is not only about the directory being changed. It also depends on what the agent can do. A small change that alters authorization, payments, production infrastructure, data deletion, or public API behavior may demand a higher review tier than a large refactor in a constrained internal module.

Levr supports this model by keeping workflow states and quality gates on the same work objects used by humans and coding agents. Teams can define intent, let an agent work from structured acceptance criteria, record verification results, and retain human approval where the policy requires it. See how Levr’s agentic workflow operates.

What does a software factory look like in practice?

A software factory is an agent-driven delivery system where work enters through structured requests, moves through implementation and verification stages, and produces data that improves future runs. Its purpose is not fully autonomous coding, but repeatable software production with visible controls and feedback loops.

The starting point is a well-formed issue. Instead of issuing an isolated prompt, a team captures the desired outcome, constraints, acceptance criteria, and relevant context in a work item. An orchestrated agent process can then pick up that work, apply the right skills, make changes, run checks, and prepare a reviewable result.

The critical difference is that the process does not end when code is generated. It includes quality gates, risk classification, review, and learning from outcomes. This creates a loop where every recurring defect can become a candidate for better instructions, better checks, or a more appropriate workflow policy.

For teams operating multiple coding agents, the control plane becomes important. Levr allows Claude Code, Cursor, Codex, Copilot, and other Model Context Protocol clients to work on the same shared projects, issues, tests, and workflow gates. Its activity trail attributes changes to either a human or agent, which makes automated throughput easier to inspect without losing accountability.

How do you start practicing harness engineering?

Start harness engineering with one invariant, one enforceable check, and one small dataset of past delivery signals. This creates a manageable improvement loop: define a rule, apply it in the workflow, measure its effect, and revise the harness based on real evidence rather than assumptions.

A 30-day starting framework

1. Choose one recurring problem: Pick an issue that repeatedly appears in code review, CI failures, or production defects.
2. Name the invariant: Write a concise statement describing what must always be true.
3. Choose enforcement: Decide whether the rule belongs in agent instructions, a deterministic check, an agent review prompt, or a merge policy.
4. Apply it narrowly: Start with one repository area or work type rather than adding it everywhere immediately.
5. Collect evidence: Monitor failures, false positives, reviewer feedback, cycle time, and follow-up fixes.
6. Refine the rule: Simplify overly broad instructions, remove contradictions, and strengthen checks that reliably catch real defects.

The goal is not to create a giant document that agents nominally receive and partially ignore. It is to make the important expectations clear, timely, testable, and observable. Short, targeted context plus reliable enforcement is generally more useful than an overloaded instruction set.

Technical Deep Dive FAQ

What is harness engineering?

Harness engineering is the practice of designing the environment around coding agents so their work remains aligned with engineering standards. It includes task context, reusable skills, deterministic checks, code review criteria, risk tiers, monitoring, and feedback loops. The term emphasizes that strong outcomes come from the surrounding system, not only from the intelligence of the model. A harness engineer helps convert implicit practices into explicit rules and evidence-based workflows that can scale as automated coding throughput increases.

What is an AI coding agent?

An AI coding agent is a software system that can interpret a development task, inspect a repository, use tools, modify files, run commands, and propose or complete code changes. Unlike autocomplete, a coding agent can operate across multiple steps and interact with an engineering environment. Its capabilities do not remove the need for engineering controls. Agents can misunderstand requirements, choose weak implementations, misuse tools, or create code that passes a narrow test while conflicting with broader system constraints.

Why do coding agents pass benchmarks but fail in real repositories?

Benchmarks commonly evaluate bounded tasks with explicit success conditions. Long-lived repositories contain hidden dependencies, conventions, historical decisions, incomplete documentation, and changing requirements. An agent may solve an isolated implementation problem while introducing coupling, duplication, inconsistent patterns, or weak tests that only become costly later. Real repository work also requires sequencing changes safely and preserving codebase health across many tasks. Harnesses provide this missing system context through invariants, checks, and risk-aware review paths.

How do you write effective skills for coding agents?

Write skills as focused, situational guidance rather than exhaustive manuals. State when the skill applies, what outcome it supports, the important constraints, and examples of correct behavior when needed. Avoid stuffing many unrelated instructions into every task because agents can ignore or confuse conflicting context. Test skills against representative work, then measure whether the relevant behavior improves. If an instruction is nonnegotiable, pair the skill with a deterministic check or review gate instead of relying on the agent to remember it perfectly.

What should be deterministic in an agentic development workflow?

Requirements that can be stated unambiguously should be deterministic wherever practical. Examples include forbidden imports, required dependency versions, formatting, type safety, schema validation, test execution, branch protections, and known security rules. Deterministic checks are fast, repeatable, and easier to audit than subjective review. They should not replace broader engineering judgment, but they reduce avoidable review work. Use agent review and human judgment for concerns that involve tradeoffs, incomplete context, or architectural interpretation.

How can you prevent conflicting agent instructions?

Prevent conflicting instructions by keeping guidance modular, scoped, and testable. Every skill should clearly describe when it applies, and teams should avoid loading unrelated rules into the same task. Review new skills against existing ones for contradictory requirements, especially around architecture, naming, libraries, and implementation sequence. When conflicts cannot be avoided, define a clear priority order. More importantly, enforce critical constraints outside the prompt through tests, static analysis, or merge gates, because prompts alone are not a reliable policy engine.

What metrics matter for coding agent performance?

Useful coding agent metrics include task completion quality, rework rate, pull request cycle time, CI failure rate, review comment recurrence, test pass trends, escaped defects, rollback frequency, and cost per accepted change. Agent logs can also reveal repeated tool failures, long loops, or excessive retries. Avoid measuring only volume, such as lines changed or pull requests opened, because output volume can hide quality problems. The goal is to understand whether agents improve the delivery system, not merely whether they generate more code.

When should an agent be allowed to auto-merge code?

An agent should auto-merge only when the change is low risk, clearly scoped, covered by appropriate automated checks, and governed by an explicit policy. Suitable candidates might include constrained internal tooling or routine updates with strong test coverage. Changes involving sensitive data, authentication, production infrastructure, public interfaces, or complex architectural decisions should require human approval. Auto-merge is a policy decision, not a model capability. Teams should continually review incident and rework data to adjust which changes qualify.

How does a control plane help teams manage coding agents?

A control plane gives humans and coding agents a shared source of truth for work, requirements, state, testing, and approvals. Instead of leaving context in scattered prompts, chats, branches, and external trackers, it connects an issue to its acceptance criteria, implementation progress, test evidence, and activity history. This helps teams see what an agent did and why. In Levr, projects, issues, tests, workflow gates, and attributed activity remain connected across human and agent work.

Can agentic code review replace human code review?

Agentic code review can reduce repetitive review work and surface issues that humans might miss, but it should not be treated as an unquestionable authority. Automated review is most valuable when it evaluates explicit criteria and provides evidence for a human or downstream policy to assess. It is weaker at resolving ambiguous requirements, product tradeoffs, and complex system consequences. A mature workflow uses automated review to increase coverage and speed while reserving human attention for decisions with meaningful uncertainty or impact.

At a Glance

Harness engineering turns agentic development from a prompt-driven activity into an observable software delivery system.

• Define invariants: Capture the system properties that must survive every code change.
• Verify in layers: Combine deterministic CI checks, focused agent review, and human judgment.
• Use operational data: Mine logs, reviews, tests, and failures for recurring patterns.
• Apply risk tiers: Match auto-merge and approval requirements to the potential impact of each change.
• Improve continuously: Treat every recurring failure as a possible harness improvement.
  

Further reading

• Stack Overflow Developer Survey 2024
• GitHub documentation: About code scanning
• GitHub Octoverse
• Levr agent-first project control plane
• Levr features for humans and coding agents
• Levr agentic workflow
  

Ship at agent speed

Give your coding agents a control plane, not just a prompt.

Levr connects your coding agents, including Claude Code, Cursor, Codex, and Copilot, to shared project context with issues, gates, test suites, and activity history. Free access is available during beta.

Get early access, it’s free

No credit card required during beta.