1 Introduction: The Evolution of Vibe Coding
Vibe coding started as a playful phrase, but senior engineering teams should treat it as a serious change in the software delivery model. The core question is no longer, “Can AI write code?” It can. The better question is, “What engineering controls are required when code is produced through natural language, tool access, repo context, and autonomous loops?”
This article covers the first three foundations of agentic engineering: how vibe coding evolved, how to classify different AI-assisted developer roles, and how to choose between IDE-native tools, plugins, and terminal agents.
1.1 Origin of the Phrase: Andrej Karpathy’s Paradigm Shift
Andrej Karpathy popularized the phrase “vibe coding” in February 2025, describing a style where the developer can “forget that the code even exists” and guide the system through intent, feedback, and runtime observation rather than manual syntax authoring.
That line resonated because it named something many developers were already doing. They were no longer asking a chatbot for one function at a time. They were asking the model to build a screen, repair a failing test, create a migration, explain an exception, and adjust multiple files in one pass.
The important shift is not that English became a programming language. That idea is too loose. The real shift is that modern coding agents can hold enough context to reason across files, dependency boundaries, error messages, and existing conventions. A developer can say:
Add tenant-level feature flags to the billing API.
Follow the existing middleware pattern.
Add integration tests for disabled, enabled, and missing flag cases.
Do not change the public response contract.A traditional code generator would produce isolated snippets. An agentic coding tool should inspect the current project, find the middleware pattern, modify the right files, run tests, and return a diff. The engineer still owns the decision, but the mechanical path from intent to implementation is compressed.
This matters because experienced developers spend less time fighting syntax and more time supervising design integrity.
1.2 The Semantic Drift: From “Weekend Prototyping” to Production Realities
Early vibe coding was mostly associated with quick experiments: landing pages, CRUD apps, prototypes, internal utilities, and “build me a demo by tonight” workflows. That is still a valid use case. But the center of gravity has moved.
AI development platforms are now part of serious engineering conversations because they are tied to measurable delivery compression. Lovable publicly said it crossed $100 million in ARR in July 2025, eight months after its first $1 million ARR milestone. Cursor later stated that it had crossed $1 billion in annualized revenue by November 2025, with millions of developers and major engineering organizations using the product.
The exact revenue numbers are less important than the signal: teams are paying for tools that reduce the distance between intent, code, test, and deployment.
But production work exposes the weakness of pure vibe coding. A prototype can tolerate a messy service layer. A regulated enterprise system cannot. A weekend app can duplicate validation logic. A payments module cannot. A generated API can “mostly work” in a demo but fail under concurrency, tenancy, audit, or backward compatibility constraints.
The semantic drift is clear:
Prototype vibe coding:
"Build a claims dashboard with filters and charts."
Production agentic engineering:
"Implement the claims dashboard using our existing RBAC model, preserve audit logging,
avoid N+1 queries, enforce agency-level data partitioning, add Playwright coverage,
and update the OpenAPI contract only if the response schema changes."The second prompt is not just more detailed. It encodes architecture, security, performance, and test expectations. That is where senior engineers add value.
1.3 The Modern Paradigm: Shifting from Syntax Authoring to Intent Supervision
Traditional productivity metrics do not map well to agentic development. Lines written per hour becomes a misleading measure when an agent can modify ten files in one run. The bottleneck moves from typing to judgment.
A useful way to think about the work is three loops:
Inner loop:
Edit code, run tests, fix syntax, repair local errors.
Middle loop:
Guide the agent, inspect diffs, validate behavior, adjust scope, enforce architecture.
Outer loop:
Decide system boundaries, delivery sequencing, risk posture, release strategy, and governance.Vibe coding collapses part of the inner loop. Agentic engineering strengthens the middle loop. Senior architects still own the outer loop.
The middle loop is where most teams struggle. They either under-supervise the agent and accept risky changes, or over-review every generated line and lose the productivity gain. The better approach is process supervision: define boundaries, let the agent work inside them, and validate outcomes through automated checks.
A simple working agreement helps:
agent_policy:
allowed:
- modify application source files
- add or update tests
- run local test commands
- update documentation tied to changed behavior
restricted:
- change authentication flow without approval
- introduce new runtime dependencies without approval
- modify database schema without migration review
- alter public API contracts without OpenAPI diff
required_checks:
- npm test
- npm run lint
- npm run typecheck
- npx playwright testThis turns “AI wrote code” into “AI operated within an engineering control system.”
2 Taxonomies of the AI-Assisted Era: Defining the Roles
Not every developer using AI is working the same way. The difference is not the tool. The difference is the level of verification, autonomy, and system ownership.
2.1 Vibe Coder vs. Vibe Engineer vs. Agentic Coder
2.1.1 The Vibe Coder
The vibe coder works primarily through natural language. They describe the desired result, accept the generated code, run it, paste errors back, and keep iterating.
This is useful for exploration. It is also dangerous in production if the developer does not understand the generated design.
Incorrect:
Build JWT authentication for this app. Use whatever library is best.Why this fails: the model may introduce a new dependency, ignore existing identity provider rules, store secrets incorrectly, or create a parallel auth path.
Better:
Add JWT validation using the existing AuthMiddleware pattern.
Do not introduce a new auth library.
Use the configured JWKS endpoint from appsettings.
Add tests for expired token, invalid signature, and missing role claim.The vibe coder asks for output. The engineer defines constraints.
2.1.2 The Vibe Engineer
The vibe engineer uses natural language, but does not trust natural language alone. They combine prompts with runtime validation, integration tests, local builds, type checks, and architecture rules.
A vibe engineer does not ask, “Does the code look good?” They ask, “Did the system behavior improve without breaking contracts?”
Recommended workflow:
git checkout -b feature/tenant-flags
npm run typecheck
npm test
npm run lint
npx playwright test
git diff --stat
git diff -- src/api src/testsThe important habit is baseline-first development. Run the checks before the agent changes anything. Then run the same checks afterward. If the baseline was already broken, the agent should not be blamed for unrelated failures.
2.1.3 The Agentic Coder
The agentic coder moves beyond chat. They design autonomous loop parameters and give the agent controlled access to the repo, terminal, file system, and test runner.
The prompt becomes closer to a work order:
Goal:
Fix the intermittent timeout in PaymentReconciliationJob.
Constraints:
- Do not change the public payment status enum.
- Do not increase retry count above 3.
- Preserve existing audit log fields.
- Add a regression test that fails without the fix.
Process:
1. Inspect recent changes related to PaymentReconciliationJob.
2. Identify likely cause.
3. Propose the patch before editing.
4. Apply the smallest safe change.
5. Run the focused test suite.
6. Summarize diff, risk, and remaining concerns.This is no longer “generate code.” It is delegated engineering execution.
2.2 Understanding Boundaries of System Responsibility
The biggest mistake is assuming the model should own everything. It should not.
Humans remain load-bearing for:
- architectural boundaries
- domain invariants
- regulatory obligations
- data ownership rules
- security posture
- release risk
- backward compatibility
- incident impact
- normalization and state designLLMs are useful for:
- finding similar patterns in the codebase
- drafting repetitive implementation code
- generating tests from known rules
- explaining unfamiliar modules
- refactoring local code safely
- updating documentation from diffs
- creating migration scaffoldingFor example, an agent can implement a repository method. But the architect should decide whether the system needs a repository at all, whether the aggregate boundary is correct, and whether the transaction model is safe.
Use this rule: agents can accelerate decisions that have already been made; they should not silently make decisions that change the architecture.
2.3 The Verification Gap: Managing the “Review Tax”
The review tax is the hidden cost of AI-generated code. If the agent creates 600 lines of plausible code, someone must still determine whether those lines are correct.
Generated code often fails in subtle ways:
- duplicate dependencies
- inconsistent error handling
- missing edge cases
- hidden breaking changes
- fake abstractions copied from nearby files
- tests that assert implementation instead of behavior
- optimistic handling of nulls, retries, and timeoutsThe answer is not manual review of every token. The answer is layered verification.
Recommended guardrails:
verification:
static:
- type checking
- lint rules
- dependency policy
- public API diff
behavioral:
- unit tests
- integration tests
- contract tests
- browser or API regression tests
architectural:
- dependency direction checks
- module boundary rules
- database migration review
- threat-model review for sensitive flowsA useful pattern is “agent writes, automation judges, human decides.” The agent can produce the patch. CI can reject unsafe changes. The engineer reviews the remaining design risk.
3 The Three Interfaces of Modern AI Development
AI-assisted development now happens through three main interfaces: AI-native IDEs, integrated plugins, and terminal agents. Each has a different operational profile.
3.1 IDE-Native Environments
IDE-native environments such as Cursor, Windsurf/Devin Desktop, and Google Antigravity are designed around AI-first workflows rather than treating AI as an add-on. Cursor describes itself as a coding agent where developers can hand off tasks while focusing on decisions. Google describes Antigravity as an agentic development platform where agents can plan, execute, and verify tasks across editor, terminal, and browser surfaces.
The strength of these environments is context. They can index the codebase, track open files, understand recent edits, and apply multi-file changes through a composer or agent workflow.
Use IDE-native tools when:
- the task spans several files
- the agent needs repository context
- visual diff review matters
- the team wants fast interactive iteration
- developers are comfortable adopting a new editor workflowThe trade-off is tool lock-in. AI-native IDEs move quickly, and product behavior can change month to month. For enterprise teams, that means onboarding guides, approved settings, and security review cannot be one-time activities.
3.2 Integrated Plugins
Integrated plugins are a better fit when the organization already standardizes on VS Code, Visual Studio, Rider, IntelliJ, or another established IDE. GitHub Copilot supports coding suggestions and chat in VS Code, and GitHub documents setup across multiple editor environments. GitHub also introduced Copilot Extensions support for JetBrains IDEs in public preview, allowing teams to query third-party tools or private data from inside the IDE.
Plugins work best when AI should assist but not dominate the workflow.
Use plugins when:
- developers need inline completion
- enterprise IDE standards are already fixed
- security teams prefer mature editor controls
- teams want gradual adoption
- tasks are local rather than deeply autonomousThe limitation is orchestration depth. A plugin can be excellent for suggestions, explanations, and small refactors. But for long-running autonomous work, terminal-first or agent-native tools are often more effective.
3.3 Terminal and CLI Agents
Terminal agents changed the conversation because they operate where software is actually built: git, shell, package managers, test runners, containers, and deployment scripts.
Claude Code is described by Anthropic as an agent that reads a codebase, edits files, and runs commands across terminal and IDE workflows. Gemini CLI is documented as an open-source AI agent that runs in the terminal and uses a ReAct loop with built-in tools and MCP servers for tasks such as bug fixing, feature creation, and test coverage improvements. Amp similarly positions itself as a coding agent for terminal and editor workflows, with extensibility through plugins and policy hooks.
Terminal agents are powerful because they can execute the real feedback loop:
git status
npm test -- --runInBand
npm run lint
docker compose up -d postgres redis
npm run test:integration
git diffThis is closer to how a senior developer validates work. The agent does not just write code. It runs the system, observes failure, patches, and reruns.
3.3.1 Why the Terminal Unexpectedly Reclaimed Developer Productivity
The terminal became important again because agentic coding is not only about text generation. It is about tool execution.
A GUI-heavy workflow is useful for review, but terminal loops are faster for autonomous repair:
1. inspect failing test
2. search codebase
3. edit focused files
4. run test
5. inspect error
6. patch again
7. produce final diffThere is less rendering overhead, less manual clicking, and a clearer audit trail. The terminal also makes sandboxing easier. You can run agents inside a container, mount only the target repo, restrict network access, and preserve a clean git diff.
Recommended starting setup:
git checkout -b ai/bugfix-payment-timeout
docker compose up -d
npm ci
npm testThen give the agent a narrow task and require a summary:
Fix the failing PaymentReconciliationJob timeout test.
Use the smallest safe change.
Do not modify schema or public API contracts.
Run the focused test and show the final diff summary.That is the practical direction of modern AI development: not blind vibe coding, not manual-only engineering, but supervised agentic execution with strong boundaries.
4 The 2026 Inflection Point: Models and Capability Frontiers
The practical difference in 2026 is not that models became “smarter” in a vague sense. The difference is that frontier models became more useful across longer execution loops. They can hold more context, stay aligned to a task for longer, recover from tool errors more often, and produce fewer circular fixes where the same broken patch is applied under different wording.
For architects, this changes the planning model. A coding agent is no longer just a faster autocomplete system. It is becoming an execution unit that can inspect a codebase, plan a change, call tools, run checks, and revise its own work. That does not remove engineering discipline. It raises the cost of weak discipline because a poorly bounded agent can create a larger mess faster than a junior developer.
4.1 Next-Gen Frontier Models
The newest model families are better at long-horizon software tasks because they combine stronger reasoning, tool-use reliability, and larger working context. For engineering teams, the useful metric is not benchmark rank alone. The useful metric is whether the model can take a multi-step change and avoid losing the original constraint after the third or fourth tool call.
A common failure in older agent workflows looked like this:
Task:
Refactor OrderService to support partial shipment cancellation.
Failure pattern:
1. Agent updates the service method correctly.
2. Agent adds a test but mocks the wrong dependency.
3. Agent sees the failing test and changes the production code to match the bad mock.
4. Agent breaks the real integration path.
5. Human reviewer spends an hour untangling intent from side effects.Modern frontier models reduce this pattern, but they do not eliminate it. The safer approach is to make the model’s operating contract explicit. Instead of asking for “the refactor,” ask for staged execution.
work_order:
objective: "Support partial shipment cancellation"
allowed_files:
- "src/orders/**"
- "tests/orders/**"
protected_files:
- "src/payments/**"
- "src/auth/**"
constraints:
- "Do not change public API response fields"
- "Do not modify payment capture behavior"
- "Preserve existing audit event names"
checkpoints:
- "Explain current flow before editing"
- "List files that require changes"
- "Apply patch"
- "Run focused tests"
- "Summarize behavior change and residual risk"This structure matters because better models are still probabilistic systems. Their reliability improves when the task is decomposed into verifiable checkpoints. The goal is not to micromanage every line. The goal is to prevent silent architectural drift.
For production systems, model choice should follow task shape. Use the strongest model for unclear, cross-cutting, or high-risk work. Use a cheaper model for repetitive updates, documentation passes, small test generation, or schema-to-client mapping. A senior team should not route every task to the most expensive model by default.
from enum import Enum
class Risk(Enum):
LOW = "low"
MEDIUM = "medium"
HIGH = "high"
def select_model(task_type: str, risk: Risk, files_touched: int) -> str:
if risk == Risk.HIGH:
return "frontier-reasoning-model"
if task_type in {"architecture_review", "security_review"}:
return "frontier-reasoning-model"
if files_touched > 8:
return "strong-coding-model"
if task_type in {"docs", "test_scaffold", "rename"}:
return "cost-efficient-coding-model"
return "balanced-coding-model"This is simple, but it reflects the right operating principle: route by risk, not by habit.
4.2 Mitigating Context Fragmentation
Large context windows changed how agents analyze codebases, but they did not remove the need for context engineering. More tokens do not automatically mean better reasoning. A model can ingest a large amount of code and still focus on the wrong files if the prompt does not tell it what is load-bearing.
Context fragmentation happens when the agent sees pieces of the system but fails to maintain the relationship between them. In a real application, the behavior may be spread across a route handler, domain service, validation rule, database migration, background job, and UI state transition. The bug is rarely in just one file.
A useful pattern is to create a context manifest before the agent begins implementation.
{
"feature": "partial shipment cancellation",
"entry_points": [
"src/api/orders/cancel-shipment.controller.ts",
"src/orders/order.service.ts"
],
"domain_rules": [
"Cancellation is allowed only before carrier handoff",
"Captured payments must be refunded through RefundService",
"Audit event must be written for every state transition"
],
"related_tests": [
"tests/orders/order.service.spec.ts",
"tests/api/cancel-shipment.contract.spec.ts"
],
"do_not_change": [
"src/payments/payment-capture.service.ts",
"src/audit/audit-event-names.ts"
]
}The manifest gives the agent a stable working map. It also gives the reviewer a way to detect whether the agent ignored important files.
For large codebases, the best pattern is not “load the whole repository.” It is layered retrieval. Start with architecture documents and dependency maps. Then retrieve files by call graph, ownership, and recent changes. Finally, add focused test files. This reduces noise and keeps the model from treating every file as equally important.
git log --oneline -- src/orders src/api/orders tests/orders | head -20
rg "CancelShipment|RefundService|carrier handoff" src tests
npm run test -- orders --watch=falseThe agent can run these commands, but the architect should define why those commands matter. Context strategy is now an engineering skill. The team that controls context gets better output than the team that simply buys a larger context window.
4.3 Model Context Protocol (MCP) and Real-Time Tool Execution
Model Context Protocol is important because agent workflows need a controlled way to reach external tools. Without a standard interface, every tool integration becomes a custom bridge with inconsistent authentication, logging, and permission behavior.
In practice, MCP is useful for exposing safe capabilities to the agent: query Jira issues, inspect a database schema, read service metadata, search internal documentation, or call a local static analysis tool. The key word is safe. Do not expose raw production access to an autonomous coding agent.
A local MCP gateway should act like a narrow adapter, not an open tunnel.
from mcp.server.fastmcp import FastMCP
import sqlite3
mcp = FastMCP("engineering-context")
@mcp.tool()
def get_table_schema(table_name: str) -> dict:
allowed_tables = {"orders", "shipments", "refunds"}
if table_name not in allowed_tables:
return {"error": "table not allowed"}
conn = sqlite3.connect("readonly_schema.db")
rows = conn.execute(f"PRAGMA table_info({table_name})").fetchall()
return {
"table": table_name,
"columns": [
{"name": row[1], "type": row[2], "nullable": not bool(row[3])}
for row in rows
]
}
if __name__ == "__main__":
mcp.run()The example is intentionally narrow. The agent can inspect schema, but it cannot run arbitrary SQL. That distinction matters. Tool access should be permissioned, logged, and scoped to the task.
For enterprise use, every MCP server should answer three questions:
What can the agent read?
What can the agent change?
How will we audit what happened?If those answers are unclear, the integration is not ready for production engineering workflows.
Here is a revised and more detailed version with stronger practical examples and cleaner flow.
5 The 8 Levels of AI Trust and Adoption (The Yegge Framework)
The eight-level model is a practical way to describe how much autonomy a team gives to AI-assisted engineering. The point is not to rush toward Level 8 because it sounds advanced. The point is to understand where your team actually operates today, what kind of work is safe at that level, and what controls are required before moving higher.
A team can be Level 5 for test generation and Level 2 for authentication code. That is normal. Trust should vary by domain risk, codebase maturity, team experience, and the quality of automated verification. For example, using an agent to generate UI validation tests is very different from allowing the same agent to modify payment settlement logic or authorization middleware.
The useful question is not, “How much AI are we using?” The better question is, “How much unsupervised decision-making are we allowing, and do we have the engineering controls to support it?”
5.1 Level 1: Chatbot
At Level 1, the model is outside the codebase. Developers paste snippets into a chat interface, ask design questions, request explanations, or use the model as a thinking partner. This is the lowest-risk level because the model has no direct access to the repository, file system, terminal, secrets, database, or build process.
This level is useful for isolated reasoning. A developer might ask why a LINQ query is slow, compare retry patterns, draft a regular expression, explain a compiler error, or review a small function. The limitation is context. The model only knows what the developer pasted, and production behavior usually depends on files, constraints, and runtime rules that were not included.
Example:
public bool CanCancel(Order order)
{
return order.Status != OrderStatus.Shipped;
}A chatbot can suggest that the function should handle null values or use a more expressive domain method. But it cannot know whether Shipped means warehouse dispatch, carrier handoff, customer notification, invoice posting, or payment capture. It also cannot know whether cancellation should be blocked because of a fraud review, refund status, or audit lock.
A better Level 1 prompt gives business context without pretending the chatbot can complete the implementation:
Review this cancellation rule for possible edge cases.
Context:
- Orders can be Created, Paid, Packed, Shipped, Delivered, or Cancelled.
- Cancellation is allowed before carrier handoff.
- Paid orders require refund workflow.
- Audit logging is handled elsewhere.
Question:
What cases should this method consider before we implement the final domain rule?At this level, the output should be treated as design input, not final code. The developer still brings the answer back into the real codebase, checks the domain model, and validates the implementation locally.
Level 1 is best for thinking support, syntax lookup, quick comparison of options, and early design exploration. It is not appropriate for final production changes unless the engineer independently verifies the result.
5.2 Level 2: Copilot / IDE Assistant
At Level 2, AI is inside the editor. It can see the current file, nearby code, and sometimes a limited slice of the workspace. The developer still approves individual suggestions, accepts or rejects completions, and remains responsible for each change.
This level is highly productive for repetitive and pattern-based work. DTOs, mapping code, test scaffolding, validation messages, simple controller methods, and framework boilerplate are good examples. The AI is not truly autonomous here. It is assisting the developer while the developer stays in control.
Example:
export interface ShipmentCancellationRequest {
orderId: string;
shipmentId: string;
reasonCode: "CUSTOMER_REQUEST" | "INVENTORY_ERROR" | "ADDRESS_ISSUE";
}This is a good fit for Level 2 because the model can infer naming conventions from nearby TypeScript interfaces. The developer still needs to verify whether reasonCode values match the backend enum, API documentation, analytics events, and database constraints.
A practical Level 2 workflow looks like this:
export function validateCancellationRequest(
request: ShipmentCancellationRequest
): string[] {
const errors: string[] = [];
if (!request.orderId?.trim()) {
errors.push("Order ID is required.");
}
if (!request.shipmentId?.trim()) {
errors.push("Shipment ID is required.");
}
if (!request.reasonCode) {
errors.push("Cancellation reason is required.");
}
return errors;
}This generated code may look fine, but a senior developer should still ask: does validation belong in the UI, API layer, or shared schema? Should errors be returned as strings, codes, or localized resources? Should reason codes be hardcoded or imported from a generated contract?
Level 2 is safe when suggestions are small and reviewable. It becomes risky when developers accept larger completions without understanding the surrounding system. The control point is granular human review.
5.3 Level 3: IDE Agent in YOLO Mode
At Level 3, the AI moves from suggestion to execution. The IDE agent can modify multiple files with fewer permission prompts. Tools such as composer-style agents can update components, services, tests, and configuration in a single run. This is where productivity improves sharply, but so does risk.
Use this mode for bounded, low-to-medium-risk work. Good examples include adding a new field to an existing form, updating labels across a feature module, creating a new API client method based on an existing pattern, or fixing lint errors inside a specific folder.
Before starting, create a branch and define a narrow scope:
git checkout -b ai/add-cancellation-reason
git status --short
npm run test:ordersA good Level 3 instruction is specific about files, constraints, and validation:
Add cancellationReason to the shipment cancellation flow.
Scope:
- Update the Angular form.
- Update the TypeScript request model.
- Update the API client method.
- Add or update unit tests for the form validation.
Do not:
- Modify authentication.
- Change routing.
- Change backend enum values.
- Update unrelated shared components.After the agent runs, review the file list before reviewing the code. The file list often reveals scope drift faster than the diff itself.
git diff --name-onlyExpected output might be:
src/app/orders/cancel-shipment/cancel-shipment.component.ts
src/app/orders/cancel-shipment/cancel-shipment.component.html
src/app/orders/models/shipment-cancellation-request.ts
src/app/orders/services/orders-api.client.ts
src/app/orders/cancel-shipment/cancel-shipment.component.spec.tsUnexpected output might include:
src/app/auth/auth.guard.ts
src/app/shared/date-utils.ts
package.jsonThat is a warning sign. The human does not need to approve every keystroke, but they must monitor boundaries. If the agent edits unrelated infrastructure, shared utilities, security code, or dependency files without a clear reason, stop the run and reset.
5.4 Level 4: The Process Review Shift
Level 4 is the first major mindset shift. The engineer stops reviewing every line in real time and starts reviewing the process of execution. The question changes from “Do I approve this edit?” to “Did the agent follow a safe and logical path from objective to verified result?”
This matters because line-by-line review does not scale when an agent can produce a large, multi-file patch. A small diff created through a poor process can be more dangerous than a larger diff created through a clean process with proper validation.
A Level 4 review looks at the agent’s plan, files touched, commands run, tests executed, assumptions made, and any unexpected behavior. The review artifact may look like this:
{
"agent_run_review": {
"objective": "Add cancellation reason to shipment cancellation flow",
"planned_files": 5,
"actual_files": 6,
"unexpected_files": ["src/app/shared/date-utils.ts"],
"commands_run": [
"npm run test:orders",
"npm run lint",
"npm run typecheck"
],
"tests_added": [
"should require cancellation reason",
"should submit cancellation reason to API client"
],
"risk": "medium",
"requires_human_review": true
}
}This metadata helps the reviewer focus. If the agent touched an unexpected shared utility, the first question is why. Maybe the change was valid. Maybe the agent tried to fix a date formatting issue by changing a global function. The process review catches this before it becomes a hidden regression.
A practical Level 4 checklist is:
process_review:
scope:
- Did the agent stay inside the requested module?
- Were unexpected files touched?
reasoning:
- Did the agent explain why each file changed?
- Did it identify assumptions clearly?
validation:
- Were focused tests run?
- Were type checks and lint checks run?
- Did the agent weaken or remove any test?
risk:
- Did the change affect public contracts?
- Did the change affect security, payments, or data access?At this level, the engineer becomes a supervisor of execution quality. The agent can move faster, but the process must be observable.
5.5 Level 5: Terminal-First Autonomous Agents
At Level 5, the agent works directly through command-line tools and local build systems. It can inspect files, run tests, search the repository, read errors, patch code, and rerun checks. This is powerful because the terminal exposes the real engineering feedback loop.
This level is especially effective for bugs with observable failures. For example, a regression test fails after a recent change to shipment cancellation. The agent can run the failing test, inspect the stack trace, search for related code, make a patch, and verify the fix.
Example starting point:
git checkout -b ai/fix-cancel-shipment-timeout
npm run test -- cancel-shipment.contract.spec.ts
npm run lint -- src/app/orders
npm run typecheckA strong Level 5 instruction should include both the goal and the boundaries:
Fix the failing cancel-shipment contract test.
Rules:
- Do not weaken or delete tests.
- Do not change the public API contract.
- Do not modify authentication, routing, or payment logic.
- Prefer the smallest production code change that explains the failure.
Before editing:
- Explain the likely cause.
- List the files you plan to inspect.
- Run the focused test first.The main risk at this level is overfitting. The agent may make a test pass by changing the test, bypassing validation, loosening an assertion, or adding a special case that only handles the test data. That is not a real fix.
A useful guard is to compare behavior before and after:
npm run test -- cancel-shipment.contract.spec.ts
npm run test -- orders.service.spec.ts
git diff -- tests srcThe engineer should review whether the fix addresses the domain issue or simply satisfies the failing test. Terminal autonomy is valuable only when the agent is forced to validate through the same build and test process that humans use.
5.6 Level 6: Local Multi-Agent Parallelism
At Level 6, the developer runs multiple agents at the same time on separate branches or worktrees. One agent may work on API changes, another on UI updates, another on tests, and another on documentation. The developer shifts from implementer to local git integrator and branch coordinator.
This approach works best when the work can be split cleanly. For example, suppose the team is adding shipment cancellation reason capture across a full stack application. The work can be separated into backend contract changes, frontend form changes, regression tests, and release notes.
A practical setup uses git worktrees:
git worktree add ../agent-api ai/api-cancellation-reason
git worktree add ../agent-ui ai/ui-cancellation-reason
git worktree add ../agent-tests ai/tests-cancellation-reason
git worktree add ../agent-docs ai/docs-cancellation-reasonEach agent receives a different assignment:
parallel_agents:
api_agent:
branch: "ai/api-cancellation-reason"
objective: "Add cancellationReason to API request handling"
owns:
- "src/api/orders/**"
- "src/domain/orders/**"
ui_agent:
branch: "ai/ui-cancellation-reason"
objective: "Add cancellation reason field to Angular form"
owns:
- "src/app/orders/**"
qa_agent:
branch: "ai/tests-cancellation-reason"
objective: "Add regression tests for cancellation reason"
owns:
- "tests/orders/**"
docs_agent:
branch: "ai/docs-cancellation-reason"
objective: "Update release notes and API usage documentation"
owns:
- "docs/**"This works only if contracts are stable before the agents begin. If the API agent names the field reasonCode and the UI agent names it cancellationReason, merge work becomes messy. Before parallel execution, define shared decisions such as field names, enum values, validation rules, error codes, and event names.
The failure mode at Level 6 is conflicting assumptions. The solution is contract-first coordination.
5.7 Level 7: Hand-Managed Swarms
At Level 7, a human manages many agents manually. This can be useful for short, intense bursts of work, such as modernizing a module, adding tests across many services, or cleaning up a large migration. But the human coordination burden becomes significant.
The issue is not whether ten agents can produce code. They can. The issue is whether ten outputs combine into one coherent system. Without structure, the developer becomes a merge-conflict clerk, reviewing overlapping changes, reconciling naming differences, and trying to remember which agent made which assumption.
A hand-managed swarm needs strong ownership boundaries:
swarm_assignments:
api_agent:
owns:
- "src/api/orders/**"
cannot_touch:
- "src/auth/**"
- "infra/**"
domain_agent:
owns:
- "src/domain/orders/**"
cannot_touch:
- "src/payments/**"
migration_agent:
owns:
- "db/migrations/**"
requires_approval_for:
- "drop column"
- "rename table"
- "data backfill"
qa_agent:
owns:
- "tests/orders/**"
cannot:
- "delete existing tests"
- "weaken existing assertions"
docs_agent:
owns:
- "docs/**"A practical use case is test expansion. Ten agents can each cover a different service or module. That is safer than letting ten agents all edit the same domain layer.
Example assignment:
Agent 1: Add missing tests for OrderService cancellation rules.
Agent 2: Add missing tests for ShipmentService carrier handoff rules.
Agent 3: Add missing tests for RefundService refund eligibility rules.
Agent 4: Add missing tests for AuditService event creation.
Agent 5: Update API contract tests for cancellation endpoints.Level 7 is useful, but fragile. It depends heavily on the human’s ability to coordinate scope, review outputs, and stop agents that drift. Once the management overhead becomes larger than the work itself, the team needs Level 8 orchestration rather than more manual supervision.
5.8 Level 8: Hierarchical System Orchestration
Level 8 introduces a structured hierarchy. A master orchestrator receives the goal, breaks it into work packages, delegates to manager agents, and coordinates specialized worker agents. The important part is not the number of agents. The important part is the management layer.
At this level, agentic engineering starts to look like distributed systems design. The system needs state, ownership, locking, policy enforcement, conflict detection, observability, rollback, and verification gates. Without those controls, a “swarm” is just parallel chaos.
A simplified work package model might look like this:
from dataclasses import dataclass, field
from typing import Literal
Status = Literal["planned", "assigned", "in_progress", "blocked", "verified", "merged"]
@dataclass
class WorkPackage:
id: str
owner: str
objective: str
files: list[str]
status: Status = "planned"
depends_on: list[str] = field(default_factory=list)
def can_assign(package: WorkPackage, active_packages: list[WorkPackage]) -> bool:
active_files = {file for p in active_packages for file in p.files}
has_file_conflict = any(file in active_files for file in package.files)
return not has_file_conflict
packages = [
WorkPackage(
id="api-1",
owner="api-manager",
objective="Add cancellation reason to backend API",
files=["src/api/orders/cancel.ts", "src/domain/orders/order.ts"]
),
WorkPackage(
id="ui-1",
owner="ui-manager",
objective="Add cancellation reason to Angular form",
files=["src/app/orders/cancel-shipment.component.ts"],
depends_on=["api-1"]
),
WorkPackage(
id="qa-1",
owner="qa-manager",
objective="Add contract and regression tests",
files=["tests/orders/cancel-shipment.contract.spec.ts"],
depends_on=["api-1", "ui-1"]
)
]This example is small, but the idea is important. The orchestrator should not allow two agents to modify the same critical file at the same time. It should not allow UI work to proceed before the API contract is stable. It should not mark the work complete until tests and review gates pass.
A Level 8 workflow might look like this:
level_8_workflow:
master_orchestrator:
input: "Implement shipment cancellation reason across the platform"
responsibilities:
- "Break goal into work packages"
- "Assign packages to manager agents"
- "Track dependencies"
- "Prevent file conflicts"
- "Enforce verification gates"
manager_agents:
api_manager:
delegates_to:
- "api_worker"
- "domain_worker"
ui_manager:
delegates_to:
- "angular_worker"
- "accessibility_worker"
qa_manager:
delegates_to:
- "unit_test_worker"
- "contract_test_worker"
verification_gates:
- "Type checks must pass"
- "Unit tests must pass"
- "Contract tests must pass"
- "No protected files modified without approval"
- "Final diff summary required"The practical advice is simple: do not jump to Level 8 because it sounds advanced. Earn it. Get Level 3 and Level 4 working first. Then introduce terminal agents. Then introduce parallel worktrees. Then introduce managed swarms. Only after that does hierarchical orchestration make sense.
The teams that succeed with Level 8 will not be the teams with the most prompts. They will be the teams that treat agent orchestration as architecture: clear state, clear ownership, clear policy, clear verification, and clear accountability.
6 Real-World Architecture: Implementing a Level 8 Multi-Agent Swarm
Level 8 only becomes useful when orchestration is treated as a software architecture problem. A swarm is not a group of agents running at the same time. It is a controlled execution system where work is decomposed, assigned, verified, merged, and audited.
The practical target is simple: let specialized agents move quickly without allowing them to corrupt shared state, overwrite each other’s work, or make hidden architecture decisions. That requires a factory model. The factory receives a goal, breaks it into work packages, assigns ownership, enforces checkpoints, and refuses unsafe changes.
6.1 Designing the Factory: Goal Decomposition and State Management
A Level 8 system starts with goal decomposition. The master orchestrator should not hand a broad instruction directly to ten worker agents. It should first convert the goal into bounded work packages with clear owners, dependencies, file access rules, and success criteria.
Example input:
Production issue:
The billing API intermittently returns 504 errors when generating monthly account summaries.
Known facts:
- Failures started after release 2026.06.14.
- Error appears under high account volume.
- Stack trace points to SummaryProjectionService.
- No database outage was reported.The orchestrator should turn that into separate work packages:
work_packages:
investigation:
owner: auditor_manager
objective: "Identify likely regression source"
allowed_paths:
- "src/billing/**"
- "logs/**"
- "tests/billing/**"
patch:
owner: coding_manager
objective: "Implement the smallest safe fix"
depends_on:
- investigation
allowed_paths:
- "src/billing/summary/**"
- "tests/billing/summary/**"
validation:
owner: qa_manager
objective: "Prove the patch fixes timeout behavior without weakening tests"
depends_on:
- patch
allowed_paths:
- "tests/billing/**"
- "performance/**"The key is that every package has ownership and boundaries. Two agents should not edit the same service at the same time unless the orchestrator explicitly allows it. Race conditions in multi-agent systems often look like normal merge conflicts, but the deeper issue is conflicting reasoning. One agent optimizes query performance while another changes the projection model, and the final system becomes internally inconsistent.
A state register helps prevent this. The register is append-only. Agents do not overwrite each other’s conclusions. They add observations, decisions, and artifacts.
from dataclasses import dataclass, field
from datetime import datetime
from typing import Literal
Status = Literal["planned", "running", "blocked", "verified", "rejected"]
@dataclass(frozen=True)
class StateEvent:
package_id: str
actor: str
event_type: str
message: str
timestamp: datetime = field(default_factory=datetime.utcnow)
@dataclass
class WorkState:
package_id: str
status: Status
owned_paths: list[str]
events: list[StateEvent] = field(default_factory=list)
def append_event(self, actor: str, event_type: str, message: str) -> None:
self.events.append(StateEvent(self.package_id, actor, event_type, message))This looks basic, but it enforces an important rule: the system remembers how it reached a decision. That matters when the patch fails later and the team needs to understand which agent made which assumption.
Verification checkpoints should also be deterministic. An agent should not mark its own work complete just because it believes the code is correct. Completion should require objective checks.
verification_checkpoints:
investigation:
required:
- "Regression commit identified or ruled out"
- "At least one reproducible failure path documented"
patch:
required:
- "No public API contract change"
- "No database schema change"
- "Focused tests pass"
validation:
required:
- "Timeout regression test added"
- "Performance test result attached"
- "Final diff reviewed by human owner"This is the difference between agentic automation and uncontrolled generation. The agent can act, but the factory decides whether the action is accepted.
6.2 Open-Source Ecosystem and Tooling Frameworks
There is no single framework that solves multi-agent engineering by itself. The better approach is to combine tools by responsibility: orchestration, role execution, context retrieval, model routing, and verification.
6.2.1 Orchestration Frameworks
LangGraph is a good fit when the workflow needs explicit state transitions, branching, and deterministic control. For example, a production patch pipeline should not move from investigation to patching unless the investigation node produces enough evidence. A graph structure makes that rule visible.
A simplified LangGraph-style flow might look like this:
from typing import TypedDict, Literal
class PatchState(TypedDict):
incident_id: str
status: Literal["new", "investigated", "patched", "validated", "rejected"]
suspected_files: list[str]
patch_summary: str
test_results: str
def investigate(state: PatchState) -> PatchState:
state["suspected_files"] = [
"src/billing/summary/SummaryProjectionService.cs",
"tests/billing/SummaryProjectionServiceTests.cs"
]
state["status"] = "investigated"
return state
def patch(state: PatchState) -> PatchState:
state["patch_summary"] = "Replace per-account sequential query with batched projection query."
state["status"] = "patched"
return state
def validate(state: PatchState) -> PatchState:
state["test_results"] = "unit=pass; integration=pass; perf=within-threshold"
state["status"] = "validated"
return stateCrewAI is useful when role clarity matters. You can model an auditor, patch engineer, QA validator, release-note writer, and security reviewer as separate agents with different goals and tool access. That maps well to how engineering teams already divide responsibility.
AutoGen is useful when the work is naturally conversational: one agent proposes a plan, another critiques it, a third executes, and a human can interrupt. That pattern is valuable for design review, migration planning, and incident analysis where the answer benefits from structured debate.
The trade-off is complexity. Do not introduce a multi-agent framework for a task that a single well-tooled agent can complete. Use orchestration when the problem has real dependency management, separate responsibilities, or high verification needs.
6.2.2 Context and Memory Pipelines
A swarm without good context will produce confident but shallow work. The context layer should retrieve architecture records, code ownership, dependency maps, recent commits, runbooks, incident logs, and relevant tests.
LlamaIndex can be used to build retrieval over internal engineering documents and code metadata. The goal is not to dump every document into the prompt. The goal is to retrieve the few pieces that change the decision.
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
documents = SimpleDirectoryReader(
input_dir="engineering_context",
recursive=True
).load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=5)
response = query_engine.query(
"What are the performance constraints for monthly billing summary generation?"
)
print(response)LiteLLM fits a different layer: model routing and cost control. A production swarm should not hardcode one model everywhere. It should route expensive reasoning tasks to stronger models and simple summarization or formatting tasks to cheaper models.
model_routing:
auditor_agent:
model: "frontier-reasoning"
max_budget_usd: 2.00
patch_coder_agent:
model: "strong-coding"
max_budget_usd: 3.50
qa_validator_agent:
model: "balanced-coding"
max_budget_usd: 1.25
release_notes_agent:
model: "cost-efficient"
max_budget_usd: 0.20This makes cost visible at the architecture level instead of discovering it after several expensive agent runs.
6.3 Concrete Implementation Scenario: An Automated Bug Triage and Production Patch Pipeline
A realistic Level 8 pipeline starts with an incident, not a vague coding request. For example, the production monitoring system reports repeated database timeout errors in a .NET billing service.
6.3.1 The Master Orchestrator
The master orchestrator ingests the exception report, extracts structured signals, and builds the workflow topology. It does not write the fix. It decides who should investigate, who can patch, and what must be proven before a pull request is opened.
{
"incident_id": "INC-2026-0619-042",
"service": "billing-api",
"error": "SqlException: Timeout expired",
"entry_point": "GET /api/billing/accounts/{id}/summary",
"suspected_component": "SummaryProjectionService",
"first_seen": "2026-06-19T04:12:00Z",
"severity": "high"
}The orchestrator assigns investigation first. Patch work is blocked until the auditor provides a suspected cause.
6.3.2 The Manager Agents
Manager agents coordinate planning. The auditor manager retrieves recent commits, stack traces, service ownership, and related runbooks. The coding manager waits for evidence before allowing edits. The QA manager prepares validation strategy early so the patch is judged against the right behavior.
git log --oneline --since="7 days ago" -- src/billing
rg "SummaryProjectionService|AccountSummary|Timeout" src tests logs
dotnet test tests/Billing.Tests --filter SummaryProjectionThis is where managers prevent waste. If the failure is caused by a database index regression, the patch coder should not rewrite the API controller. If the failure is caused by an N+1 query, QA should add a volume-oriented test rather than only a happy-path unit test.
6.3.3 The Worker Agents
Worker agents operate inside sandboxes. The auditor identifies the regression. The patch coder edits only approved files. The QA validator adds tests and runs verification. No worker gets broad production access.
FROM mcr.microsoft.com/dotnet/sdk:8.0
WORKDIR /workspace
COPY . .
RUN dotnet restore
CMD ["bash"]A safe patch instruction might be:
Patch Coder Agent:
Fix the timeout in SummaryProjectionService.
Rules:
- Edit only src/billing/summary and tests/billing/summary.
- Do not change API response shape.
- Do not add new infrastructure dependencies.
- Prefer batching or projection changes over increasing command timeout.
- Add a regression test proving the high-volume path completes.The QA validator then runs the build and rejects weak fixes.
dotnet test tests/Billing.Tests --filter SummaryProjection
dotnet test tests/Billing.IntegrationTests --filter AccountSummary
dotnet format --verify-no-changesThe final pull request should include the incident ID, files changed, tests run, risk level, and rollback note. That is how a multi-agent swarm becomes operationally reviewable.
7 Operational Realities: Cost, Security, and Governance
The technical architecture is only half the system. The operating model decides whether agentic engineering is sustainable. Cost can grow quietly, security risk can hide inside tool access, and daily product announcements can distract teams from actual delivery leverage.
7.1 Breaking Down the Token Cost Curve
Multi-agent systems consume tokens in layers: planning, retrieval, tool output, code generation, test failure analysis, patch revision, and summarization. The expensive part is usually not the final answer. It is the repeated loop of reading context, trying a change, reading errors, and trying again.
A simple cost ledger helps teams see what is happening.
from dataclasses import dataclass
@dataclass
class AgentCost:
agent: str
input_tokens: int
output_tokens: int
cost_usd: float
runs = [
AgentCost("auditor", 82000, 9000, 1.84),
AgentCost("patch_coder", 64000, 18000, 2.35),
AgentCost("qa_validator", 41000, 7000, 0.92),
]
print(sum(run.cost_usd for run in runs))Cost optimization should not mean always choosing the cheapest model. It means using the right model at the right stage. Expensive models are often worth it for root-cause analysis and architecture-sensitive patches. Cheaper models are usually fine for release notes, repetitive test scaffolding, formatting, and summarizing logs.
Local open-weight models can help for low-risk tasks, especially when paired with tools such as llama.cpp. A team can start by using local models for code search summarization, documentation cleanup, or first-pass test naming. Commercial tools can then be reserved for deeper reasoning or high-value coding work.
7.2 Defensive Architecture and Quality Guardrails
Autonomous tool execution should be treated as untrusted by default. The agent may be useful, but its commands should run in a controlled environment. It should not have unrestricted access to the host machine, personal files, production credentials, or shared developer secrets.
A defensive setup uses containers, read-only mounts where possible, restricted environment variables, and explicit network rules.
docker run --rm -it \
--name agent-runner \
--network none \
-v "$PWD:/workspace" \
-w /workspace \
agent-dotnet-runner bashFor some tasks, the network must be enabled to restore packages. In that case, separate dependency restore from agent execution. Build the image first, then run the agent in a locked container.
Quality gates should run before code is committed. AST checks are useful because they inspect structure, not only formatting. For example, a TypeScript guard can reject changes that introduce direct access to localStorage inside secure modules.
import ts from "typescript";
import fs from "fs";
const fileName = "src/security/session.service.ts";
const source = ts.createSourceFile(
fileName,
fs.readFileSync(fileName, "utf8"),
ts.ScriptTarget.Latest,
true
);
function visit(node: ts.Node) {
if (
ts.isPropertyAccessExpression(node) &&
node.expression.getText(source) === "window" &&
node.name.getText(source) === "localStorage"
) {
throw new Error("Direct localStorage access is not allowed in session service.");
}
ts.forEachChild(node, visit);
}
visit(source);This is the kind of guardrail agents respect better than prose. A prompt says “do not do this.” A gate says “this cannot merge.”
7.3 Separating Signal from Noise
AI engineering tools change quickly, but not every announcement deserves architecture attention. A senior team needs a filter. The filter should be based on measurable leverage, not novelty.
A simple evaluation framework is enough:
tool_evaluation:
production_fit:
- "Does it work with our repo size?"
- "Can it run inside our security model?"
- "Does it support audit logs or exportable run history?"
engineering_value:
- "Does it reduce cycle time for real tasks?"
- "Does it improve test coverage or defect detection?"
- "Does it reduce review burden without reducing quality?"
operational_risk:
- "What data leaves our environment?"
- "Can we restrict tools and permissions?"
- "Can we disable or roll back the integration quickly?"
adoption_cost:
- "Will developers need a new IDE?"
- "Does it fit CI/CD?"
- "Can it be piloted on one repository first?"The best pilot is not a demo project. Use a real but bounded workflow: flaky test repair, documentation from code changes, API client generation, migration test coverage, or production log triage. Measure baseline time, agent-assisted time, defects introduced, review effort, and rework.
The practical end state is not “AI everywhere.” It is selective autonomy. Give agents the work where they create measurable leverage. Keep humans in control of architecture, risk, accountability, and final acceptance. That is how agentic engineering moves beyond vibe coding and becomes a disciplined delivery model.
