Originally published on tamiz.pro.

The current state of AI-assisted development is largely defined by autocomplete. Tools like GitHub Copilot and Cursor have revolutionized the speed at which we write code, but they remain passive assistants. They suggest; we decide. They complete; we verify. However, the next frontier of software engineering is not just faster typing—it is autonomous agency. We are moving from "AI that helps me write code" to "AI that verifies and ships features."

Building a pipeline where an AI agent does not just generate tokens, but actually engineers a feature end-to-end, requires a fundamental shift in architecture. It demands moving away from trust-based integration ("I hope this works") to evidence-based integration ("Here is the test that proves it works"). This deep dive explores the architectural patterns required to build these systems: Test-Driven Development (TDD) as a verification engine, LLM-based review guardrails to prevent hallucination-induced bugs, and multi-agent coordination to handle complex system design.

1. The Limitations of "Vibe Coding"

"Vibe coding"—writing software based on intuition and rapid AI generation without rigorous verification—works well for prototypes. It fails catastrophically in production. The core problem is that Large Language Models (LLMs) are probabilistic, not deterministic. They do not understand state; they predict the next likely token. An agent that simply writes a UserService and a UserRepository is operating in a vacuum. It has no feedback loop.

To build a production-grade pipeline, we must close the loop. The agent must not stop after code generation. It must stop only when the evidence is irrefutable. This evidence comes from three sources:

  1. Behavioral Evidence: Unit and integration tests that pass.
  2. Static Evidence: Linters, type checkers, and security scanners that pass.
  3. Logical Evidence: A review agent that verifies the logic aligns with the requirements.

2. TDD as a State Machine for Agents

Traditionally, Test-Driven Development is a human discipline. It forces you to write the test before the implementation, ensuring the code is actually testable. For AI agents, TDD is not a discipline; it is a control flow mechanism.

In an autonomous pipeline, the "Test" is the acceptance criterion. If the agent cannot write a failing test that clearly describes the requirement, it does not understand the requirement.

The TDD Loop in Code

Imagine a CodeAgent class. It does not have a single generate method. It has a refactor, implement, and verify lifecycle.

class FeatureAgent: def __init__(self, llm: BaseLLM, file_system: LocalFS, test_runner: ShellRunner): self.llm = llm self.fs = file_system self.test_runner = test_runner def execute_tdd_cycle(self, requirement: str): """ Executes a TDD cycle: Red -> Green -> Refactor. """ # 1. RED: Generate a failing test  test_code = self._generate_test(requirement) self.fs.write_file("tests/test_feature.py", test_code) # Verify it actually fails (sanity check)  result = self.test_runner.run("pytest tests/test_feature.py") if not result.has_failures(): raise RuntimeError("Test passed immediately. Requirement is trivial or test is flawed.") # 2. GREEN: Implement the code to pass the test  implementation = self._generate_implementation(requirement, test_code) self.fs.write_file("src/feature.py", implementation) # 3. VERIFY  result = self.test_runner.run("pytest tests/test_feature.py") if not result.all_passed(): # Feed error back into LLM for correction  return self._auto_fix(result.output) # 4. REFACTOR (Optional)  return True

The critical component here is the test_runner. The agent is not guessing that the code works. It is executing it. This shifts the architecture from a "Black Box" (input prompt -> output code) to a "Reinforcement Loop" (input prompt -> output code -> environment feedback -> error correction).

3. Review Guardrails: The LLM Critic

Even if the tests pass, the code might be bad. It might use deprecated APIs, ignore security best practices, or violate architectural patterns (e.g., a controller talking to the database). This is where the "Guardrail" agent comes in.

In this pipeline, we treat the LLM as a reviewer, not a coder. We use a distinct prompt context that focuses on critique. This is often called the "LLM-as-a-Judge" pattern.

Architecting the Critic

The Critic agent should be stateless and fast. It reads the diff generated by the TDD agent and outputs a structured JSON verdict.

{ "severity": "high", "issue": "SQL Injection vulnerability", "location": "src/db.py:42", "suggestion": "Use parameterized queries instead of string interpolation." }

By forcing the Critic to output structured data, we make the pipeline deterministic. The main orchestrator can then parse this JSON. If severity is "high", the pipeline halts and feeds the suggestion back to the TDD agent. This creates a "Constitutional AI" constraint, where the agent is bound by rules defined in the Critic's prompt.

4. Multi-Agent Coordination: The Orchestrator

A single agent working in a monolithic prompt context will quickly hit the limits of its context window and become confused. Complex features (e.g., "Add a Stripe billing system") require separation of concerns.

We move to a Supervisor-Worker architecture.

  1. The Architect Agent: Reads the high-level requirement and breaks it down into a task list (e.g., "1. Create Stripe Client Wrapper", "2. Add Webhook Handler", "3. Update Database Schema").
  2. The Workers: Specialized agents (DB Agent, API Agent, Test Agent) that handle specific parts of the task.
  3. The Orchestrator: A deterministic Python/TypeScript script that manages the DAG (Directed Acyclic Graph) of tasks. It does not use an LLM for control flow. It uses the LLMs only for the work items.

Why Deterministic Orchestration Matters

You might think: "Let the LLM decide which agent to call next." This is a recipe for infinite loops and unpredictable costs. The LLM should not be the conductor; it should be the player. The conductor is your code.

async function runPipeline() { // 1. Plan const plan = await architectAgent.plan(featureDescription); // 2. Execute Tasks Sequentially or in Parallel for (const task of plan.tasks) { if (task.dependsOn.length === 0) { await workerAgent.execute(task); } else { // Check dependencies are met if (task.dependsOn.every(id => completedTasks.has(id))) { await workerAgent.execute(task); } } } // 3. Final Review const diff = git.getDiff(); const review = await criticAgent.review(diff); if (review.blockers > 0) { // Trigger a fix loop await workerAgent.fix(review.issues); } }

5. Implementing the Pipeline: A Concrete Example

Let's look at how this actually works in a real-world scenario. We are adding a "Forgot Password" endpoint to a Node.js/Express app.

Step 1: The Architect

Prompt: "Design a system for password reset tokens. Consider expiration (24h), security (SHA-256 hashing), and database schema changes."

Output (Architect Agent):

{ "tasks": [ { "id": "db", "desc": "Add 'reset_token_hash' and 'reset_expires_at' to users table" }, { "id": "logic", "desc": "Create generateToken service" }, { "id": "api", "desc": "Create /auth/forgot-password endpoint" } ] }

Step 2: The TDD Worker (Database)

Prompt: "Write a test that ensures the database migration adds the new columns. Then write the migration file."

The worker writes tests/migration.test.ts:

test('db schema update', async () => { const schema = await getSchema(); expect(schema.tables.users.columns).toContain('reset_token_hash'); });

It runs the test. It fails (Red). It writes the migration. It runs the test. It passes (Green).

Step 3: The TDD Worker (Logic)

Prompt: "Implement the token generation logic based on the test in tests/token.test.ts."

The worker generates the logic, ensuring the token is hashed with SHA-256 before storage.

Step 4: The Critic

Prompt: "Review the diff for the last two commits. Check for: 1. Is the token stored in plain text? 2. Is the expiration time respected in the query? 3. Are SQL injections possible?"

The Critic analyzes the code. It finds that the SELECT query in the API endpoint uses string interpolation for the expiration check. It flags this as a high severity issue.

Step 5: The Orchestrator

The orchestrator sees the high severity flag. It takes the Critic's suggestion: "Use parameterized queries." It passes this back to the API Worker. The Worker updates the code to use ? placeholders. The Critic re-reviews. It passes. The pipeline commits the changes.

6. Handling Edge Cases and Failure Modes

Building these pipelines is not plug-and-play. There are specific failure modes that will break your automation.

Context Rot

As the agent makes more changes, the context window fills with previous code. The agent begins to "forget" the requirements from the beginning of the prompt.

  • Solution: Sliding Window Memory. Summarize the conversation history into a "Current State" object. Pass this state into the next prompt, rather than the raw history.

The "Hallucinated Dependency" Problem

The agent writes import { RedisClient } from 'redis' but the project uses ioredis.

  • Solution: The TDD step must include a npm install or pip install check, or better yet, the Critic must have access to the package.json / requirements.txt and verify imports against the actual dependencies.

Infinite Loops

The agent fails the test, tries to fix it, fails again, and gets stuck in a cycle.

  • Solution: Circuit Breakers. Limit the number of retries (e.g., 3 attempts). If it fails 3 times, stop and raise a HumanInterventionRequired exception.

7. Security Considerations

Giving an AI agent write access to your codebase and execution privileges is terrifying.

  • Sandboxing: Always run the agent in an ephemeral Docker container. No agent should touch the developer's local environment directly.
  • Least Privilege: The agent should have read access to the codebase and write access to a specific branch or a temporary copy. It should not have push access to main. The final commit should be a PR generated by the pipeline, reviewed by a human, and merged manually (or via strict CI checks).
  • Prompt Injection: If the agent reads external data (like GitHub Issues or StackOverflow answers) and feeds it into the LLM, it is vulnerable to prompt injection. Sanitize all external inputs before they reach the LLM context.

8. The Future: Agentic SWE

We are currently in the early days of this. The "Agentic SWE" (Software Engineering) pipeline is the bridge to fully autonomous development. However, the bottleneck is not the LLM's intelligence; it is the LLM's reliability.

By wrapping the LLM in a TDD loop, we are essentially using the compiler and the test runner as our guardrails. The LLM is the "brain," but the test runner is the "safety net."

For now, do not try to build a general-purpose agent that can do everything. Build specialized agents that do one thing well, and coordinate them with a deterministic orchestrator. This is the path to software that is not just written by AI, but verified by it.

Frequently Asked Questions

How much context window do I need for these agents?

For the TDD Worker, a 16k-32k context window is usually sufficient to hold the requirement, the current file, and the error output. For the Critic, you need enough space to hold the entire diff plus the project's style guide. 32k-128k is recommended for complex reviews.

Should I use a different LLM for the Critic than the Coder?

Yes. Using a different model (or even the same model with a temperature of 0.0 and a highly specific "judge" persona) prevents the agent from "rubber-stamping" its own work. A second opinion is always more valuable than a confirmation bias.

How do I handle multi-file changes?

The orchestrator must decompose the task. The TDD agent should not be asked to rewrite the whole backend. It should be asked to "Update the User model to support soft deletes" and then "Update the UserService to respect soft deletes". This keeps the context small and the diffs reviewable.

For more insights on AI engineering and developer tooling, check out Tamiz's Insights.

Multi-Agent Coordination Patterns

While a single agent can handle focused tasks, real-world codebases demand specialization. The most effective autonomous pipelines employ multi-agent architectures where distinct agents own different concerns.

The Role-Based Agent Mesh 

┌─────────────────────────────────────────────────────────┐ │ Orchestrator Agent │ │ (Task decomposition, state management, conflict res.) │ └──────────────┬──────────────────────────────────────────┘ │ ┌──────────┼──────────────┬──────────────────┐ ▼ ▼ ▼ ▼ ┌────────┐ ┌────────┐ ┌───────────┐ ┌────────────┐ │Coder │ │Tester │ │Reviewer │ │Doc Writer │ │Agent │ │Agent │ │Agent │ │Agent │ └────────┘ └────────┘ └───────────┘ └────────────┘

Each agent receives a narrow system prompt with domain-specific context, not the full codebase. The orchestrator manages the DAG of dependencies:

# orchestrator.py from dataclasses import dataclass, field from typing import Literal from enum import Enum class AgentRole(Enum): Coder = "coder" Tester = "tester" Reviewer = "reviewer" DocWriter = "doc_writer" @dataclass class TaskNode: id: str role: AgentRole description: "str" dependencies: list[str] = field(default_factory=list) status: Literal["pending", "in_progress", "completed", "failed"] = "pending" output: str | None = None class Orchestrator: def __init__(self, agents: dict[AgentRole, "BaseAgent"]): self.agents = agents self.task_graph: dict[str, TaskNode] = {} self.execution_order: list[str] = [] def add_task(self, task: TaskNode) -> None: self.task_graph[task.id] = task def topological_sort(self) -> list[str]: """Resolve execution order respecting dependencies.""" visited = set() temp_visited = set() order = [] def visit(node_id: str): if node_id in temp_visited: raise ValueError(f"Circular dependency detected at {node_id}") if node_id in visited: return temp_visited.add(node_id) node = self.task_graph[node_id] for dep in node.dependencies: visit(dep) temp_visited.remove(node_id) visited.add(node_id) order.append(node_id) for node_id in self.task_graph: visit(node_id) return order async def execute(self) -> dict[str, str]: """Execute tasks in dependency order.""" results = {} for task_id in self.topological_sort(): node = self.task_graph[task_id] node.status = "in_progress" # Gather context from dependencies  dep_context = { dep_id: self.task_graph[dep_id].output for dep_id in node.dependencies if self.task_graph[dep_id].output } agent = self.agents[node.role] try: output = await agent.execute( task_description=node.description, context=dep_context ) node.output = output node.status = "completed" results[task_id] = output except Exception as e: node.status = "failed" results[task_id] = f"FAILED: {e}" # Propagate failure to dependent tasks  self._mark_dependents_failed(task_id) return results def _mark_dependents_failed(self, failed_id: str) -> None: for node in self.task_graph.values(): if failed_id in node.dependencies and node.status == "pending": node.status = "failed" node.output = f"SKIPPED: dependency {failed_id} failed"

Inter-Agent Communication Protocol

Agents don't share memory directly—they exchange structured artifacts. This prevents context pollution and makes the pipeline auditable:

# agent_protocol.py from pydantic import BaseModel, Field from typing import Any class Artifact(BaseModel): """Standardized output format between agents.""" task_id: str author_role: str summary: str = Field(..., description="One-sentence summary of what was done") files_modified: list[str] = Field(default_factory=list) tests_added: list[str] = Field(default_factory=list) evidence: dict[str, Any] = Field(default_factory=dict) confidence: float = Field(..., ge=0.0, le=1.0) follow_up_needed: bool = False follow_up_reason: str | None = None class ReviewArtifact(Artifact): """Specialized artifact from the reviewer agent.""" verdict: Literal["approved", "changes_requested", "blocked"] critical_issues: list[str] = Field(default_factory=list) suggestions: list[str] = Field(default_factory=list) security_concerns: list[str] = Field(default_factory=list)

The Tester agent receives the Coder's artifact and produces its own:

# tester_agent.py class TesterAgent(BaseAgent): def __init__(self, model: str = "gpt-4", test_runner: TestRunner | None = None): super().__init__(model=model) self.test_runner = test_runner or LocalTestRunner() async def execute(self, task_description: "str, context: dict[str, str]) -> Artifact:" # Extract the code changes from the coder's output  code_context = context.get("coder_output", "") # Generate tests if none exist  test_plan = await self._generate_test_plan(task_description, code_context) # Write test files  test_files = self._write_tests(test_plan) # Execute the test suite  results = await self.test_runner.run( test_files=test_files, coverage_threshold=80.0 ) return Artifact( task_id=self.current_task_id, author_role="tester", summary=f"Generated {len(test_files)} test files, {results.passed} passed, {results.failed} failed", files_modified=[f.path for f in test_files], tests_added=[t.name for t in test_files], evidence={ "coverage_percent": results.coverage, "pass_rate": results.pass_rate, "failed_tests": [f"{t.name}: {t.error}" for t in results.failures], }, confidence=min(results.pass_rate / 100.0, 1.0), follow_up_needed=results.failed > 0, follow_up_reason=f"{results.failed} tests failing" if results.failed > 0 else None, Source：https://dev.to/tamizuddin/building-autonomous-ai-coding-agent-pipelines-tdd-evidence-guardrails-and-multi-agent-1c22
My favorites
PreviousRunning full n8n on Vercel's free plan (with a free Postgres and your own domain)NextHow can researchers measure the x402 market without using self-reported calls?

Related posts

A Demand-Driven Decision Support Platform for EcoFit Smart Bottle

A Demand-Driven Decision Support Platform for EcoFit Smart Bottle

dev.to
Building a Production RAG Pipeline on AWS: Bedrock Knowledge Bases, OpenSearch Serverless, and Terraform

Building a Production RAG Pipeline on AWS: Bedrock Knowledge Bases, OpenSearch Serverless, and Terraform

dev.to
ollama web search api — the search isn't local

ollama web search api — the search isn't local

dev.to
Projekt Orbitstraße (4): Firmensitz Orbitstraße 1

Projekt Orbitstraße (4): Firmensitz Orbitstraße 1

dev.to
Stop Giving AI Agents More Context Than They Need

Stop Giving AI Agents More Context Than They Need

dev.to
Valve "played capitalism so well they broke it" and now Steam can't be replaced

Valve "played capitalism so well they broke it" and now Steam can't be replaced

Hacker News