Why AI Pipelines Fail: Lessons from 9 Unexpected Bugs in My System

After months of designing what looked like a flawless AI-driven content pipeline, I decided to stress-test it. The results shocked me. Out of six test runs, nine distinct bugs emerged—none of which involved the AI model itself. Every failure originated in the system’s infrastructure, the environment around the model. This is the story of how harness engineering, not the AI’s output, became the real bottleneck.

Building the illusion of autonomy

I constructed a three-phase autonomous pipeline using Claude Code, where each phase operated as a separate AI session:

• Observer: Scanned trending topics, competitor content, and performance metrics to identify opportunities.
• Strategist: Selected a topic, defined the angle, and drafted an outline based on the Observer’s findings.
• Marketer: Expanded the outline into a full article, performed quality checks, and scheduled publication.

Designed on paper, the pipeline seemed bulletproof. The Observer would trigger the Strategist, which would then trigger the Marketer—no human intervention required. Reality, however, had other plans.

Nine failures that had nothing to do with the AI

After six rounds of testing, I documented every failure. They clustered into four broad categories, each exposing flaws in the system’s scaffolding rather than the model’s capabilities.

Execution control: When timing sabotages intent

Two bugs emerged from poor scheduling logic, creating race conditions that undermined the pipeline’s workflow.

• Parallel execution conflict: All three cron jobs fired simultaneously, overwhelming the system. The Marketer started before the Strategist had finished ingesting the Observer’s output, resulting in empty or corrupted inputs. The fix was simple but transformative: replace time-based triggers with event-driven dependencies using after clauses.

• Cron stagger races: Even when staggered (07:00, 07:30, 08:00), the Strategist occasionally exceeded its 30-minute window. The solution remained the same—schedule by completion, not by clock.

Data integrity: When duplication erodes reliability

Three bugs stemmed from inadequate safeguards against redundant operations, undermining the pipeline’s consistency.

• Topic duplication: Without an exclusion list, the Observer repeatedly selected the same trending topic (e.g., "LLMO"). The fix involved injecting a pre-selection filter that excluded already-published articles before topic selection.

• Calendar entry duplication: The system registered calendar events without verifying existing entries, leading to duplicate bookings. The resolution required checking for conflicts before insertion.

• Scheduling conflicts: The auto-scheduler assigned dates already reserved for prior articles, creating gaps in the publishing calendar. A date-availability check ensured new articles were slotted into open slots only.

Quality assurance: When self-assessment fails

Two bugs revealed fundamental flaws in how quality was evaluated—both hinging on flawed assumptions about independence.

• Self-reported quality checks: The AI graded its own work using the same session that generated the content, effectively marking its own homework. The breakthrough came from isolating the quality-check phase in a separate AI session with no contextual memory of the writing process.

• Missing wit check: While the pipeline detected AI-generated slop (repetitive phrases, robotic tone), it overlooked wit—the human spark that transforms competent prose into engaging content. Introducing a dedicated check requiring two instances of wit (e.g., self-deprecation, unexpected metaphors) closed this gap.

Infrastructure: When shell commands betray you

Two bugs exposed vulnerabilities in the pipeline’s underlying commands and job management.

• Bash syntax error from angle brackets: A prompt template used <devto_id> as a placeholder. Bash interpreted the < as an input redirection operator, silently corrupting the command. Escaping or quoting the placeholder resolved the issue.

• `at` job duplication: The scheduler used at for timed publication but failed to check for existing jobs tied to the same article ID. Re-executing the pipeline queued duplicate publish commands, risking overlapping publications. Deleting matching jobs before scheduling new ones eliminated the problem.

The harness is the bottleneck, not the model

None of these failures were model-related. The AI generated coherent content, but the system around it collapsed under its own weight. This pattern aligns with emerging insights from AI engineering circles:

• Prompt engineering focuses on refining inputs to the model.
• Context engineering optimizes the data and tools fed to the model.
• Harness engineering addresses the environment and workflows the model inhabits.

Every bug I encountered fell squarely into the harness category. Industry data supports this observation: Y Combinator’s internal analysis reveals that 40% of AI agent projects fail, with the primary cause rarely being model quality. Public post-mortems from 2026 consistently highlight missing evaluation suites, race-prone queues, and self-destructive retry logic—all harness failures.

One change that transformed the system

The most impactful fix was migrating from time-based cron scheduling to event-driven dependencies. The refactored architecture looked like this:

observer:
  schedule: "0 7 * * 1"  # Runs every Monday at 7 AM

strategist:
  after: observer        # Starts only after Observer completes

marketer:
  after: strategist      # Starts only after Strategist completes

Each phase writes its output to a designated location, and the next phase activates only upon successful completion of the previous one. Failures now halt the pipeline instead of propagating corrupted data. This shift didn’t require better AI—it demanded better engineering around the AI.

A cautionary tale for AI-first systems

The lesson is clear: a brilliant AI model is only as reliable as the system that surrounds it. Before blaming the model for failures, audit the harness. Are dependencies race-free? Are data flows validated? Are quality checks independent? The answers to these questions will determine whether your AI pipeline succeeds—or collapses under its own complexity.