How real LLM traces expose hidden costs and inefficiencies in AI workflows

Observability for large language models (LLMs) remains one of the most underrated challenges in AI infrastructure. Engineers often encounter unsettling surprises—unexplained cost spikes, models returning nonsensical outputs without warnings, or no way to pinpoint which service triggered the surge. These gaps persist even in self-hosted setups, where installation friction and steep learning curves deter quick experimentation.

To bridge this divide, Torrix introduces a zero-setup live demo that surfaces 30 days of real LLM traces across three simulated projects. With no sign-up, containers, or credentials required, users can explore cost anomalies, model inefficiencies, and full agentic workflows in an interactive sandbox. The demo replicates production-grade complexity—mixed model routing, agentic pipelines, and real-time cost tracking—while ensuring all data remains read-only for safe exploration.

Inside the 30-day trace dataset

The dataset spans three simulated environments, each modeled after common real-world use cases:

• Production API: Handles live user requests using GPT-4o and Claude Sonnet 3.5. This environment captures high-volume, unpredictable traffic patterns.
• Data Pipeline: Executes batch summarization tasks with GPT-4o-mini handling the bulk of processing. Ideal for evaluating throughput and cost efficiency at scale.
• Customer Support Bot: Routes queries dynamically—using Haiku for simple issues and Sonnet for complex ones. Demonstrates multi-model cost optimization in action.

Across these projects, the dataset logs 640 distinct runs involving five models, with complete token-level cost calculations and trace metadata. Every interaction is timestamped, and all prompts are preserved for forensic review.

Revealing cost spikes and hidden inefficiencies

One of the demo’s most eye-opening features is its ability to uncover latent cost anomalies without manual digging. On days 14 and 15, the system recorded a threefold spike in daily requests—55 queries compared to the baseline of 18. Each outlier is automatically flagged with a SPIKE badge, allowing users to inspect the exact prompt, selected model, token breakdown, and response in a single click.

Beyond traffic surges, the demo highlights model-level inefficiencies that often go unnoticed. In the Production API, Claude Sonnet 3.5 handles 35% of traffic at a rate of $3.00 per million input tokens and $15.00 per million output tokens. Meanwhile, GPT-4o-mini—at $0.15/$0.60 per million tokens—is 20 times cheaper despite handling 20% of requests. The Analytics tab displays this breakdown instantly, eliminating the need for manual exports or SQL queries.

Tracing agent workflows from start to finish

LLM applications rarely operate in isolation. The demo includes a five-step agentic pipeline that mirrors real-world orchestration:

• Orchestrator: Routes incoming requests based on complexity and available models.
• Researcher: Gathers contextual data or external knowledge before synthesis.
• Synthesizer: Compiles findings into structured outputs.
• Formatter: Converts results into user-friendly formats.
• Validator: Checks for factual accuracy, completeness, or policy compliance.

Each step logs execution time, input prompts, model responses, and token consumption. Engineers can examine the full reasoning chain in a single view, making it easier to debug multi-agent systems and validate end-to-end behavior.

Performance benchmarks and live SQL access

The demo also includes evaluation results across three test datasets to assess agent reliability:

• Capital Cities Quiz: 70% pass rate
• Customer FAQ: 87.5% pass rate
• Email Classification: 75% pass rate

Failed runs are displayed side by side with expected outputs, enabling rapid root-cause analysis. For advanced users, a built-in SQL interface allows direct querying of the underlying SQLite database. Need to find the most expensive models? Run:

SELECT model, COUNT(*) AS runs, SUM(cost_usd) AS total_cost
FROM runs
GROUP BY model
ORDER BY total_cost DESC;

Results can be exported to CSV or browsed via the integrated schema explorer—no external tools required.

How the live demo stays consistent and secure

The demo runs in a controlled environment powered by Fly.io, resetting on every deployment to prevent data drift. It is enabled using the TORRIX_DEMO=true environment flag, which seeds a pre-populated SQLite database at startup. All write endpoints return HTTP 403, ensuring data remains immutable for exploration.

For teams ready to deploy their own instance, Torrix offers a lightweight Docker container with zero external dependencies. Deployment is as simple as:

docker run -d -p 8088:8088 -v torrix_data:/data torrixai/torrix:latest

Whether you're debugging a surprise bill or optimizing a multi-model agent system, observability should never be a barrier to action. The live demo removes setup friction, turning opaque LLM operations into transparent, actionable insights.