Last month, developers using large language models noticed a familiar frustration: every fresh turn in a long session triggered a full rerun of the entire conversation history. Even after changes were confirmed or old data became irrelevant, the client still resent every prior input, token by token, driving up costs with redundant data. This isn’t a flaw in the model—it’s the stateless nature of most LLM APIs, where context is reconstructed from scratch with each request.
To tackle this, a new open-source tool called PromptCrunch acts as a lightweight proxy between your client and the LLM provider. Instead of sending raw conversation history, it rewrites each request in real time. It removes superseded code snippets, compresses outdated tool outputs, and summarizes older turns while preserving recent structured data exactly as required. The proxy only regenerates a request when doing so actually reduces the total token count, ensuring savings without altering the model’s output.
Setup takes seconds. Developers can redirect their API base URL and add a single header to start using PromptCrunch immediately. The tool works with any LLM provider—Claude Code was simply where the creator first encountered the issue. The same problem appears in agent workflows, customer support bots, and long-form conversational apps, where each new prompt resends growing chat histories.
How is this different from prompt caching?
Prompt caching reduces costs by discounting repeated prefixes, but its benefits expire quickly. Most caches clear after about 5 minutes of inactivity, leaving long sessions exposed to full token charges once the cache cools. Real-world usage is bursty: you write code, review changes, step away, then return hours later. In those gaps, the cache expires, and every resumed session resends all prior turns, triggering a fresh billing spike.
In tests with PromptCrunch, users saw input token usage drop by about 75% during cold-cache gaps. When caching was active, savings averaged 7 to 10%. Together, the two layers address different parts of the session: caching handles the hot window, while PromptCrunch manages the long tail. This combination shifts billing from turn count to actual work performed, making long sessions significantly more affordable.
What savings can I expect?
The tool delivers the strongest value in extended, iterative workflows where context builds over dozens or hundreds of turns. Short prompts or one-off requests see minimal benefit, so developers should target sessions expected to run for several minutes or longer. PromptCrunch offers a zero-cost trial: point it at a single real session and monitor per-request savings on its dashboard. New users receive $5 of free credits with no payment details required, and the tool includes a zero-retention mode for privacy-conscious teams.
PromptCrunch is designed to integrate seamlessly into existing setups without vendor lock-in. Keys are never stored, and the proxy forwards all traffic directly to the chosen LLM provider. For engineering teams managing agentic workflows or customer-facing chatbots, this approach can turn long sessions from cost centers into predictable expenses aligned with real productivity.
AI summary
Uzun LLM oturumlarında gizli token maliyetlerini PromptCrunch ile azaltın. Kolay kurulum, %75'e varan tasarruf ve sıfır veri saklama seçeneğiyle.
