Why a Cloudflare engineer chose Gemma 4 MoE after a model shutdown

A sudden deprecation notice from Cloudflare in early May forced one developer to rethink a 100,000-document knowledge engine in just 22 days. The system relied on a model slated for removal, and the recommended replacement promised higher costs and uncertain reasoning depth. After testing, the developer chose Gemma 4 MoE—a mixture-of-experts model that delivers both edge efficiency and multi-document synthesis without leaving the platform.

A production pipeline disrupted by a model shutdown

On May 8, Cloudflare announced the deprecation of @cf/moonshot/kimi-k2.5, the model powering a personal knowledge engine built by developer Danny Waneri. The system, called bookmark-cli, indexes 45,053 tweets, 7,155 photo tweets with AI-generated descriptions, and over 100,000 documents synced daily via a Cloudflare Worker. The entire pipeline—from embedding to retrieval, reranking, and reflection—operates within a single Cloudflare Worker, keeping costs under $5 per month.

The reflection layer synthesizes connections across unrelated documents to surface hidden insights. For example, it combined fragments from tweets about AI and work into a single coherent observation about non-technical users relying on AI agents without proper review—a nuance no single tweet contained. This capability hinges on a model’s ability to reason across multiple documents, not just summarize them.

Why Gemma 4 MoE beat both dense alternatives

Cloudflare Workers AI offers three Gemma 4 variants, each suited to different use cases:

• - gemma-4-e4b-it: 4 billion total parameters, dense architecture, ideal for memory-constrained environments.
• - gemma-4-27b-it: 27 billion total parameters, dense, optimized for maximum quality at the cost of higher compute.
• - gemma-4-26b-a4b-it (MoE): 26 billion total parameters with 4 billion active per forward pass, designed for edge inference and deep reasoning.

The reflection engine requires a model capable of multi-document synthesis—reading five related chunks and producing a structured three-sentence insight. A 4B dense model lacks the reasoning depth, while a 27B dense model introduces latency at the edge. Gemma 4 MoE strikes a balance by activating only a subset of its parameters during each inference, enabling efficient edge deployment without sacrificing reasoning power.

Critically, Gemma 4 MoE is natively supported in Cloudflare Workers AI, meaning the entire pipeline—embed, retrieve, rerank, reflect—runs within a single Worker. No external API calls or data egress is required, preserving performance and privacy.

The migration: one line of code, three critical fixes

The switch required minimal code changes. The system already used an environment variable, REFLECTION_MODEL, to dynamically select the model. The migration involved:

wrangler secret put REFLECTION_MODEL
# Enter: gemma-4
wrangler deploy

Despite the simplicity, three adjustments were essential to avoid failure:

• - Token limits: The old model used max_tokens: 180, which was insufficient for Gemma 4’s reasoning chain. Increasing it to max_tokens: 2048 prevented truncated responses.
• - Response extraction: Gemma 4 returns reasoning in choices[0].message.content, not .reasoning or .response. Using the wrong field would yield empty outputs.
• - Prompt structure: Verbose rule lists triggered Gemma 4’s constraint-analysis behavior, causing it to restate instructions instead of executing them. Simplifying prompts and ending with a direct action cue resolved the issue.

A benchmark endpoint was added to compare Gemma 4 MoE and Kimi K2.5 in parallel. For nine real queries, the MoE model maintained or improved latency while delivering deeper insights, all within the same cost structure.

Looking ahead: edge-native reasoning without compromise

The Gemma 4 MoE migration proved that edge-native models can rival cloud-based alternatives in reasoning depth without sacrificing efficiency. For developers building personal knowledge tools, AI agents, or retrieval-augmented systems, the MoE architecture offers a compelling path forward—one that keeps data local, costs predictable, and reasoning robust.

As models continue to evolve, the key takeaway is clear: the right architecture isn’t just about scale—it’s about matching the model’s design to the task at hand. For deep, multi-document synthesis at the edge, mixture-of-experts models like Gemma 4 MoE are redefining what’s possible.