Zyphra’s ZAYA1-8B delivers high-tier AI reasoning with just 8B parameters

In the ongoing race to build ever-larger AI models, a Palo Alto-based startup is taking a different path—one focused on efficiency rather than sheer scale. Zyphra, a lesser-known lab in the generative AI space, has introduced ZAYA1-8B, a new open-source reasoning model that delivers competitive performance despite containing just 8.4 billion total parameters. Of those, only 760 million are active during inference, a fraction of the scale used by industry leaders like OpenAI and Anthropic.

Despite its compact size, ZAYA1-8B achieves results on par with larger models such as GPT-5-High and DeepSeek-V3.2 on third-party benchmarks, including a remarkable 91.9% score on the AIME '25 mathematics competition. The model is available now under an Apache 2.0 license on Hugging Face, and developers can test it immediately via Zyphra’s cloud inference platform. What sets this release apart, however, is how it was trained—on a full stack of AMD Instinct MI300 GPUs, challenging Nvidia’s dominance in AI hardware.

Architectural breakthroughs behind the efficiency gains

Zyphra attributes the model’s performance to its proprietary MoE++ architecture, a refined version of the Mixture-of-Experts (MoE) approach that selectively activates only the most relevant model components for each input. The architecture introduces three key innovations designed to optimize both training and inference:

• Compressed Convolutional Attention (CCA): Traditional attention mechanisms in large language models (LLMs) scale poorly with longer context windows due to memory constraints tied to the KV-cache. CCA addresses this by performing sequence mixing in a compressed latent space, reducing the KV-cache size by 8x compared to standard multi-head attention. This enables more efficient processing of extended contexts without sacrificing performance.

• ZAYA1 MLP Router: Most MoE models rely on linear routers to assign tokens to experts. Zyphra replaces this with a multi-layer perceptron (MLP)-based design that improves decision-making granularity. To stabilize training—often a challenge for MoE architectures—the team implemented a bias-balancing scheme inspired by PID controllers from classical control theory, ensuring consistent expert utilization.

• Learned Residual Scaling: As data propagates through the model’s 40 layers, residual connections can lead to gradient vanishing or explosion. Zyphra’s solution introduces a learned scaling mechanism that regulates residual norm growth with minimal computational overhead, maintaining training stability.

Reasoning integrated from pretraining, not bolted on

A standout feature of ZAYA1-8B is its reasoning-first pretraining, which embeds logical reasoning capabilities into the model during its initial training phase rather than adding them later through fine-tuning. To handle complex, multi-step problems that exceed the model’s 4K pretraining context window, Zyphra developed Answer-Preserving (AP) Trimming.

AP-trimming works like an editor cutting a long film scene: instead of discarding the entire sequence or losing the solution, it preserves the problem setup and final answer while trimming intermediate reasoning steps. This ensures the model learns the critical relationship between problems and solutions, even when the full internal logic exceeds memory limits. During testing on Zyphra Cloud, the approach effectively handled queries about household tasks like removing countertop stains, demonstrating practical reasoning capabilities.

Markovian RSA: redefining efficient test-time compute

The most significant performance leap comes from Markovian RSA, a novel test-time compute (TTC) methodology that decouples reasoning depth from context size. Traditional methods rely on extending chain-of-thought (CoT) traces, which often leads to "context bloat" as the model’s history grows unwieldy.

Markovian RSA addresses this by simulating a recursive peer-review process:

• The model generates multiple parallel reasoning traces for a given input.
• It extracts only the final 4,000 tokens (the "tails") from these traces.
• These tails are subsampled and presented to the model in a new aggregation prompt, which evaluates and synthesizes the different approaches into a refined solution.

By carrying forward only the most recent reasoning steps, the model avoids context overflow while maintaining deep reasoning. In practice, this allows ZAYA1-8B—with just 760 million active parameters—to match or exceed the performance of models with 30 to 50 times its active parameter count.

A model built for local deployment and enterprise flexibility

The compact footprint of ZAYA1-8B—just 8.4 billion total parameters—positions it as a prime candidate for on-device deployment and local LLM applications. For enterprises, this opens doors to high-tier reasoning capabilities traditionally reserved for massive cloud-based models, while addressing critical concerns around data residency, latency, and persistent API costs.

The model’s open-source availability under the Apache 2.0 license further lowers barriers to adoption, enabling developers and startups to customize and deploy it without restrictive licensing fees. As AI innovation shifts toward efficiency and accessibility, ZAYA1-8B represents a compelling alternative to the resource-intensive models dominating current research.

Looking ahead, Zyphra’s approach could inspire broader experimentation with smaller, more efficient models—proving that high performance doesn’t always require astronomical scale.