Engineer-Crafted LLM Glossary Bridges Theory and Production Reality

Engineers building with large language models often hit a wall: official definitions explain what terms mean, but not what they do. Quantization, KV cache, top-k sampling—these aren’t just academic concepts. They drive critical production decisions: latency budgets, memory footprints, and model accuracy. Without clarity on trade-offs, teams risk deploying systems that work in demos but fail under load.

That realization led one engineer to build LLM Field Notes, a living glossary where each entry pairs a plain-English definition with the engineering implications teams face at scale. The project now spans 30+ terms across eight pillars—from core architecture to agentic workflows—with cross-linked concepts so developers can trace dependencies without jumping between scattered resources.

Why Traditional Definitions Fall Short in Production

Most explanations stop at "what a term is." For example:

• Quantization: Described as reducing model precision (e.g., from float32 to int8).
• KV Cache: Defined as storing key-value pairs during inference.

These definitions don’t answer the questions engineers ask:

• How does quantization affect my inference latency? (Answer: Can cut memory usage by 4x and speed up decoding by 2–3x, but may reduce accuracy by 1–3%.)
• When does a KV cache become a bottleneck? (Answer: When batch size or sequence length grows, memory usage spikes, forcing smaller batches or model sharding.)

LLM Field Notes bridges this gap by focusing on operational impact. Each term includes:

• A concise definition
• Common use cases
• Failure modes to watch for
• Configuration levers and trade-offs

How the Project Scales Beyond Definitions

The glossary isn’t static. It’s designed for exploration. Terms are grouped into interconnected pillars:

• Core Architecture: Transformer, Attention, Feed-Forward Networks
• Memory & Compute: KV Cache, Quantization, Inference Vectors
• Generation & Sampling: Temperature, Top-p, Top-k
• Training & Alignment: Fine-tuning, LoRA, RLHF

Each entry links to related concepts, allowing engineers to follow the stack from prompts to outputs. For example, reading about quantization automatically surfaces links to memory budgets, batch processing, and inference speed—key factors in deployment decisions.

A Companion Guide for the Full Prompt Journey

To complement the glossary, the same creator published What Happens When You Prompt, a deep-dive reference tracing every layer of the stack from keystroke to streamed token. It answers the engineer’s hidden question: What actually happens when I press Send?

The guide covers:

• Tokenization and vocabulary mapping
• Prefill and decode phases in inference
• Streaming responses via Server-Sent Events (SSE)
• Memory optimization with KV cache management

Unlike high-level tutorials, this project targets engineers who already grasp transformer mechanics. It delivers production intuition—not another intro to attention mechanisms. Contributions are welcome; gaps in infrastructure specifics (e.g., provider-specific optimizations) are acknowledged and grounded in open-source frameworks and public API docs.

A Tool Built by Engineers, for Engineers

LLM Field Notes reflects a shift in how teams learn LLMs: from passive reading to active, context-driven understanding. It’s especially useful for:

• Startups prototyping LLM features
• Platform teams optimizing inference pipelines
• Engineers debugging deployment failures tied to sampling strategies or memory limits

By replacing ambiguity with actionable insight, the glossary helps teams move from experimentation to reliable production systems—without reinventing the wheel.

As LLMs evolve from research curiosities to core infrastructure, resources like this one will only grow in value. The next frontier? Bridging the gap between model behavior and system-level resilience—ensuring that what works in a notebook survives the chaos of real-world traffic.