Why AI agents need real memory—not just storage

Modern AI agents are built to process vast amounts of information, yet most struggle to retain context beyond a single interaction. The problem isn’t storage—it’s memory. Ed Huang, CEO of PingCAP, argues in a recent piece for The New Stack that agents need a structured approach to memory, one that goes beyond simple persistence to include selection, compression, decay, and contamination prevention.

The difference between persistence and memory

Many teams confuse memory with other functionalities like idempotency, workflow state management, or transactional consistency. These features are essential for system stability, but they don’t equate to memory. True memory requires the ability to recall past interactions in a meaningful way—something that raw persistence alone cannot provide.

Huang outlines five critical layers that define real memory:

• Persistence: Ensures history survives system restarts.
• Selection: Filters out irrelevant data to focus on what matters.
• Compression: Summarizes raw interactions into queryable insights.
• Decay: Reduces the influence of outdated information over time.
• Contamination prevention: Flags and quarantines incorrect memories before they mislead the agent.

Without these layers, agents risk becoming slower, more expensive, and increasingly unreliable as they accumulate irrelevant or conflicting data.

Why traditional storage solutions fall short

Single-purpose databases like Redis and vector databases, while powerful in their domains, are ill-equipped to handle the complexity of episodic memory. Redis excels at fast key-value lookups but lacks the relational querying capabilities needed for tasks like retrieving all successful refunds issued for users in a specific region over the past 30 days. Vector databases, on the other hand, are adept at semantic search but struggle with exact predicate matching—such as filtering by user ID, time window, or outcome.

These limitations highlight a fundamental gap: agents need a system that can combine structured querying with vector-based recall, all while managing confidence decay and contamination in real time.

A schema designed for agent memory

Huang proposes a schema that integrates episodic and semantic memory into a single database, with built-in mechanisms for decay and contamination control. The schema includes two primary tables:

-- Episodic memory: records of specific interactions
CREATE TABLE episodic_memory (
  id BIGINT PRIMARY KEY AUTO_INCREMENT,
  agent_id VARCHAR(64) NOT NULL,
  user_id BIGINT NOT NULL,
  summary TEXT, -- Compressed summary of the interaction
  raw_payload JSON, -- Full details for audit
  outcome ENUM('success', 'failure', 'pending'),
  confidence FLOAT DEFAULT 1.0,
  superseded_by BIGINT NULL, -- Points to a corrected memory
  created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
  embedding VECTOR(1536), -- For semantic similarity
  INDEX idx_agent_user_time (agent_id, user_id, created_at),
  INDEX idx_embedding USING HNSW (embedding)
);

-- Semantic memory: distilled knowledge with decay tracking
CREATE TABLE semantic_memory (
  id BIGINT PRIMARY KEY AUTO_INCREMENT,
  agent_id VARCHAR(64) NOT NULL,
  fact TEXT NOT NULL,
  confidence FLOAT DEFAULT 1.0,
  source_count INT DEFAULT 1, -- Tracks how often this fact is confirmed
  last_confirmed_at DATIME NOT NULL, -- For decay calculations
  contradicted_at DATETIME NULL, -- Marks conflicting information
  embedding VECTOR(1536),
  INDEX idx_embedding USING HNSW (embedding)
);

The superseded_by field in the episodic memory table ensures that incorrect memories aren’t deleted but instead annotated and excluded from recall. Meanwhile, the source_count and last_confirmed_at fields enable decay functions that adjust the weight of memories at query time, not write time.

To retrieve relevant memories, agents can use queries that incorporate decay and contamination filtering:

-- Semantic recall with decay and contamination checks
SELECT 
  fact,
  confidence * EXP(-DATEDIFF(NOW(), last_confirmed_at) / 90.0) AS effective_weight,
  VEC_COSINE_DISTANCE(embedding, @task_vec) AS distance
FROM semantic_memory
WHERE 
  (user_id = @user_id OR user_id IS NULL)
  AND contradicted_at IS NULL
  AND VEC_COSINE_DISTANCE(embedding, @task_vec) < 0.30
ORDER BY distance, effective_weight DESC
LIMIT 10;

The decay term EXP(-days/90) ensures that memories fade gradually, with a half-life of approximately 60 days. This approach allows for tuning—customer preferences might decay slowly, while inventory updates could fade much faster.

The substrate matters

Huang emphasizes that the database substrate itself doesn’t create intelligence, but it can enable or hinder an agent’s ability to implement real memory. Without a system that supports structured episodic queries, vector recall, confidence decay, and contamination invalidation, agents will continue to struggle in production environments.

"The substrate doesn't do those things. The agent does. But the agent can't do those things without a substrate that supports them."

While Huang’s pitch highlights TiDB as a potential solution—thanks to its distributed SQL and native vector support—the core insight transcends any specific technology. The challenge of building agents with true memory is universal, and the solutions Huang outlines offer a roadmap for teams looking to move beyond simple data storage to meaningful, long-term recall.

As AI systems become more integrated into customer-facing roles, the ability to remember, adapt, and forget appropriately will separate the agents that merely function from those that genuinely improve over time.