PostgreSQL WAL Explained: How Logging Powers Crash Recovery and Data Safety

PostgreSQL’s durability guarantee hinges on a single principle: no committed data is truly safe until its changes are recorded in the Write-Ahead Log (WAL) and persisted to disk. This mechanism, rooted in the database’s reliability architecture, ensures that transactions marked as COMMIT survive system crashes or power failures—even if the actual data files remain unwritten. Developers rarely interact directly with WAL syntax, but they face its impact when pg_wal/ fills up unexpectedly, replication slots stall progress, or crash recovery stalls under heavy load.

Why PostgreSQL Requires WAL Before Data Persistence

Every modification to a PostgreSQL database—whether an INSERT, UPDATE, or DELETE—begins in memory within the shared buffers. Instead of writing changes directly to data files, PostgreSQL generates a WAL record that describes the delta: the type of operation, the affected table or index, the block number, and the payload. This record is temporarily stored in a shared memory buffer called wal_buffers before being flushed to disk. Only after the WAL record is fsync’d—ensuring it’s permanently written to storage—does PostgreSQL acknowledge the transaction as committed. The actual data pages remain dirty in memory until the background checkpointer process later flushes them to disk, decoupling durability from immediate persistence.

PostgreSQL organizes WAL into fixed-size segment files (default 16 MB) stored in the $PGDATA/pg_wal/ directory. Each WAL file is identified by a Log Sequence Number (LSN), a 64-bit value that acts as a monotonic clock for the database. An LSN like 0/1A2B3C40 represents a byte offset from the start of the WAL, and commands such as SELECT pg_current_wal_lsn(); let you inspect the current position.

-- Check the current WAL positionSELECT pg_current_wal_lsn();

-- Calculate bytes written between two LSNsSELECT pg_wal_lsn_diff('0/1A2BE018', '0/1A2B3C40') AS bytes_written;

-- Find the WAL file containing a specific LSNSLECT pg_walfile_name(pg_current_wal_lsn());

Crash Recovery: How PostgreSQL Rebuilds State from WAL

When a PostgreSQL instance restarts after a crash, it enters recovery mode. The database reads the pg_control file to locate the redo point—the LSN of the most recent completed checkpoint—and then replays every WAL record from that point forward. This process reconstructs the in-memory state by applying changes to shared buffers, effectively rebuilding the database as it existed before the crash. Only transactions with WAL records that were fully fsync’d before the crash are recovered; any uncommitted changes vanish. This contract underpins PostgreSQL’s durability guarantee.

A critical safeguard in this process is full-page writes, enabled by default via full_page_writes = on. When a data page is modified for the first time after a checkpoint, PostgreSQL writes the entire 8 KB page to the WAL—not just the delta. This prevents torn writes, a scenario where a power failure could leave the operating system in a state where only part of a page is updated. During recovery, the full page in the WAL overwrites any corrupted version on disk, ensuring data integrity. The trade-off is a temporary spike in WAL volume immediately following each checkpoint, though this stabilizes as frequently updated pages are written once.

Common Production Pitfalls and How to Avoid Them

1\. Replication Slots Consuming Unlimited WAL Space

Replication slots—whether logical or physical—prevent PostgreSQL from recycling WAL segments until the consuming client confirms it has processed them. A forgotten slot, paused connector, or ad-hoc debugging slot can pin WAL indefinitely, causing pg_wal/ to expand beyond available disk space. When the partition fills, PostgreSQL panics with an error like PANIC: could not write to file ... No space left on device, halting the cluster until space is freed.

Mitigation: Regularly audit slots and set hard limits using max_slot_wal_keep_size. PostgreSQL 13+ lets you invalidate lagging slots instead of risking disk exhaustion.

-- Inspect slot lag and retentionSELECT slot_name, active, restart_lsn,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained
FROM pg_replication_slots;

-- Set a safety cap for WAL retentionALTER SYSTEM SET max_slot_wal_keep_size = '20GB';
SELECT pg_reload_conf();

2\. Silent Archiving Failures Leading to Disk Exhaustion

When archiving is enabled (archive_mode = on), PostgreSQL only recycles WAL segments after the archive_command exits successfully. If commands like aws s3 cp or wal-g wal-push fail due to expired credentials or policy changes, segments linger in pg_wal/. The database continues running, but repeated errors like archive command failed with exit code N may go unnoticed—until the disk fills. This scenario mirrors the well-documented GitLab outage of 2017, where multiple safeguards failed silently.

Mitigation: Monitor archiving status proactively and implement alerts for failed archive commands.

3\. Misunderstanding Data Persistence Without WAL

Some developers mistakenly conclude that PostgreSQL is "lazy" about writing data after observing that data files are rarely updated. In reality, durability is ensured by WAL fsync, not immediate data file persistence. Disabling fsync, turning off full_page_writes, or setting synchronous_commit = off without understanding the risks invites unrecoverable corruption during power failures. PostgreSQL’s documentation explicitly warns that fsync = off may result in "unrecoverable data corruption."

4\. WAL Spikes After Checkpoints Increasing Replica Lag

Immediately following a checkpoint, PostgreSQL generates full-page writes for any hot pages modified for the first time since the last checkpoint. In tables undergoing scattered updates, this can cause WAL volume to surge for several seconds, spiking network traffic to replicas and increasing replication lag. Applications sensitive to lag may experience transient delays during peak checkpoint operations.

Mitigation: Adjust checkpoint frequency or buffer pool size to smooth WAL generation.

Best Practices for Managing WAL in Production

PostgreSQL’s WAL is a powerful but sometimes unforgiving mechanism. To avoid unexpected downtime, follow these guidelines:

• Monitor pg_wal/ size and replication slot lag using system views like pg_replication_slots and pg_stat_wal.
• Set conservative limits with max_wal_size and max_slot_wal_keep_size to cap WAL retention.
• Validate archiving pipelines and implement alerts for failed archive commands.
• Avoid disabling critical durability settings like fsync or full_page_writes without rigorous testing.
• Understand that WAL spikes are normal after checkpoints; tune checkpoint parameters to balance performance and lag.

As PostgreSQL continues to evolve, WAL remains the cornerstone of its reliability model. Whether you’re managing high-traffic clusters or tuning for performance, mastering WAL fundamentals ensures your data stays durable—and your applications remain available.