Why NornicDB Replaced Unix Timestamps with Atomic Counters for MVCC

NornicDB’s journey to a reliable multi-version concurrency control (MVCC) system uncovered a subtle but critical flaw in using Unix timestamps for ordering transactions. The engineering team discovered that relying on time.Now().UnixNano() introduced non-monotonic behavior, which caused snapshot-isolation checks to flag false conflicts. To resolve this, they implemented a process-wide atomic counter to generate a global, monotonic sequence for commit ordering.

The Problem with Wall-Clock Timestamps in MVCC

The initial design assigned each committed write a pair of values: (CommitTimestamp, CommitSequence), where CommitTimestamp was derived from time.Now().UnixNano(). The snapshot-isolation system ordered versions primarily by sequence, but the timestamp was still used as a tiebreaker. This approach worked in theory—until real-world clock behavior intervened.

Linux’s clock_gettime(CLOCK_REALTIME) underpins time.Now(), and it is vulnerable to several sources of non-monotonicity:

• NTP corrections: The kernel may adjust the clock backward or forward to synchronize with a time server, even during active transactions.
• Hardware-assisted timekeeping: Technologies like PTP or live VM migration can abruptly shift the system clock.
• Per-CPU TSC skew: When goroutines run on different CPU cores, their clock_gettime calls may return differing timestamps due to unsynchronized Time Stamp Counter (TSC) registers.

A concrete example illustrates the issue. Imagine two transactions running in sequence:

1. Transaction A commits at timestamp 1_700_000_000_000_000_100.
2. Before Transaction B begins, the kernel applies an NTP correction that steps the clock backward by 500 nanoseconds.
3. Transaction B samples a timestamp of 1_700_000_000_000_000_050 and reads data committed by Transaction A.
4. The snapshot-isolation check compares the read timestamp (50) against the commit timestamp (100) and incorrectly flags a conflict, even though no concurrent writer existed.

This scenario demonstrates how clock drift can create phantom conflicts, undermining the reliability of MVCC-based systems.

Performance Demands Exposed Timestamp Limitations

NornicDB’s custom Cypher parser operates at a speed that renders Unix timestamps ineffective for ordering. Benchmarks show that a basic MATCH (n) RETURN n query parses and validates in just 39 nanoseconds, with zero heap allocations. This translates to over 25 million queries per second on a single goroutine.

Compare this to traditional parsing libraries like ANTLR, which require 4,725 nanoseconds per query and allocate memory per parse. The speed of NornicDB’s parser is not incidental—it is the reason why Unix timestamps fail to provide meaningful ordering. Slow parsers naturally space commits further apart in time, allowing timestamps to remain monotonic. But when commits occur within the same nanosecond bucket, ordering becomes ambiguous.

The probability of two commits landing in the same timestamp bucket increases with query throughput. For example, at 1 million commits per second and an effective timestamp resolution of 20 nanoseconds, the collision probability is approximately 1.98%. Over a short burst of activity, this can result in hundreds of unordered commits, forcing the system to rely on tiebreakers that are themselves unreliable.

The Solution: A Global Monotonic Counter

To eliminate the risks posed by non-monotonic timestamps, NornicDB replaced CommitTimestamp with a process-wide atomic uint64 counter. Each commit increments the counter atomically, ensuring a strict total order that cannot be disrupted by external factors like NTP corrections or TSC skew.

This approach offers several advantages:

• Guaranteed monotonicity: The counter increments sequentially, providing a reliable order for all commits.
• High throughput: Even at 1 billion commits per second, the counter would take 584 years to overflow, far exceeding practical needs.
• Consistency across goroutines: The atomic operation ensures that all threads observe the same sequence, regardless of CPU scheduling or clock corrections.

The new ordering logic prioritizes CommitSequence over any timestamp-based tiebreakers. The MVCCVersion.Compare() function now enforces this hierarchy:

func (a MVCCVersion) Compare(b MVCCVersion) int {
    if a.CommitSequence != b.CommitSequence {
        if a.CommitSequence < b.CommitSequence {
            return -1
        }
        return 1
    }
    if a.CommitTimestamp < b.CommitTimestamp {
        return -1
    }
    if a.CommitTimestamp > b.CommitTimestamp {
        return 1
    }
    return 0
}

This shift resolves the phantom conflicts seen in CI tests, such as TestExecuteCypher_SetInvalidatesManagedEmbeddings, where transactions were incorrectly flagged as conflicting despite no concurrent writes.

Looking Ahead: Building for Scale and Reliability

The decision to adopt a monotonic counter reflects a broader trend in distributed systems: the need to prioritize deterministic ordering over reliance on external clocks. For databases and transactional systems operating at high speeds or in heterogeneous environments, wall-clock timestamps are increasingly unreliable. Monotonic counters, whether atomic or derived from distributed consensus mechanisms, provide a robust alternative.

NornicDB’s experience underscores the importance of validating assumptions about system components—even those as fundamental as the system clock. By replacing timestamps with a simple yet effective counter, the team has eliminated a class of intermittent failures and positioned the database for even greater scalability. As query engines and transactional workloads continue to push the boundaries of performance, such design choices will become essential to maintaining correctness and predictability.