Design Pattern Evangelist Blog

Smart pointers about software design

Mastering Time in Software Testing - Strategies for Temporal Behavior Verification

Become a Time Lord and control time in your tests


The Persistence of Time

Introduction

Tsumani

The Y2K Problem was viewed as a nonevent by most of the general public, not because it was overhyped, but because the software industry dedicated many resources to ensure that the world didn’t start the new millennium beneath a digital tsunami when the calendar rolled over from 1999 to 2000 and computers around the world might assume that the year was 1900.

Simpsons 2000 Ball Drop

Efforts to address the Y2K problem started well before 2000 arrived. When a Y2K problem was found in code, it would be addressed. How would we know if it was effective? What about other Y2K problems still unrevealed? We just couldn’t wait for the ball to drop at the end of 1999, and hope for the best.

At my company, we reset the system clocks to 11:59 PM on December 31, 1999, and observed what happened when they rolled over a minute later. We addressed issues we found and reset the clocks again. This wasn’t a great technique, but it was about the best we could at the time.

Time and Clocks

What is Time? Time is what a Clock reads, according to Physicists.

While I tease at taking control of time, I’m actually presenting ways to take control of clocks. These techniques present the illusion of time control, but they’re just moving the hands on a clock to display whatever time we desire in a time dependent test.

Time Has No Functional Role

Does anybody really know what time it is? Does anybody really care?Robert Lamm founding member of the band Chicago

Maybe we don’t care about time. Many behaviors don’t depend upon time. For example, most mathematical functions don’t depend upon time; therefore, unit tests confirming mathematical functions are time agnostic.

In other cases, while time may technically be a behavior, it may not be a critical behavior. For example, the created or updated timestamp for an object may be observable behavior, but it may not be critical enough to confirm in a test. Let’s assume that we’re testing a method that returns an Order object, and we want to confirm the attributes of the Order being returned. Our test would have asserts for some of the Order object’s attributes such as: Customer, Items, Quantity, Price, etc., since those attributes may be part of the behavior specification. Order may also contain a timestamp attribute, but it may not be a criticial behavior specification, and therefore, there’s no compelling need to assert it.

Asserts could be more tricky when using Approval Testing, and the String representation of the Order contains the timestamp. Since the timestamp will be a different value each time the test is executed, additional effort may be needed to redact it so that it won’t cause the assertion to fail.

Time May Play A Functional Role

Sometimes time is a critical element to behavior. Y2K was a ticking timebomb. There may be a behavior scenario in which a test needs to confirm the timestamp for an Order object.

Here are a few scenarios where time plays a significant role:

Do we implement what we think will work, hope for the best and observe what happens? Do we reset the machine clocks repeatedly?

Heartbeat or Heart Attack?

What about functionality that takes a long time to playout?

About two decades ago, I worked for a startup that made a large fiber optic switch, which had slots for around 500 circuit packs. I implemented the feature that rebooted unresponsive circuit packs when more than two minutes had transpired since their last heartbeat notification.

One day, our customer support team called me into their lab. Multiple circuit packs were rebooting for our customers across the country. The circuit packs weren’t rebooting en mass, but they were rebooting in clusters. For example, the circuit packs on one shelf would reboot in succession every several seconds as if one rebooting circuit pack caused its neighbor to reboot two or three seconds later. Then another cluster might reboot a few minutes later on the same switch, but on a different shelf, or hours later another cluster would reboot on a switch in a different city.

We were stymied. I had no logical explanation of how my code could generate a pattern like this. The circuit packs had been running continuously for several weeks. Then we noticed something. None of our customers’ circuit packs had been running for more than 42 continous days in production.

Some were at 41 days, so we kept an eye on them, and sure enough, they rebooted on day 42. Someone theorized that we had a rollover issue. Each circuit pack had a ticks counter that issued a heartbeat every several seconds with an implementation similar to this:

// If at least 100 ticks have transpired since the last heartbeat, then issue a new heartbeat
if (ticks > lastIssuedHeartbeatTickCount + 100) {
    issueHeartbeat();
    lastIssuedHeartbeatTickCount = ticks;
}
ticks++;

When the ticks counter surpassed its maximum value, it rolled over, essentially resetting it. The if condition was never true again, since it was essentially zero, so it never issued another heartbeat. After two minutes without a heartbeat notification, my feature rebooted the stuck circuit pack. We calculated how long it would take for ticks to roll over, and it was 42 days.

The confusing behavior of clustered circuits rebooting in succession on the same shelf every few seconds now made sense. 42 days previously, a technician had manually inserted the circuit packs into their slots, which required a few seconds between each insertion to ensure the circuit pack was seated properly in the slot with its latches closed. Each circuit pack’s ticks counter began at a staggered start time compared to its neighbor like the staggered start of a road rally race.

The circuit pack team fixed the problem, which took a few days to deploy. Our customer support team reset ticks manually for the circuit packs that were getting close to 42 days old so that our customers wouldn’t experience any more random reboots until the fix was fully deployed.

I chucked to myself that we would have had to let our circuit packs soak continuously in the test lab for 42 days before we noticed it ourselves. We were lucky if the switch ran uninterrupted in the test lab for more than several hours without being rebooted due testing reseating reboot scenario or a new version of the circuit pack software.

Become a Time Lord

If I could turn back the hands of time.Bruce Springsteen

Tardis Blueprint

How could we have possibly tested a circuit pack for more than 42 days?

Become a Time Lord. Take control of time in your tests. We can do the following in our unit tests:

The Phoenix Project Revisited

The Phoenix Project presents the story of the technical struggles for the fictional company, Parts Unlimited, as it’s desperately trying to release its new Phoenix Project application while trying to keep the rest of the company afloat.

Spoiler Alert: the book has a happy ending. But their story is not over. I’ve continued their adventures in my fan fiction. Let’s listen in:

Bill Palmer, VP of IT Operations, leaned back in his chair, staring at the charts projected on the conference room screen. Sarah Madsen, Head of Analytics, tapped at her laptop. “We’ve confirmed it—95% of canceled orders happen within 15 minutes of placement. If we delay fulfillment by just that much, we eliminate most of the restocking costs.”

John Stowe, Senior Project Manager, leaned forward. “So, can we do it? Can we just hold the orders before sending them to Fulfillment?”

Brent Geller, Lead Engineer, arms crossed, let out a long breath. “It’s never that simple. The fulfillment system isn’t designed to ‘pause’ an order once it’s in the queue. We’d need a holding mechanism upstream.”

“Or,” Patty McKee, Director of Development, interjected, “we could introduce an order confirmation buffer. Users place an order, but it sits in a pending state for 15 minutes before finalizing. That way, cancellations happen before the system commits to fulfillment.”

“Wouldn’t that create a bad user experience?” John asked. “People expect orders to go through instantly.”

“Maybe not,” Sarah countered. “We could frame it as a ‘grace period’ for order changes. Some customers might even appreciate the flexibility.”

Bill nodded. “Alright, let’s assume we build this buffer. How do we test it?”

Brent rubbed his chin. “Unit testing this will be tricky. We’ll need a way to simulate time passing in our tests without actually waiting 15 minutes.”

Patty added, “Mocking time-dependent behavior can get messy. We’ll need robust test cases that verify orders are only processed after the soak period expires, without introducing flakiness.”

Bill tapped the table. “Sounds like a plan. Let’s scope this out, prototype it, and validate with data. If it saves money without hurting customers, it’s a win.”

Everyone nodded. Another problem, another solution—just another day at Parts Unlimited.

The Phoenix Project doesn’t provide implementation examples, but let’s assume that the subsequent code examples illustrate how their code may:

Within the Orders class, Order objects are added orders, a List<Order>, as the Orders are placed. Then once a minute, a separate process thread calls Orders.fulfillOrders(), which sends each Order to Fulfillment to be fulfilled and clears its orders List. Here’s an example of what fulfillOrders() looks like without any pending state delay:

    public void fulfillOrders() {
        for (Order order : orders) {
            fulfillment.fulfill(order);
        }
        orders.clear();
    }

NOTE: The entire code example is listed at the bottom of the blog.

Here’s one way to implement Patty’s 15-minute pending state suggestion:

    public void fulfillOrders() {
        // Iterator avoids Concurrent Modification Exceptions
        ListIterator<Order> orderIterator = orders.listIterator();
        while (orderIterator.hasNext()) {
            Order order = orderIterator.next();
            if (isMature(order)) {
                fulfillment.fulfill(order);
                orderIterator.remove();
            }
        }
    }

    private boolean isMature(Order order) {
        return order.getPlacedOrderTime().plusMinutes(15).isBefore(LocalDateTime getNow());
    }

This works … probably. But how can we be sure? A test for this would take at least 15 minutes to complete. Fifteen minutes is much too long for a test, and what if we were to change the pending policy to 15 hours or 15 days? We probably wouldn’t hold pending orders this long, but other applications could take this long. For example, it may take days before the notification for a delinquent payment is sent. How can we test something that doesn’t trigger for days?

Time is an External Dependency

If you don’t consider time an input value, think about it until you do — it is an important concept.John Carmack

The call to LocalDateTime.getNow() gives us the current time, but time is an external dependency. The fulfillOrders() method is tightly coupled static call.

Mockito will allow you to mock static calls, but as I mentioned in Mocking Frameworks, if you have to mock a static method, that’s probably an indication of a code smell that needs to be addressed.

We can also introduce a seam by extracting LocalDateTime.getNow() into its own method, resulting in:

    private boolean isMature(Order order) {
        return order.getPlacedOrderTime().plusMinutes(15).isBefore(getNow());
    }

    LocalDateTime getNow() {
        return LocalDateTime.now();
    }

isMature(Order order) delegates to getNow(), which being a protected-private method, we can override to return any value desired. When not overridden, it will return the current system time.

In the test, we can do something like:

Orders orders = new Orders() {
    @Override
    LocalDateTime getNow() {
        return LocalDateTime.of(2025, 04, 01, 10, 30);
    }
}

This is quite useful when we want a specific timestamp returned for a String comparison, but to confirm the new 15-minute feature in Orders for Parts Unlimited we’re going to need something a bit more sophisticated.

Since time is an external dependency, I am treating it like any other external dependency, like I would with a database or filesystem dependency. See: Hexagonal Architecture Structure.

I’ve defined a Clock interface:

interface Clock {
    LocalDateTime getNow();
}

Orders depend upon Clock, which will be injected into it. This simplifies isMature(Order order) to:

    private boolean isMature(Order order) {
        return order.getPlacedOrderTime().plusMinutes(15).isBefore(clock.getNow());
    }

Here’s an example of the implementation of the injected SystemClock, which would be used in production. SystemClock is an Adapter:

class SystemClock implements Clock {
    @Override
    public LocalDateTime getNow() {
        return LocalDateTime.now();
    }
}

I created the ClockStub Test Double, which allows me to advance forward any number of desired minutes:

class ClockStub implements Clock {
    private LocalDateTime localDateTime;

    public ClockStub(LocalDateTime localDateTime) {
        this.localDateTime = localDateTime;
    }

    public void advanceMinutes(int minutes) {
        localDateTime = localDateTime.plusMinutes(minutes);
    }

    @Override
    public LocalDateTime getNow() {
        return localDateTime;
    }

}

And finally with all the parts defined, I can create unit tests that take control of time. My demo, found in full below, has several tests, but here is the test that emulates the passage of 42 minutes functionally but only requires milliseconds to execute.

    private static void ordersOperationalScenario() throws Exception {
        // Given
        ClockStub clockStub = new ClockStub(LocalDateTime.of(2025, 04, 01, 10, 30));
        FulfillmentSpy fulfillmentSpy  = new FulfillmentSpy();
        Orders orders = new Orders(clockStub, fulfillmentSpy);

        // When-Then: Full scenario with multiple Orders calls.

        orders.placeOrder(new Order(1, 41, 1001, 1));
        clockStub.advanceMinutes(5);
        orders.fulfillOrders();
        assertEquals("[]", fulfillmentSpy.getFulfillmentOrders().toString());

        orders.placeOrder(new Order(2, 42, 1002, 2));
        clockStub.advanceMinutes(5);
        orders.fulfillOrders();
        assertEquals("[]", fulfillmentSpy.getFulfillmentOrders().toString());

        orders.placeOrder(new Order(3, 43, 1003, 3));
        orders.cancelOrder(1);
        clockStub.advanceMinutes(16);
        orders.fulfillOrders();
        assertEquals("[Order(orderNumber=2, customerId=42, item=1002, quantity=2, placedOrderTime=2025-04-01T10:35), Order(orderNumber=3, customerId=43, item=1003, quantity=3, placedOrderTime=2025-04-01T10:40)]", fulfillmentSpy.getFulfillmentOrders().toString());

        orders.placeOrder(new Order(4, 44, 1004, 4));
        clockStub.advanceMinutes(16);
        orders.fulfillOrders();
        assertEquals("[Order(orderNumber=2, customerId=42, item=1002, quantity=2, placedOrderTime=2025-04-01T10:35), Order(orderNumber=3, customerId=43, item=1003, quantity=3, placedOrderTime=2025-04-01T10:40), Order(orderNumber=4, customerId=44, item=1004, quantity=4, placedOrderTime=2025-04-01T10:56)]", fulfillmentSpy.getFulfillmentOrders().toString());
    }

Summary

Effective testing of time-sensitive code requires more than just a few sleep statements and crossed fingers. It demands isolating time as a dependency, using mocks or controlled time providers, and accounting for edge cases like daylight savings, leap years, and clock drift. By treating time as a first-class concern in our tests, we can avoid costly bugs and unexpected failures.

References

Here are some free resources:

Here are some resources that can be purchased or are included in a subscription service:

Full Disclosure

I did not write The Phoenix Project fan fiction provided above in The Phoenix Project Revisited. It was written by ChatGPT.

I had already written the demo code, listed below. I wanted a story to provide context for a 15-minute delay. Rather than invent a story from scratch, The Phoenix Project came to mind. I asked ChatGPT if it knew the book and whether it could create some fan fiction for me. When it said it could, I gave it this prompt:

Here are the main story point elements that must be in this fan fiction version of “The Phoenix Project”:

Create a Phoenix Project inspired fan fiction story for this. It should be less than 300 words.

ChatGPT’s original story included A/B Testing, which I see how it generated from the third or fourth bullet I provided. I provided a follow up prompt to guide ChatGPT to focus upon unit testing the 15-minute delay rather than the A/B Testing. Its original A/B Testing suggestion would be a good way to determine whether the 15-minute delay actually worked in production with the users as expected.

I added a few more prompts but only with respect to how characters were introduced and how their names were presented.

While some original characters from the book appear in the generated fan fiction, ChatGPT created new convincing characters as well. It also generated the “grace period” concept as well as the specifics generally associated with time in unit tests including how they can be flaky if not implemented correctly.

Complete Demo Code

Here’s the entire implementation up to this point as one file. Copy and paste it into a Java environment and execute it. If you don’t have Java, try this Online Java Environment. Add more tests. Play with the implementation. Refactor some of the code.

import java.time.*;
import java.util.*;

public class TimeLord {
    public static void main(String[] args) throws Exception {
        Test.test();
    }
}

///////////// SOFTWARE UNDER TEST //////////////////

class Orders {
    private final Clock clock;
    private final Fulfillment fulfillment;
    private final List<Order> orders = new LinkedList<>();

    public Orders(Clock clock, Fulfillment fulfillment) {
        this.clock = clock;
        this.fulfillment = fulfillment;
    }

    public void placeOrder(Order order) {
        order.orderPlaced(clock.getNow());
        orders.add(order);
    }

    public void cancelOrder(int orderNumber) {
        // Iterator avoids Concurrent Modification Exceptions
        ListIterator<Order> orderIterator = orders.listIterator();
        while (orderIterator.hasNext()) {
            Order order = orderIterator.next();
            if (order.getOrderNumber() == orderNumber) {
                orderIterator.remove();
            }
        }
    }

    public void fulfillOrders() {
        // Iterator avoids Concurrent Modification Exceptions
        ListIterator<Order> orderIterator = orders.listIterator();
        while (orderIterator.hasNext()) {
            Order order = orderIterator.next();
            if (isMature(order)) {
                fulfillment.fulfill(order);
                orderIterator.remove();
            }
        }
    }

    private boolean isMature(Order order) {
        return order.getPlacedOrderTime().plusMinutes(15).isBefore(clock.getNow());
    }

}

class Order {
    private final int orderNumber;
    private final int customerId;
    private final int itemId;
    private final int quantity;
    private LocalDateTime placedOrderTime;

    public Order(int orderNumber, int customerId, int itemId, int quantity) {
        this.orderNumber = orderNumber;
        this.customerId = customerId;
        this.itemId = itemId;
        this.quantity = quantity;
    }

    public void orderPlaced(LocalDateTime placedOrderTime) {
        this.placedOrderTime = placedOrderTime;
    }

    public LocalDateTime getPlacedOrderTime() {
        return placedOrderTime;
    }

    public int getOrderNumber() {
        return orderNumber;
    }

    @Override 
    public String toString() {
        return String.format("Order(orderNumber=%d, customerId=%d, item=%d, quantity=%d, placedOrderTime=%s)", orderNumber, customerId, itemId, quantity, placedOrderTime.toString());
    }

}

//////////// DEPENDENCY INTERFACES AND PRODUCTION IMPLEMENTATIONS ///////////////////

interface Clock {
    LocalDateTime getNow();
}

// NOTE: This is the Adapter Design Pattern. See: https://jhumelsine.github.io/2023/09/29/adapter-design-pattern.html
class SystemClock implements Clock {
    @Override
    public LocalDateTime getNow() {
        return LocalDateTime.now();
    }
}

interface Fulfillment {
    void fulfill(Order order);
}

// We don't need ProductionFulfillment for this demo.

////////////// TEST DOUBLES /////////////////

// This Stub allows us to take control of time.
class ClockStub implements Clock {
    private LocalDateTime localDateTime;

    public ClockStub(LocalDateTime localDateTime) {
        this.localDateTime = localDateTime;
    }

    public void advanceMinutes(int minutes) {
        localDateTime = localDateTime.plusMinutes(minutes);
    }

    @Override
    public LocalDateTime getNow() {
        return localDateTime;
    }

}

// This Spy records interactions with Fulfillment
class FulfillmentSpy implements Fulfillment {
    private List<Order> fulfilledOrders = new LinkedList<>();

    @Override
    public void fulfill(Order order) {
        fulfilledOrders.add(order);
    }

    public List<Order> getFulfillmentOrders() {
        return fulfilledOrders;
    }
}

/////////////// TESTS ////////////////
class Test {
    public static void test() throws Exception {
        placedOrderNotMatureEnoughToFulfill();

        placedOrderNotQuiteMatureEnoughToFulfill();

        placedOrderMatureEnoughToFulfill();

        multipleMaturePlacedOrdersAreFulfilledButLastIsNotMature();

        canceledOrderBeforeMaturityNotFulfilled();

        ordersOperationalScenario();

        System.out.format("End Tests");
    }

    private static void placedOrderNotMatureEnoughToFulfill() throws Exception {
        // Given
        FulfillmentSpy fulfillmentSpy  = new FulfillmentSpy();
        Orders orders = new Orders(new SystemClock(), fulfillmentSpy);

        orders.placeOrder(new Order(1, 1, 1001, 1));

        // When
        orders.fulfillOrders();

        // Then
        assertEquals("[]", fulfillmentSpy.getFulfillmentOrders().toString());
    }

    private static void placedOrderNotQuiteMatureEnoughToFulfill() throws Exception {
        // Given
        ClockStub clockStub = new ClockStub(LocalDateTime.of(2025, 04, 01, 10, 30));
        FulfillmentSpy fulfillmentSpy  = new FulfillmentSpy();
        Orders orders = new Orders(clockStub, fulfillmentSpy);

        orders.placeOrder(new Order(1, 1, 1001, 1));

        clockStub.advanceMinutes(15);

        // When
        orders.fulfillOrders();

        // Then
        assertEquals("[]", fulfillmentSpy.getFulfillmentOrders().toString());
    }

    private static void placedOrderMatureEnoughToFulfill() throws Exception {
        // Given
        ClockStub clockStub = new ClockStub(LocalDateTime.of(2025, 04, 01, 10, 30));
        FulfillmentSpy fulfillmentSpy  = new FulfillmentSpy();
        Orders orders = new Orders(clockStub, fulfillmentSpy);

        orders.placeOrder(new Order(1, 1, 1001, 1));
        clockStub.advanceMinutes(16);

        // When
        orders.fulfillOrders();
        orders.fulfillOrders(); // Confirms idempotence. Fulfill should only receive the order once.

        // Then
        assertEquals("[Order(orderNumber=1, customerId=1, item=1001, quantity=1, placedOrderTime=2025-04-01T10:30)]", fulfillmentSpy.getFulfillmentOrders().toString());
    }

    private static void multipleMaturePlacedOrdersAreFulfilledButLastIsNotMature() throws Exception {
        // Given
        ClockStub clockStub = new ClockStub(LocalDateTime.of(2025, 04, 01, 10, 30));
        FulfillmentSpy fulfillmentSpy  = new FulfillmentSpy();
        Orders orders = new Orders(clockStub, fulfillmentSpy);

        orders.placeOrder(new Order(1, 41, 1001, 1));
        clockStub.advanceMinutes(5);

        orders.placeOrder(new Order(2, 42, 1002, 2));
        clockStub.advanceMinutes(5);

        orders.placeOrder(new Order(3, 43, 1003, 3));
        clockStub.advanceMinutes(15);

        // When
        orders.fulfillOrders();
        orders.fulfillOrders(); // Confirms idempotence. Fulfill should only receive an order once.

        // Then
        assertEquals("[Order(orderNumber=1, customerId=41, item=1001, quantity=1, placedOrderTime=2025-04-01T10:30), Order(orderNumber=2, customerId=42, item=1002, quantity=2, placedOrderTime=2025-04-01T10:35)]", fulfillmentSpy.getFulfillmentOrders().toString());
    }

    private static void canceledOrderBeforeMaturityNotFulfilled() throws Exception {
        // Given
        ClockStub clockStub = new ClockStub(LocalDateTime.of(2025, 04, 01, 10, 30));
        FulfillmentSpy fulfillmentSpy  = new FulfillmentSpy();
        Orders orders = new Orders(clockStub, fulfillmentSpy);

        orders.placeOrder(new Order(1, 41, 1001, 1));
        clockStub.advanceMinutes(5);

        orders.cancelOrder(1);
        clockStub.advanceMinutes(15);

        // When
        orders.fulfillOrders();

        // Then
        assertEquals("[]", fulfillmentSpy.getFulfillmentOrders().toString());
    }

    private static void ordersOperationalScenario() throws Exception {
        // Given
        ClockStub clockStub = new ClockStub(LocalDateTime.of(2025, 04, 01, 10, 30));
        FulfillmentSpy fulfillmentSpy  = new FulfillmentSpy();
        Orders orders = new Orders(clockStub, fulfillmentSpy);

        // When-Then: Full scenario with multiple Orders calls.

        orders.placeOrder(new Order(1, 41, 1001, 1));
        clockStub.advanceMinutes(5);
        orders.fulfillOrders();
        assertEquals("[]", fulfillmentSpy.getFulfillmentOrders().toString());

        orders.placeOrder(new Order(2, 42, 1002, 2));
        clockStub.advanceMinutes(5);
        orders.fulfillOrders();
        assertEquals("[]", fulfillmentSpy.getFulfillmentOrders().toString());

        orders.placeOrder(new Order(3, 43, 1003, 3));
        orders.cancelOrder(1);
        clockStub.advanceMinutes(16);
        orders.fulfillOrders();
        assertEquals("[Order(orderNumber=2, customerId=42, item=1002, quantity=2, placedOrderTime=2025-04-01T10:35), Order(orderNumber=3, customerId=43, item=1003, quantity=3, placedOrderTime=2025-04-01T10:40)]", fulfillmentSpy.getFulfillmentOrders().toString());

        orders.placeOrder(new Order(4, 44, 1004, 4));
        clockStub.advanceMinutes(16);
        orders.fulfillOrders();
        assertEquals("[Order(orderNumber=2, customerId=42, item=1002, quantity=2, placedOrderTime=2025-04-01T10:35), Order(orderNumber=3, customerId=43, item=1003, quantity=3, placedOrderTime=2025-04-01T10:40), Order(orderNumber=4, customerId=44, item=1004, quantity=4, placedOrderTime=2025-04-01T10:56)]", fulfillmentSpy.getFulfillmentOrders().toString());
    }
    
    private static void assertEquals(String expected, String actual) throws Exception {
        if (!expected.equals(actual)) {
            System.out.format("expected=%s, actual=%s\n", expected, actual);
            throw new Exception();
        }
    }

}

Comments

Previous: Approval Testing - A Test Strategy for those who are reluctant to try Test-Driven Development

Next: Humble Objects - Designing Code You Don’t Hate Testing

Home: Design Pattern Evangelist Blog