Cache Keys Must Come From the Resolved Call

Every cache should have one property, and plenty do not: turning it on changes how fast your code runs and nothing else. If attaching a cache can change the answers, it has stopped being an optimization and become a correctness feature with a performance side effect.

The most common way to lose that property in Python is a decorator that builds its key out of kwargs.

The bug shape

A hand-rolled memoization decorator often starts like this, because it’s the obvious thing to write:

def multicache(key_list):
    def decorator(func):
        def wrapper(self, *args, **kwargs):
            key = "_".join(str(kwargs.get(k)) for k in key_list)
            ...
        return wrapper
    return decorator

Now consider a method decorated with key_list=["branch", "limit"] and these two calls:

repo.commit_history("main", 100)          # positional
repo.commit_history(branch="main", limit=100)  # keyword

They are the same call. Python does not care which form you use. But kwargs is empty in the first case, so kwargs.get("branch") is None, and the key becomes "None_None".

Two things follow, and the second is much worse than the first. The first is that positional calls never hit a cache populated by keyword calls, so you lose the speedup. The second is that every positional call with different arguments produces the identical key "None_None", so the first result gets stored under it and every subsequent call with different parameters gets that first answer back. Ask for one branch, then another, and get the first branch’s data. The cache is now returning wrong results, quietly, and only when it’s enabled.

The fix: bind against the signature

Python hands you the machinery to resolve a call into canonical form. inspect.signature gives you the parameter list, bind maps whatever the caller passed onto it, and apply_defaults fills in the parameters the caller left alone:

import inspect
from functools import wraps

def multicache(key_list):
    def decorator(func):
        sig = inspect.signature(func)

        # Validate the declared key names once, at decoration time.
        params = set(sig.parameters)
        if not any(p.kind is inspect.Parameter.VAR_KEYWORD
                   for p in sig.parameters.values()):
            unknown = set(key_list) - params
            if unknown:
                raise ValueError(f"unknown key_list names: {sorted(unknown)}")

        @wraps(func)
        def wrapper(self, *args, **kwargs):
            try:
                bound = sig.bind(self, *args, **kwargs)
                bound.apply_defaults()
                values = bound.arguments
            except TypeError:
                # Let the wrapped function raise the real argument error.
                return func(self, *args, **kwargs)

            key = "||".join(str(values.get(k)) for k in key_list)
            ...
        return wrapper
    return decorator

Three things in there are worth calling out individually, because each one is its own small lesson.

bind plus apply_defaults is the whole fix. After binding, a positional call and a keyword call produce identical arguments dicts, so they produce identical keys. apply_defaults matters just as much: without it, calling f(x=1) and f(x=1, limit=None) differ, even though the default for limit is None, so you get two cache entries for one logical call.

Validate the declared key names at decoration time. This is the change I’d push hardest. A typo in key_list, say ignore_blobs where the parameter is ignore_globs, means that argument silently never enters the key. Every call that differs only in that argument now collides. Nothing raises, nothing logs, and you find out when a filtered query returns unfiltered data. Since the misspelled name is not a real parameter, the signature knows it’s wrong, so raise a ValueError at import time and turn a silent data bug into a loud startup error. Skip the check when the function takes **kwargs, since then anything goes.

Pick a delimiter that can’t occur in a value. Joining with _ is asking for trouble when your values are branch names and file globs, which frequently contain underscores. ("a_b", "c") and ("a", "b_c") both render as a_b_c. Use something that will not appear in your values, like ||.

And keep @wraps. Without it the decorator erases the signature, so anything that introspects your functions to build another interface sees *args, **kwargs. That matters more than it used to: I generate MCP tool definitions from these same methods, and an unwrapped decorator turns a nicely typed tool into one that accepts a shapeless blob.

The test that proves it

The first test any memoization decorator deserves is that the two calling conventions agree:

def test_positional_and_keyword_calls_share_a_key():
    cache = EphemeralCache()
    repo = Repository(path, cache_backend=cache)

    a = repo.commit_history("main", 100)
    entries_after_first = len(cache._cache)
    b = repo.commit_history(branch="main", limit=100)

    assert len(cache._cache) == entries_after_first   # a cache hit, not a new entry
    assert a.equals(b)

Then the one that catches the dangerous version:

def test_distinct_positional_calls_do_not_collide():
    first = repo.commit_history("main", 10)
    second = repo.commit_history("other-branch", 10)
    assert not first.equals(second)   # fails loudly if both keyed as "None_None"

That second test is the one that would have caught the original bug, and it looks so trivial that it feels like a waste of a test until you understand what it pins.

While you’re in there, one more property to decide on deliberately. Most cache backends store and return the object itself, so a caller who mutates a cached value has rewritten what every future cache hit returns.

With DataFrames this is easy to do by accident:

def punchcard(self, ...):
    df = self.commit_history(...)     # possibly a cached frame
    df["hour_of_day"] = ...           # in-place write on the cached object

After one punchcard() call, commit_history() returns two extra columns for the rest of the process. Nothing errored, nothing was reported, and every downstream consumer now sees a frame with a different shape than the one documented.

The fix is a .copy() before mutating, and the test is a snapshot comparison over the whole cache:

def test_operations_do_not_mutate_cached_frames():
    repo.commit_history()                    # populate
    before = {k: (v.columns.tolist(), v.shape, v.copy())
              for k, v in cache._cache.items()}
    repo.punchcard()
    for k, (cols, shape, frame) in before.items():
        assert cache._cache[k].columns.tolist() == cols
        assert cache._cache[k].shape == shape
        assert cache._cache[k].equals(frame)

One extra frame copy per call is almost always cheaper than the class of bug it removes.

Two lines to steal

A cache key is a claim that two calls are the same call. Any part of the call that can change the answer must be in the key, and the only reliable way to know what the caller actually passed is to resolve it against the signature.

Two lines to steal:

key(f(1)) must equal key(f(x=1)). Write that test first.
Validate the names you key on against the real signature at decoration time, so a typo is an error rather than a collision.

The bug shape

The fix: bind against the signature

The test that proves it

The related trap: caching by reference

Two lines to steal

Stay in the loop