Using @functools.lru_cache With Dictionary Arguments

I have a method that takes (among others) a dictionary as an argument. The method is parsing strings and the dictionary provides replacements for some substrings, so it doesn't hav

Solution 1:

Instead of using a custom hashable dictionary, use this and avoid reinventing the wheel! It's a frozen dictionary that's all hashable.

https://pypi.org/project/frozendict/

Code:

deffreezeargs(func):
    """Transform mutable dictionnary
    Into immutable
    Useful to be compatible with cache
    """    @functools.wraps(func)defwrapped(*args, **kwargs):
        args = tuple([frozendict(arg) ifisinstance(arg, dict) else arg for arg in args])
        kwargs = {k: frozendict(v) ifisinstance(v, dict) else v for k, v in kwargs.items()}
        return func(*args, **kwargs)
    return wrapped

and then

@freezeargs@lru_cache
def func(...):
    pass

Code taken from @fast_cen 's answer

Note: this does not work on recursive datastructures; for example, you might have an argument that's a list, which is unhashable. You are invited to make the wrapping recursive, such that it goes deep into the data structure and makes every dict frozen and every list tuple.

(I know that OP nolonger wants a solution, but I came here looking for the same solution, so leaving this for future generations)

Solution 2:

Here is a decorator that use @mhyfritz trick.

defhash_dict(func):
    """Transform mutable dictionnary
    Into immutable
    Useful to be compatible with cache
    """classHDict(dict):
        def__hash__(self):
            returnhash(frozenset(self.items()))

    @functools.wraps(func)defwrapped(*args, **kwargs):
        args = tuple([HDict(arg) ifisinstance(arg, dict) else arg for arg in args])
        kwargs = {k: HDict(v) ifisinstance(v, dict) else v for k, v in kwargs.items()}
        return func(*args, **kwargs)
    return wrapped

Simply add it before your lru_cache.

@hash_dict@functools.lru_cache()
def your_function():
    ...

Solution 3:

What about creating a hashable dict class like so:

classHDict(dict):
    def__hash__(self):
        returnhash(frozenset(self.items()))

substs = HDict({'foo': 'bar', 'baz': 'quz'})
cache = {substs: True}

Solution 4:

How about subclassing namedtuple and add access by x["key"]?

classX(namedtuple("Y", "a b c")):
    def__getitem__(self, item):
        ifisinstance(item, int):
            returnsuper(X, self).__getitem__(item)
        returngetattr(self, item)

Solution 5:

Here's a decorator that can be used like functools.lru_cache. But this is targetted at functions that take only one argument which is a flat mapping with hashable values and has a fixed maxsize of 64. For your use-case you would have to adapt either this example or your client code. Also, to set the maxsize individually, one had to implement another decorator, but i haven't wrapped my head around this since i don't needed it.

from functools import(_CacheInfo, _lru_cache_wrapper, lru_cache,
                       partial, update_wrapper)
from typing import Any, Callable, Dict, Hashable

def lru_dict_arg_cache(func: Callable) -> Callable:
    def unpacking_func(func: Callable, arg: frozenset) -> Any:
        return func(dict(arg))

    _unpacking_func = partial(unpacking_func, func)
    _cached_unpacking_func = \
        _lru_cache_wrapper(_unpacking_func, 64, False, _CacheInfo)

    def packing_func(arg: Dict[Hashable, Hashable]) -> Any:
        return _cached_unpacking_func(frozenset(arg.items()))

    update_wrapper(packing_func, func)
    packing_func.cache_info = _cached_unpacking_func.cache_info
    return packing_func


@lru_dict_arg_cache
def uppercase_keys(arg: dict) -> dict:
    """ Yelling keys. """return {k.upper(): v for k, v in arg.items()}


assert uppercase_keys.__name__ == 'uppercase_keys'assert uppercase_keys.__doc__ == ' Yelling keys. 'assertuppercase_keys({'ham': 'spam'}) == {'HAM': 'spam'}
assertuppercase_keys({'ham': 'spam'}) == {'HAM': 'spam'}
cache_info = uppercase_keys.cache_info()
assert cache_info.hits == 1assert cache_info.misses == 1assert cache_info.maxsize == 64assert cache_info.currsize == 1assertuppercase_keys({'foo': 'bar'}) == {'FOO': 'bar'}
assertuppercase_keys({'foo': 'baz'}) == {'FOO': 'baz'}
cache_info = uppercase_keys.cache_info()
assert cache_info.hits == 1assert cache_info.misses == 3assert cache_info.currsize == 3

For a more generic approach one could use the decorator @cachetools.cache from a third-party library with an appropriate function set as key.