stucchio / python-lru-cache Goto Github PK

When cache expires in Python3, will have the following error:
ERROR:root:Exception calling application: OrderedDict mutated during iteration
Traceback
File "/opt/conda/lib/python3.6/site-packages/lru/init.py", line 32, in withlock
return func(self, *args, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/lru/init.py", line 160, in setitem
self.cleanup()
File "/opt/conda/lib/python3.6/site-packages/lru/init.py", line 32, in withlock
return func(self, *args, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/lru/init.py", line 184, in cleanup
for k in self.__expire_times:
RuntimeError: OrderedDict mutated during iteration

I found similar issues in coddingtonbear/python-myfitnesspal#26, suggest change the code to: for k in list(self.__expire_times.keys())

Hi
The __access_times used in the follow is a unordered dictionary. How can make sure delete the least recently used item here?

while (len(self.__values) > self.max_size):
    for k in self.__access_times.iterkeys():
        self.__delete__(k)
        break

Thanks

The cleanup function modifies the __expire_times dict during iteration, throwing an error introduced in python 3.5

File "/home/me/code/lru/__init__.py", line 32, in withlock
  return func(self, *args, **kwargs)
File "/home/me/code/lru/__init__.py", line 160, in __setitem__
  self.cleanup()
File "/home/me/code/lru/__init__.py", line 32, in withlock
  return func(self, *args, **kwargs)
File "/home/me/code/lru/__init__.py", line 184, in cleanup
  for k in self.__expire_times:
RuntimeError: OrderedDict mutated during iteration

for k in self.__expire_times:
    if self.__expire_times[k] < t:
        self.__delete__(k)

Will give:

>>> x = {'a': 1, 'b': 2}
>>> for k in x:
...     if k == 'a':
...             del x['a']
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration

If you try this code from the docs:

d = LRUCacheDict(max_size=3, expiration=3, thread_clear=True)

@lru_cache_function(cache=d)
def f(x):
    return x/2

You'll get this error:
TypeError: lru_cache_function() got an unexpected keyword argument 'cache'

pip install pylru this command is suited for this project, your project can be installed by cloning and python setup.py install.

I noticed the lack of license. Did you intend to make this opensource but haven't got around to figuring out the license yet? or is this just demo code? (or perhaps I overlooked something?).

Thanks!

I'm getting a KeyError once in a while (but not always). Here's my code:

    @lru_cache_function(max_size=1024, expiration=60)
    def shard_resolver(*resolver_args, **resolver_kwargs):
        # ... my code ...

And here is the traceback (removed irrelevant bits):

<snip>
    shard_id = shard_resolver(*resolver_args, **resolver_kwargs)
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 249, in __call__
    self.cache[key] = value
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 34, in withlock
    return func(self, *args, **kwargs)
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 160, in __setitem__
    self.cleanup()
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 34, in withlock
    return func(self, *args, **kwargs)
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 185, in cleanup
    if self.__expire_times[k] < t:
KeyError: '((74300113,), {})#shard_resolver'

Since it happen only infrequently, I would suspect it's exposes a race condition bug somewhere? Any idea?