tsroten / fcache Goto Github PK

fcache uses os.rename() to atomically write changes. But, this doesn't work across filesystems.

In some use cases, it might be useful to have multiple caches tied to the same application. Currently, this can't be done -- only one cache per application name is allowed.

Application sub caches should be named like this: 'application_name.subcache_name'. Sub caches operate entirely independent of the main application cache. They require their own object to access them. The only thing linking them together is that they "belong" to the same application (they reside in the same cache directory, but exist in different subdirectories).

One side effect of this is that application names can no longer have periods in them. Periods will be automatically interpreted as identifying a sub cache.

We should remove out-of-date TODO file because we are using GitHub Issues.

appdirs is required, but not noted in the README or documentation.

Currently, fcache.set() only works with one key/value pair. It should work with multiple.

fcache currently pickles data using pickle.HIGHEST_PROTOCOL. On Python 2, that value is 2. On Python 3 it is 3. That means that Python 2 programs cannot read fcache.Cache() files created in Python 3.

Give users some idea about how to use fcache by creating an examples directory with working code.

As this project progresses, there are some clear limitations to the current implementation. The Python standard library already includes a cache-like module, shelve. The shelve module includes a class, Shelf. According to the docs, a “shelf” is a persistent, dictionary-like object. A shelf has a dict object that functions as the cache itself.

fcache will now pursue a rewrite that implements a dictionary-like object using the Shelf class that uses file-based caching, including file locking, a function decorator, and full dict implementation. The rewrite will be heavily influenced by these projects: werkzeug.contrib.cache.FileSystemCache, django.core.cache.backends.filebased.FileBasedCache, shove.store.FileStore, dbdict, and dbm.

The documentation has a typo when it says "Keys and values are always stored as bytes objects. If data serialization is enabled, keys are returned as a bytes object. If data serialization is disabled, keys are returned as a str object." With serialization enabled, str keys are returned. With serialization disabled, bytes keys are returned.

Tox should also build the documentation to check for errors.

Thank you for switching to platformdirs.

FYI, should update the README accordingly. It still mentions appdirs:

https://github.com/tsroten/fcache/blob/main/README.rst

fcache version: Version: 0.4.7

Sample code:

from fcache.cache import FileCache

my_cache = FileCache(appname="test cache", flag="ns", app_cache_dir="<asb path to dir>")
my_cache["key 1"] = "value 1"

print(list(my_cache.keys()))
del my_cache["key 2"]

Output

['key 1']

Process finished with exit code 0

Even thought the cache does not contain the key 2 key, no exception is raised when attempting to delete it.
Is this an issue or a misuse from my side?

Edit: typo

I made a detailed explanation here:
https://stackoverflow.com/questions/58781265/portable-way-to-use-a-local-shared-read-write-storage-file-database-as-a-cache

But I suspect this is some kind of a bug.

Platform: Windows 7 x64
Python: x86/3.8.0

Example to repro:

test1.py

from fcache.cache import FileCache
import time
import sys

mycache = FileCache('1')

i = 0
count = 0
y = '111'

#class DummyFile(object):
#  def write(self, x): pass

while True:
  #prev_stderr = sys.stderr
  #sys.stderr = DummyFile()
  try:
    mycache['test'] = y
    mycache.sync()
    x = mycache['test']
    #sys.stderr = prev_stderr
    if x is None:
      print('FAILED', count, x)
    else:
      print('SUCCEED', count, x)
      i = 0
  except:
    #sys.stderr = prev_stderr
    # retry, has meaning on Windows
    i += 1
    if i % 10 == 0:
      # give to scheduler a break
      print('sleep 2')
      time.sleep(.02)
    continue
  count += 1

  if count % 10 == 0:
    # give to scheduler a break
    print('sleep 1')
    time.sleep(.02)

test2.py

<<<The same code as above>>>

Make README.rst easier to read by adding syntax-highlighting.

Currently, fcache.get() only allows getting one key's value at a time. Because every time fcache.get() is called, the cache file is opened and read from disk, fcache.get() should allow the user to get multiple key's values at the same time.

The README example uses 'cache_dir' for the argument when opening a cache instead of 'appname' as it should since fcache uses appdirs.

setup.py should use setuptools so that appdirs is automatically installed via pip. Also, tests could then be run via python setup.py test.

The package fcache.tests is not included in the PyPi release. setup.py needs to be modified to include this.

As a result user cannot catch an OSError exception to reget the value.
This is related to the workaround from here: #26

fcache should simplify cache creation by using appdirs to store files in appropriate locations.

In Python 2, fcache doesn't accept unicode keys. In Python 3, since unicode and str have been merged, there isn't a problem. fcache in Python 2 should be able to handle unicode keys.

There are some style problems with the code. All code should be PEP8 compatible.

Sphinx's build directory should be excluded from the package via MANIFEST.in.

Hi,

I am using fcache version 0.4.7

I am getting the following error:

  File "/root/app/collector/libs/eccache.py", line 15, in __init__
    self.cache = FileCache(name, flag='cs', app_cache_dir=cache_dir)
  File "/usr/local/lib/python2.7/site-packages/fcache/cache.py", line 121, in __init__
    self.create()
  File "/usr/local/lib/python2.7/site-packages/fcache/cache.py", line 141, in create
    os.makedirs(self.cache_dir)
  File "/usr/local/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)

The current code in README.rst uses print as a statement, not a function. README.rst code should run in all versions of Python that fcache supports (2.6, 2.7, 3.1, 3.2, and 3.3).

Traceback (most recent call last):
File "<pyshell#10>", line 1, in
cache = fcache.Cache("Temp", "Freud")
File "C:\Python32\lib\site-packages\fcache.py", line 106, in init
self._create()
File "C:\Python32\lib\site-packages\fcache.py", line 358, in _create
os.rename(tmp, self.filename)
WindowsError: [Error 32] El proceso no tiene acceso al archivo porque está siendo utilizado por otro proceso

Solucion:
Poner antes de la linea 358, os.close(f)