GithubHelp home page GithubHelp logo

data_cache's People

Contributors

fransisnk avatar leifwar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

data_cache's Issues

Clear cache

Hi, how can I clear all the cache? Thank you

caching error in pandas cache

@pandas_cache
def get_sentiment(text_content: str):
    """
    Classifying Content in a String

    Args:
      text_content The text content to analyze. Must include at least 20 words.
    """

    client = language_v1.LanguageServiceClient.from_service_account_json(filename=GOOGLE_APPLICATION_CREDENTIALS)

    # text_content = 'That actor on TV makes movies in Hollywood and also stars in a variety of popular new TV shows.'

    # Available types: PLAIN_TEXT, HTML
    type_ = language_v1.Document.Type.PLAIN_TEXT

    # Optional. If not specified, the language is automatically detected.
    # For list of supported languages:
    # https://cloud.google.com/natural-language/docs/languages
    language = "en"
    document = {"content": text_content, "type_": type_, "language": language}

    response = client.analyze_sentiment(request={"document": document})
    out=analyse_response(response)
    return out

@pandas_cache
def analyse_response(response)->Tuple[str]:
    magnitude=response.document_sentiment.magnitude
    score=response.document_sentiment.score

    if -.1<magnitude<.1 or -.1<score<.1:
        label= "neutral"
        
    elif score>=.1:
        label= "positive"
        
    elif score<=-.1:
        label= "negative"

    # print("magnitude: ",magnitude)
    # print("score: ",score)
    # print("label: ",label)
    return magnitude ,score ,label

text = """i'm soo happy""".strip()

res=get_sentiment(text)

Gives the error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_6589/3336704012.py in <module>
      2 text = """i'm soo happy""".strip()
      3 
----> 4 res=get_sentiment(text)

~/miniconda3/lib/python3.7/site-packages/data_cache/cache_tools.py in wrapped(*f_args, **f_kwargs)
    210                 group = "a" + hashlib.md5(inspect.getsource(func).encode("utf-8")).hexdigest()
    211                 key = "a" + hashlib.md5(json.dumps(extracted_args).encode("utf-8")).hexdigest()
--> 212                 return table_getter(group, key, func, f_args, f_kwargs, extracted_args)
    213 
    214             return wrapped

~/miniconda3/lib/python3.7/site-packages/data_cache/cache_tools.py in store_func(func_key, arg_key, func, f_args, f_kwargs, metadata)
    172             if isinstance(data, tuple):
    173                 for i, data_ in enumerate(data):
--> 174                     store.create_dataset(f"{path}/data{i}{suffix}", data=data_)
    175                     add_metadata(store[f"{path}/data{i}"], func, metadata)
    176             else:

~/miniconda3/lib/python3.7/site-packages/data_cache/cache_tools.py in create_dataset(self, key, data)
     63 class PandasStore(pd.HDFStore):
     64     def create_dataset(self, key: str, data: pd.DataFrame) -> None:
---> 65         data.to_hdf(super(), key)
     66 
     67     def __getitem__(self, key: str) -> PandasGroup:

AttributeError: 'float' object has no attribute 'to_hdf'

system details

OS: Manjaro Linux x86_64
Host: Inspiron 5567
Kernel: 5.15.57-2-MANJARO
Packages: 1589 (pacman), 6 (flatpak), 9 (snap)
Shell: zsh 5.9
python: python3.7
Terminal: vscode
CPU: Intel i7-7500U (4) @ 3.500GHz
GPU: AMD ATI Radeon R7 M260/M265 / M340/M360 / M440/M445 / 530/535 / 620/625 Mobile
GPU: Intel HD Graphics 620
Memory: 7031MiB / 15899MiB

Cache file opened simultaneously multiple times.

Using example decorator code from README.md with pandas 1.2.4 raises ValueError exception:
ValueError: The file '/tmp/workspace_analysis/data.h5' is already opened. Please close it before reopening. HDF5 v.1.8.5-patch1, FILE_OPEN_POLICY = 'strict'

Somewhere between pandas 1.1.5 and 1.2.4 pytables has been updated, and storage file can be opened only once to prevent corruption.

Pandas==1.2.4
data-cache==0.1.5
tables==3.6.1
h5py==2.10.0

Downgrading to pandas 1.1.5 same example works.

As temporary solution PandasStore.create_dataset() can be augmented to close file before creating new hdf object:

class PandasStore(pd.HDFStore):
	def create_dataset(self, key: str, data: pd.DataFrame) -> None:
		try:
			self.close()
		except pandas.io.pytables.ClosedFileError:
			pass
		data.to_hdf(self, key)

Full stack:

(openbadge) teemu@graygoose:~/projects/startuplab/workspace_analysis3> python cache_test.py 
Traceback (most recent call last):
  File "/home/teemu/projects/startuplab/workspace_analysis3/cache_test.py", line 13, in <module>
    print(simple_func())
  File "/home/teemu/opt/miniconda/envs/openbadge/lib/python3.9/site-packages/data_cache/cache_tools.py", line 212, in wrapped
    return table_getter(group, key, func, f_args, f_kwargs, extracted_args)
  File "/home/teemu/opt/miniconda/envs/openbadge/lib/python3.9/site-packages/data_cache/cache_tools.py", line 177, in store_func
    store.create_dataset(f"{path}{suffix}", data=data)
  File "/home/teemu/opt/miniconda/envs/openbadge/lib/python3.9/site-packages/data_cache/cache_tools.py", line 65, in create_dataset
    data.to_hdf(self, key)
  File "/home/teemu/opt/miniconda/envs/openbadge/lib/python3.9/site-packages/pandas/core/generic.py", line 2606, in to_hdf
    pytables.to_hdf(
  File "/home/teemu/opt/miniconda/envs/openbadge/lib/python3.9/site-packages/pandas/io/pytables.py", line 277, in to_hdf
    with HDFStore(
  File "/home/teemu/opt/miniconda/envs/openbadge/lib/python3.9/site-packages/pandas/io/pytables.py", line 561, in __init__
    self.open(mode=mode, **kwargs)
  File "/home/teemu/opt/miniconda/envs/openbadge/lib/python3.9/site-packages/pandas/io/pytables.py", line 710, in open
    self._handle = tables.open_file(self._path, self._mode, **kwargs)
  File "/home/teemu/opt/miniconda/envs/openbadge/lib/python3.9/site-packages/tables/file.py", line 288, in open_file
    raise ValueError(
ValueError: The file '/tmp/workspace_analysis/data.h5' is already opened.  Please close it before reopening.  HDF5 v.1.8.5-patch1, FILE_OPEN_POLICY = 'strict'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.