brainix / pottery Goto Github PK
View Code? Open in Web Editor NEWRedis for humans. πππ
License: Apache License 2.0
Redis for humans. πππ
License: Apache License 2.0
Hi, this looks like a nice library, but I'm having a little trouble getting started. After
pip install pottery
I'm finding that import pottery
triggers a
ModuleNotFoundError: No module named 'typing_extensions'
Maybe typing_extensions
needs to be included in the install_requires
in setup.py
? (I have not used setuptools much, so that is just my guess.)
Thanks for the great project, very clean, very solid.
Have you considered, an asyncio implementation? or do you think that the potential complexity won't pay off for practical usage?
Thanks again!
Describe the bug
Readme claims RedisSimpleQueue
is compatible with Pythonβs SimpleQueue
, but RedisSimpleQueue().get(block=False` blocks forever
To Reproduce
Steps to reproduce the behavior:
from pottery import RedisSimpleQueue
q = RedisSimpleQueue(redis=redis, key='1')
q.get(block=False)
Expected behavior
queue.SimpleQueue()
rises an Empty
error. Expect that queue.SimpleQueue()
do the same.
Environment (please complete the following information):
OS: [e.g. macOS, Linux]
uname -a
Darwin xxx 20.6.0 Darwin Kernel Version 20.6.0: Wed Nov 10 22:23:05 PST 2021; root:xnu-7195.141.14~1/RELEASE_ARM64_T8101 arm64
Python version 3.9.15
Redis version 6.2.5
Hello.
I keep getting this error last two days. It worked fine before.
Maybe it because i did upgrade all my python libs. Any ideas how i can fix this?
Exception in thread Thread-20:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.4/dist-packages/telegram/ext/dispatcher.py", line 61, in pooled
result = func(*pargs, **kwargs)
File "main.py", line 149, in any_message
if user_id in users_dict:
File "/usr/local/lib/python3.4/dist-packages/pottery/dict.py", line 83, in __contains__
return bool(self.redis.hexists(self.key, self._encode(key)))
File "/usr/local/lib/python3.4/dist-packages/redis/client.py", line 1853, in hexists
return self.execute_command('HEXISTS', name, key)
File "/usr/local/lib/python3.4/dist-packages/redis/client.py", line 573, in execute_command
return self.parse_response(connection, command_name, **options)
File "/usr/local/lib/python3.4/dist-packages/redis/client.py", line 585, in parse_response
response = connection.read_response()
File "/usr/local/lib/python3.4/dist-packages/redis/connection.py", line 577, in read_response
response = self._parser.read_response()
File "/usr/local/lib/python3.4/dist-packages/redis/connection.py", line 238, in read_response
response = self._buffer.readline()
File "/usr/local/lib/python3.4/dist-packages/redis/connection.py", line 168, in readline
self._read_from_socket()
File "/usr/local/lib/python3.4/dist-packages/redis/connection.py", line 126, in _read_from_socket
data = self._sock.recv(socket_read_size)
File "/usr/local/lib/python3.4/dist-packages/gevent/_socket3.py", line 307, in recv
self._wait(self._read_event)
File "/usr/local/lib/python3.4/dist-packages/gevent/_socket3.py", line 147, in _wait
self.hub.wait(watcher)
File "/usr/local/lib/python3.4/dist-packages/gevent/hub.py", line 627, in wait
result = waiter.get()
File "/usr/local/lib/python3.4/dist-packages/gevent/hub.py", line 875, in get
return self.hub.switch()
File "/usr/local/lib/python3.4/dist-packages/gevent/hub.py", line 606, in switch
return greenlet.switch(self)
gevent.hub.LoopExit: ('This operation would block forever', <Hub at 0x7f23460295a0 epoll pending=0 ref=0 fileno=43>)
Hi @brainix ~
Thank you very much for your contribution first.
'RedisList' and 'list' behave inconsistently in some cases, for example:
>>> l = []
>>> l[0:0] = ['a','b', 'c']
>>> l
['a', 'b', 'c']
>>> l[:] = ['a','b', 'c'] * 20
>>> l
['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c']
Is your feature request related to a problem? Please describe.
Im currently using Serialized Redis as I use a orderedDict and much other with the package that has those features that I need. The problem is that I do not know how to use conn = serialized_redis.MsgpackSerializedRedis(host='localhost', port=6379, db=0)
in the Redlock as masters
Describe the solution you'd like
For me as a heavy user in Serialized Redis, it would be very awesome to use Redlock together with it. That would help me to solve where I instead of needing connect two redis connection to one which saves the resources and also more correctly to do.
Additional context
I have created a small script together with the serialized redis:
#!/usr/bin/python3
# -*- coding: utf-8 -*-
import time
import serialized_redis
from pottery import Redlock
conn = serialized_redis.MsgpackSerializedRedis(host='localhost', port=6379, db=0)
lock = Redlock(key='basket_test', masters={conn}, auto_release_time=15 * 1000)
try:
lock.acquire()
test = bool(lock.locked())
print("Locked test ?", test)
time.sleep(2)
test1 = bool(lock.locked())
print("Locked test1 ?", test)
lock.release()
test3 = bool(lock.locked())
print("Locked test3 ?", test)
except Exception as err:
print("Unfortunately there is an error here!", err)
which returns:
Locked test ? False
Locked test1 ? False
Unfortunately there is an error here! key='redlock:basket_test', masters=[MsgpackSerializedRedis<ConnectionPool<Connection<host=localhost,port=6379,db=0>>>]
Another code:
#!/usr/bin/python3
# -*- coding: utf-8 -*-
import time
import serialized_redis
from pottery import Redlock
from redis import Redis
conn = serialized_redis.MsgpackSerializedRedis(host='localhost', port=6379, db=0)
redis = Redis.from_url('redis://localhost:6379/0')
lock = Redlock(key='basket_test', masters={conn}, auto_release_time=15 * 1000)
try:
with lock:
print("We have locked it... I think... Let's see")
print("Locked?", bool(lock.locked()))
time.sleep(2)
print("After 'with lock', Are we released?")
print("Released?", bool(lock.locked()))
except Exception as err:
print("Unfortunately there is an error here!", err)
>>> We have locked it... I think... Let's see
>>> Locked? False
>>> Unfortunately there is an error here! key='redlock:basket_test', masters=[MsgpackSerializedRedis<ConnectionPool<Connection<host=localhost,port=6379,db=0>>>]
I assume it does it incorrectly together with serialized_redis :'(
Describe the bug
Can't install pottery when the Python version is 3.10.2
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The module can installed successfully
Environment (please complete the following information):
Additional context
Nope
Can not import AIORedlock from library:
from pottery import AIORedlock
On further inspection found out Release 3.0.0 is having wrong code altogether, there is no such entry in init.py
Release 3.0.0 is not in sync with master branch.
Is your feature request related to a problem? Please describe.
There is a case I have when using a Redlock
, where I would like to have an unlimited number of extensions.
You can currently set the maximum number of extensions when creating a Redlock
with the num_extensions
,
but this cannot be set to unlimited.
Describe the solution you'd like
Following the other conventions, setting num_extensions
to -1 for unlimited would work.
my_lock = Redlock(
key="my_key",
masters={my_master},
num_extensions=-1 # Allow unlimited calls to my_lock.extend()
)
Describe alternatives you've considered
You can set an arbitrarily large int for num_extensions
but it's not as clean.
Additional context
N/A
I cant use with
.
console output
Traceback (most recent call last):
File "/Users/fanren/pokerwinners/config/redis_info.py", line 57, in <module>
r_lock.release()
File "/opt/miniconda3/envs/pkwinners/lib/python3.8/site-packages/pottery/redlock.py", line 673, in __exit__
self.__release()
File "/opt/miniconda3/envs/pkwinners/lib/python3.8/site-packages/pottery/redlock.py", line 595, in release
raise ReleaseUnlockedLock(
pottery.exceptions.ReleaseUnlockedLock: ('redlock:printer', frozenset({Redis<ConnectionPool<Connection<host=127.0.0.1,port=6379,db=2>>>}))
printer_lock is locked
False
printer_lock is locked
In the current version of your code, the prefix "redlock:" is systematically added to any Redlock lock key created with:
object_lock = Redlock(key=lock_key, masters={redis_1, redis_2, redis_3}, auto_release_time=20)
This imposes a constraint on other programs using the lock key to coordinate access to a resource, namely to use the same prefix in front of the key. This is very inconvenient since existing code needs to be changed to adapt to this requirement.
Moreover, the use of the prefix is not documented (I discovered it by looking at the content of the Redis servers).
Is there a way to use the Redlock without imposing a specific prefix? Ideally, if a prefix is added, it should be a parameter when creating the Redlock object.
When using RedisDict with an instance of redis.cluster.RedisCluster() - a crash occurs
in base.py:
Fraceback (most recent call last):
File "/app/poc3.py", line 28, in <module>
demo()
File "/app/poc3.py", line 24, in demo
data.update({'key': 'value'})
File "/usr/local/lib/python3.9/site-packages/pottery/dict.py", line 144, in update
with self._watch(arg) as pipeline:
File "/usr/local/lib/python3.9/contextlib.py", line 119, in __enter__
return next(self.gen)
File "/usr/local/lib/python3.9/site-packages/pottery/base.py", line 232, in _watch
for context_manager in self.__context_managers(*others):
File "/usr/local/lib/python3.9/site-packages/pottery/base.py", line 220, in __context_managers
redises[connection_args(container.redis)].append(container)
File "/usr/local/lib/python3.9/site-packages/pottery/base.py", line 89, in connection_args
redis.connection_pool.connection_kwargs['host'],
AttributeError: 'RedisCluster' object has no attribute 'connection_pool'
To Reproduce
here is simple code to reproduce:
import os
from pottery import RedisDict
from redis.cluster import RedisCluster
env = os.environ
redis = RedisCluster(
host=env.get('REDIS_HOST'),
port=int(env.get('REDIS_PORT')),
username=env.get('REDIS_USER'),
password=env.get('REDIS_PASSWORD'),
charset='utf-8',
ssl_cert_reqs='none',
ssl=True,
skip_full_coverage_check=True
)
def demo():
data = RedisDict(key=f"data:test", redis=redis)
data.update({'key': 'value'})
if __name__ == "__main__":
demo()
Expected behavior
I expect it to work same regardless if it is a cluster or a single Redis instance :-/
Environment (please complete the following information):
Hello.
Why this doesnt work and how can i get it work?
>>> from pottery import RedisDict
>>> users_dict = RedisDict(redis=redis, key='users_dict')
>>> users_dict
RedisDict{34324144: {'lastname': '', 'notification': 0, 'accesslevel': 1, 'firstname': 'First Name', 'username': 'user'}}
>>> users_dict[34324144]['accesslevel']
1
>>> users_dict[34324144]['accesslevel'] = 4
>>> users_dict[34324144]['accesslevel']
1
is there any way not to add key, value in loop and do it like this :
ned = RedisDict(redis=redis, key={'key': 'val'})
Following code throws:
Exception ignored in: <bound method RedisDict.__del__ of RedisDict{}> Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/pottery/base.py", line 48, in __del__ if self.key.startswith(self._RANDOM_KEY_PREFIX): AttributeError: 'dict' object has no attribute 'startswith'
Python 3.5.2 (default, Jul 17 2020, 14:04:10)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from redis import Redis
>>> redis = Redis.from_url('redis://localhost:6379/')
>>> from pottery import RedisDict
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/lex/.local/lib/python3.5/site-packages/pottery/__init__.py", line 37, in <module>
from .bloom import BloomFilter # isort:skip
File "/home/lex/.local/lib/python3.5/site-packages/pottery/bloom.py", line 71
**kwargs: Any,
^
SyntaxError: invalid syntax
and if i fix comma error :
>>> from pottery import RedisDict
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/lex/.local/lib/python3.5/site-packages/pottery/__init__.py", line 37, in <module>
from .bloom import BloomFilter # isort:skip
File "/home/lex/.local/lib/python3.5/site-packages/pottery/bloom.py", line 263
bit_offsets: Set[int] = set()
^
SyntaxError: invalid syntax
And im not sure how should i fix this one
later:
Its variable annotation, which was implemented in 3.6 and i have earlier 3.5 python so it rises exception .
Closing issue now
Describe the bug
Redis client has option to auto-decode responses (decode_responses
) - when it is set to true, it seems to conflict with
pottery
To Reproduce
Steps to reproduce the behavior:
>>> from redis import StrictRedis
>>> from pottery import RedisDict
>>> conn = StrictRedis(host='redis', charset="utf-8", decode_responses=True)
>>> tel = RedisDict({'jack': 4098, 'sape': 4139}, redis=conn, key='tel2')
>>> tel
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.9/site-packages/pottery/dict.py", line 116, in __repr__
dict_ = {self._decode(key): self._decode(value) for key, value in items}
File "/usr/local/lib/python3.9/site-packages/pottery/dict.py", line 116, in <dictcomp>
dict_ = {self._decode(key): self._decode(value) for key, value in items}
File "/usr/local/lib/python3.9/site-packages/pottery/base.py", line 154, in _decode
decoded: JSONTypes = json.loads(value.decode('utf-8'))
AttributeError: 'str' object has no attribute 'decode'
>>>
Expected behavior
I expect it figure out when to decode
Environment (please complete the following information):
Are there any plans to support newer versions of Python?
I see there have been updates to the library to support them, but it's not released yet.
We are currently blocked from upgrading to newer versions of Python because of this package.
When trying to replicate the first example I get this error:
redis.exceptions.ResponseError: Command # 1 (HSET tel "jack" 4098 "sape" 4139) of pipeline caused error: wrong number of
arguments for 'hset' command
class RedisContext:
def __init__(self):
self.redis = Redis.from_url("redis://localhost:6379/0")
def __enter__(self):
return self.redis
def __exit__(self, exc_type, exc_value, traceback):
self.redis.close()
with RedisContext() as redis:
tel = RedisDict({'jack':4098, 'sape':4139}, redis=redis, key='tel')
print(tel)
If instead I do:
with RedisContext() as redis:
tel = RedisDict({'jack':4098}, redis=redis, key='tel')
print(tel)
Works fine.
How to use redis-cluster with RedLock, any example ?
Describe the bug
Sometimes I need to acquire and release the same lock with different objects. I am constricted by a legacy design to do it this way, because acquiring and releasing the lock will be in separate processes. However, if I open a second handle to the same lock key, I can't release the lock from there.
To Reproduce
Python 3.6.5 (default, Mar 25 2020, 10:32:15)
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pottery
>>> import redis
>>> lock = pottery.Redlock(key="test-lock", masters={redis.Redis()}, auto_release_time=40000)
>>> lock.acquire()
True
>>> lock.locked()
33200
>>> lock2 = pottery.Redlock(key="test-lock", masters={redis.Redis()}, auto_release_time=40000)
>>> lock.locked()
23395
>>> lock2.locked()
0
>>> lock2.release()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/arieroos/Workspace/dos/venv/lib/python3.6/site-packages/pottery/redlock.py", line 572, in release
redis_errors=redis_errors,
pottery.exceptions.ReleaseUnlockedLock: key='redlock:test-lock', masters=[Redis<ConnectionPool<Connection<host=localhost,port=6379,db=0>>>], redis_errors=[]
Expected behavior
Python 3.6.5 (default, Mar 25 2020, 10:32:15)
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pottery
>>> import redis
>>> lock = pottery.Redlock(key="test-lock", masters={redis.Redis()}, auto_release_time=40000)
>>> lock.acquire()
True
>>> lock.locked()
33200
>>> lock2 = pottery.Redlock(key="test-lock", masters={redis.Redis()}, auto_release_time=40000)
>>> lock.locked()
23395
>>> lock2.locked()
13395 # or some integer like that
>>> lock2.release()
# no exception thrown
Environment (please complete the following information):
Additional context
Currently I just ignore the Exception, as most locks will expire in time anyway. However, that is probably bad practice.
When using a Redlock with blocking acquire
or the syncronize
decorator, if Redis is down, the acquire
will keep being retried until Redis is back up. This could block a program indefinitely if the Redis connection cannot be recovered.
An extra argument for syncronize
and Redlock
that enables re-raising Redis errors (error will be raised if the argument is set to True and if the Redis errors happen for the majority of the masters) would provide better control so that the program can decide what to do in this case, instead of remaining blocked.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Hello. Does it now support recursive lock? Or is there a plan to implement it? Tks.
Having this file ties the repository to a specific virtualenv (assuming you're using pyenv-virtualenv), which means contributors have to use workarounds (i.e. deleting the file and remembering not to commit the change).
I use pyenv but not pyenv-virtualenv; this file breaks pyenv for me since it can't find the pottery
version.
Please ignore this issue if you don't agree, just a minor pain point!
Is your feature request related to a problem? Please describe.
I typically use(and advise others to use) the with
statement for creating a redlock instance, to avoid forgetting to release it. Using the with statement, you lose the ability to set lock acquiring semantics and timeout on the lock because you no longer call acquire() yourself but is automatically called in Redlock.__enter__()
. So you are stuck with the default blocking indefinitely semantic.
Describe the solution you'd like
from pottery import Redlock
from functools import partial
# Avoid passing around redis clients everywhere, by creating a partial
# IMHO helps keeps the lock interface smaller at the time of use, where a dev on the team
# doesn't need to worry about the redis client.
Lock = partial(Redlock, masters={redis})
NonBlockingLock = partial(Lock, blocking=False)
BlockingLock = partial(Lock, blocking=True, timeout=10)
# same as it works currently, with the defaults set
with Lock(key='printer'):
critical_section()
with BlockingLock(key='printer'):
critical_section()
with NonBlockingLock(key='printer'):
another_critical_section()
Describe alternatives you've considered
from contextlib import ContextDecorator
from pottery import Redlock
class Lock(ContextDecorator):
def __init__(self, key: str, release_after_seconds, blocking=False, blocking_timeout_seconds = -1):
self._lock = Redlock(
key=self.key,
masters={redis},
auto_release_time=release_after_seconds * 1000, # convert to milliseconds
)
self.blocking = blocking
self.blocking_timeout_seconds = blocking_timeout_seconds
def __enter__(self):
if not self._lock.acquire(blocking=self.blocking, timeout=self.blocking_timeout_seconds):
raise LockError
return self
def __exit__(self, *exc):
self._lock.release()
return False
__enter__
and __exit__
would also work? But i saw a lot of name mangling happening, so wasn't sure i was going to mess something up. But will also try that solution once.Additional context
This is more of a nice-to-have, since it's all possible to build on top of the Redlock primitive. I do think the context-manager is the proper way to use these features with block scoping, since in a larger dev team it's all too common for people to forget to call lock.release() or close file pointers etc.
IMHO, it might also make the context-manager interface slightly more expressive and be able to modify behaviour using functools.partial
. I'm all ears to see what you think, and issues you see.
PS: If you see merit in this request, i can send a PR.
Use more moderate language, and update the pottery
module docstring (in __init__.py
) to match the description in README.md
.
Is your feature request related to a problem? Please describe.
GitHub CI doesn't yet support Python 3.10.0 final, so we currently build against Python 3.10.0-rc.2.
Describe the solution you'd like
Once GitHub CI supports Python 3.10.0 final, build against it.
Describe alternatives you've considered
N/A
Additional context
N/A
Describe the bug
The code crashes when importing pottery with PYTHONOPTIMIZE=2 set.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Successful import
Environment (please complete the following information):
The AIORedlock code is not yet released.
Describe the bug
Cannot import name 'Script' from 'redis.client' with 'redis-4.1.0'
To Reproduce
Use pottery with 'redis-4.1.0'
Additional context
'redis.client.Script' has been removed.
Look at the following code
redis/redis-py@12f45ee#diff-3161b005adfa1f5266a9736bffb5ede9d14e2ab40937cb241bc16ca5647df79d
Hi thanks for the great project.
From what I've observed it looks like objects stored in say a RedisDict are "immutable" in the sense that if I were to want to store a dict in a RedisDict (e.g. to implement a Redis backed shared dictionary of objects) that'd be fine, but if I wanted to change an element of any stored object I'd have to do something like:
tmp = redis_dict[key]
tmp['field'] = "value"
redis_dict[key] = tmp
For modest sized objects that workaround seems fine, but as objects get larger I wonder about the efficiency?
Things get more awkward as one use case I've got is what amounts to a dict of lists and I'd like that to be backed by Redis so multiple instances of my application can see the same dict of lists.
Now I know that I could keep writing the list to the redis_dict and from a functional perspective I think it'd work, but my suspicion is that it'd get horribly inefficient pretty quickly and as the number of items in the list grows at a guess what'd end up happening is an increasingly expensive JSON serialisation of the list.
As far as I can tell it's not possible for a RedisDict to contain a RedisList (when I tried it barfed with a JSON error IIRC)
Do you have any thoughts on how to do this sort of thing efficiently - I'm no Redis expert and one of the reasons I looked at pottery was to try and avoid having to to get too much into the weeds of Redis but now I'm wondering whether I might as well keep digging :-)
I'm guessing RedisList is implemented as, well, a Redis list with the values of the list JSON serialised so appending an item directly to a RedisList should only be as expensive as the serialisation cost for that Item, but how to model a dict of those efficiently (and intuitively) kind of escapes me.
I guess from the RedisList example
lyrics = RedisList(redis=redis, key='lyrics')
the key is, well key - so to model something like
{"a": [], "b": [], "c": [], "d": [] .......}
Do I simply have
{"a": RedisList(redis=redis, key="a"), "b": RedisList(redis=redis, key="b", "c": RedisList(redis=redis, key="c", "d": RedisList(redis=redis, key="d" .......}
Though as the containing dict isn't stored how would other instance know about the keys
So I'm thinking the only way to do this is by dereferencing - so I have a RedisSet holding the keys and if I want to "look up" a list I first check the key is in the dict and if it isn't I look it up in the RedisSet (cause it might have been set by another instance in the cluster) and if the key is in the RedisSet I create a RedisList instance using that key and add it to my dict.
I think that'd work, but is rather less transparent than a dict of lists idiom that it is trying to model and loses some of the benefit of what is supposed to be a more pythonic container abstraction.
Do you have any thoughts? Is my observation about right or have I missed something?
To be clear I'm not being critical, I'm just thinking out loud about how to model a shared dict of lists in a more efficient way than JSON serialising the entire list for every insert and would really appreciate thoughts on this problem.
MTIA
@brainix Recently,I had been using your lib.I found it's a smart and easy tool to a redis freshman.But I want to use sorted sets,and I can not find the way.Could you please help me?
There is some reader writer problems when you develope a web application, for example a django application witch sells tickets and the problem become more complicated when you need to add cache to your application.
user_A request for available tickets user_B changer available tickets, but user_B's response posts first and empty dirty cache but user_A' response old data will dirty cache while caching the response data.
I like to have some reader writer function or decorator to stop multiple writers or a writer and a reader executing webservice+cache to stop such problems to take place.
it can be implemented by locks a counter.
I think there is 2 policies to face this on reader writer problem wait for readers to stop then let the writer do his job or don't let readers go in while there is a writer waiting or executing.
This library looks great! I was going to need something like this in about a week so thank you for saving me time :-)
I was wondering why not discard set members with SREM rather than SMOVE+DEL.
Exception ignored in: <bound method _Common.__del__ of MyRedisDict>
Traceback (most recent call last):
File "/home/axion/envs/arbitrage/lib/python3.5/site-packages/pottery/base.py", line 43, in __del__
if self.key.startswith(self._RANDOM_KEY_PREFIX):
TypeError: startswith first arg must be bytes or a tuple of bytes, not str
pottery-0.36-py3-none-any.whl
>>> from pottery import RedisSet # or other DS
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/.local/lib/python3.4/site-packages/pottery/__init__.py", line 35, in <module>
from .contexttimer import ContextTimer
File "/home/user/.local/lib/python3.4/site-packages/pottery/contexttimer.py", line 13, in <module>
from tests.base import run_doctests
ImportError: No module named 'tests.base'
Is your feature request related to a problem? Please describe.
There is room for a RedisJSON "proxy object" that would stay in sync with its Redis JSON value.
Describe the solution you'd like
>>> obj = RedisJSON({"a": 1}, redis=redis, key="json_document")
>>> obj["b"] = {"c": [2]}
>>> print(redis.json().get("json_document")
{"a": 1, "b": {"c": [2]}}
>>> redis.json().arrappend("json_document", Path("b.c"), 3)
>>> print(obj["b"]["c"])
[2, 3]
(May be we could even limit assignments to JSON/JavaScript-valid only types like string, int, float, boolean, null.)
Describe alternatives you've considered
RedisDict{}
Additional context
Not applicable.
Currently, monkey.py calls _default.default(obj)
.
But, _default.default
is json.JSONEncoder.default
, which is an unbounded method, so it needs two arguments for a successful call. A possible solution is to define _default.default = json.JSONEncoder().default
instead.
How to reproduce:
import json
import pottery
json.dumps(object())
Will throw an error
TypeError: default() missing 1 required positional argument: 'o'
Instead of an error
TypeError: Object of type 'object' is not JSON serializable
Describe the bug
If we define two objects of PotteryRedLock with same key (for example at different app instances), their locked()
methods does not show states of other object
To Reproduce
Steps to reproduce the behavior:
PotteryRedLock(key='key', masters=masters, auto_release_time=60).acquire()
PotteryRedLock(key='key', masters=masters).locked()
locked()
will return 0
, that means lock could be acquired, but second try to acquire will blocks for up to 60 seconds
Expected behavior
locked()
returns value close to auto_release_time
of first lock
Environment (please complete the following information):
OS: [e.g. macOS, Linux]
uname -a
Darwin xxx 20.6.0 Darwin Kernel Version 20.6.0: Wed Nov 10 22:23:05 PST 2021; root:xnu-7195.141.14~1/RELEASE_ARM64_T8101 arm64
Python version 3.9.15
Redis version 6.2.5
Hi there!
I would like to use async RedLock from this package, but it looks like AIORedlock was added after the last release. Could you please tell me when you plan to release the next version containing AIORedlock?
Is your feature request related to a problem? Please describe.
I'm trying to use async redis with cluster (redis.asyncio.cluster) which is only supported on versions 5+ and at the same time I want to use pottery redlock but pottery only supports versions between 4-5 no included 5
Describe the solution you'd like
Pottery to support python redis package with version 5.0.0 (5+ is optional)
Describe alternatives you've considered
Copy the redlock code in pottery and use it in my code without the need for the pottery package.
Is your feature request related to a problem? Please describe.
I'd like easy support through python of Redis SortedSet, very much like like Sets.
>>> from pottery import RedisSortedSet
>>> cars = RedisSortedSet({3: "fast", 2: "faster", 1: "fastest"}, redis=redis, key="cars")
>>> cars.add({4: "slow"})
>>> cars.remove({2: "faster"})
>>> cars
RedisSortedSet({1: "fastest", 3: "fast", 4: "slow"}
>>> fastest = cars.pop()
>>> fastest
{1: "fastest"}
Describe the solution you'd like
A mapping between the most appropriate python entity and a Redis Sorted Set, may be a dict of {sorted_criteria: value}?
Describe alternatives you've considered
Using Sorted Set directly
Additional context
None
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.