coleifer / walrus Goto Github PK
View Code? Open in Web Editor NEWLightweight Python utilities for working with Redis
Home Page: https://walrus.readthedocs.org/
License: MIT License
Lightweight Python utilities for working with Redis
Home Page: https://walrus.readthedocs.org/
License: MIT License
If I search for a particular integer field, I get results. If I search for a Boolean field, I also get it. But if I try to join the two queries the return is a mistake.
#Presence REDIS DB (no-SQL)
db = walrus.Database(host='localhost', port=6379, db=0)
class Presence(walrus.Model):
__database__ = db
channel_name = walrus.TextField(primary_key=True)
is_accessible = walrus.BooleanField(index=True, default=False)
state = walrus.IntegerField(default=MatchState.CLOSED.value, index=True)
>>> Presence.query(Presence.is_accessible == True).__next__()
<Presence: defaultuaGTz>
>>> Presence.query(Presence.state == 4).__next__()
<Presence: defaultuaGTz>
>>> Presence.query(Presence.state == 4 & Presence.is_accessible == True).__next__()
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/models.py", line 819, in query
result = executor.execute(expression)
File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/query.py", line 240, in execute
return self._mapping[op](expression.lhs, expression.rhs)
File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/query.py", line 243, in execute_eq
index = lhs.get_index(OP_EQ)
AttributeError: 'Expression' object has no attribute 'get_index'
Hello,
I think it will be more "Pythonic" if timeout parameter could be a datetime.timedelta
https://walrus.readthedocs.org/en/latest/cache.html#basic-usage
Kind regards
Its better to have an option to specify useStopWords True/False parameter before indexing phrases.
If useStopWords is True, option to provide custom stopwords file
Hello,
what do you think about integrating your library with aioredis, would it be possible ?
When we update a data struct, a coroutine is scheduled in background and will update redis as soon as possible. One caveat would be to cancel previous updates in redis if we updates the variable multiple time...
danmu_limit = redis_db.rate_limit('danmu_10s_limit',limit=2,per=10)
if try danmu_limit.limit('key') four times in 10seconds, i will get the result as follows:
1,False
2,False
3,True
4,False
Hi,
Is there any chance of something like a nestedfield ? I guess, this is sort of equivlent to a OneToOne field in Django or SQLAlchemy:
class ScheduleModel(BaseModel):
interval = walrus.FloatField()
start_at = walrus.DateTimeField()
end_at = walrus.DateTimeField()
class ScheduledJobModel(BaseModel):
name = walrus.TextField(primary_key=True)
schedule = NestedField(ScheduleModel)
.... other fields...
So in this case once I had an instance of ScheduledJobModel I could do something like job.schedule.interval
or maybe even job.schedule__interval
.
?
fixed in #26
Hi,
Is there any chance of adding TimeDeltaField ?
Cheers
S
If the cache is empty , the flush method
def flush(self):
"""Remove all cached objects from the database."""
return self.database.delete(*self.keys())
fails
File "/root/python/project/test/great_test.py", line 22, in setUp
version_properties.cache.flush()
File "/root/.virtualenvs/project/lib/python2.7/site-packages/walrus/cache.py", line 80, in flush
return self.database.delete(*self.keys())
File "/root/.virtualenvs/project/lib/python2.7/site-packages/redis/client.py", line 841, in delete
return self.execute_command('DEL', *names)
File "/root/.virtualenvs/project/lib/python2.7/site-packages/redis/client.py", line 573, in execute_command
return self.parse_response(connection, command_name, **options)
File "/root/.virtualenvs/project/lib/python2.7/site-packages/redis/client.py", line 585, in parse_response
response = connection.read_response()
File "/root/.virtualenvs/project/lib/python2.7/site-packages/redis/connection.py", line 582, in read_response
raise response
ResponseError: wrong number of arguments for 'del' command
This is pretty minor, however: thank for providing this awesome library
I would like to know if I can select a random object when doing a query? And also how do I lock a query so I do not have records conflict?
In django models I used the "select_for_update" function to ensure a single query, and an "order_by ('?') For randomization.
queryset = User.objects.select_for_update(nowait=True).filter(state=1).order_by('?')[:1]
Hello Charles,
I did evaluation of several object mappers for Redis - walrus, rom, redisco, python-stdnet.
Walrus fit best for my needs except one issue below.
Let's save two records:
class User(Model):
#database = db
login = TextField(primary_key=True)
email = TextField(index=True)
password = TextField()
roles = SetField()
created = DateTimeField(default=datetime.utcnow)
@fixture
def db(redis_real):
db = Database(host='localhost', port=16380, db=0)
User.database = db
def test_walrus(db):
User.create(login='alex.smith', email='[email protected]')
User.create(login='anna.smith', email='[email protected]')
...
After that done we will have following data in redis for walrus==0.2.0
> KEYS *
1) "user:container.roles.user:id.alex.smith"
2) "user:[email protected]"
3) "user:all"
4) "user:[email protected]"
5) "user:login.absolute.alex.smith"
6) "user:id.alex.smith"
7) "user:id.anna.smith"
8) "user:login.absolute.anna.smith"
On my project we have 30K+ online users and it would be much easier to explore data in Redis using Web UI if we would have following key convention:
> KEYS *
1) "user:container.roles.user:id:alex.smith"
2) "user:email.absolute:[email protected]"
3) "user:all"
4) "user:email.absolute:[email protected]"
5) "user:login.absolute:alex.smith"
6) "user:id:alex.smith"
7) "user:id:anna.smith"
8) "user:login.absolute:anna.smith"
We are using Redis Commander (and Redis Admin UI in past) which displays flat key-value storage as a tree.
With current convention if I try to expand values collapsed next to "user:" I will see about 90K entries:
Is it possible somehow customize current convention, so they have more hierarchical view?
In such case I would see following prefixes (with counters next to them):
http://joeferner.github.io/redis-commander/
https://github.com/ServiceStackV3/mythz_blog/blob/master/pages/381.md
And big "thank you" for your effort on this project. It's awesome.
Given the following snippet:
from walrus import Database
db = Database(host='localhost', port=6379, db=0)
cache = db.cache(default_timeout=0)
@cache.cached()
def foo():
pass # return something expensive
the result of bar is cached for 3600s. I'd expected that it is cached without TTL.
To cache without TTL, I need to call
@cache.cached(timeout=0)
def bar():
pass # return something expensive
This was a bit unexpected since I thought default_timeout=0
will be used always if there is no other timeout specified.
If you have the time, would you mind providing a quick example on how to get/set field values from within a subclass of the Model Class? I cannot seem to figure it out. Maybe I am just overthinking it ๐ Thanks in advance for any advice you can offer!
Cheers!
Looks like the string being joined into the key is using a bytestring, which causes issues on decode.
Quick script to reproduce:
>>> from walrus import Database
>>> from walrus.graph import Graph
>>> db = Database()
>>> graph = db.graph('foo')
>>> graph.store(s=u'subject', p=u'predicate', o=u'object')
>>> query = graph.query(p=u'predicate', o=u'object')
>>> next(query)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "\site-packages\walrus\graph.py", line 146, in query
start, end = self.keys_for_query(s, p, o)
File "\lib\site-packages\walrus\graph.py", line 133, in keys_for_query
return key(parts + ['']), key(parts + ['\xff'])
File "\lib\site-packages\walrus\graph.py", line 116, in <lambda>
key = lambda parts: '::'.join(parts)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
I'm looking to have a way to have a lock scoped to a single instance of a Model. It seems like a LockField construct might be the best way to handle something like that. But I just thought I'd get your opinion.
If you think it would be best to keep separate as it's own library, that's no problem. I just wanted to make sure I reached out before starting to explore the implementation.
It would be cool to add support for IN
queries on subclasses of ContainerField
that support it. For instance,
walrus = Walrus()
class Entry(Model):
database = walrus
namespace = 'myapp'
title = TextField(index=True)
content = TextField(
fts=True,
stemmer=True,
metaphone=True)
tags = SetField(index=True) # "index"
Currently you cannot add an index to a container field, nor do these fields support any operations. It makes sense, though, to support something like:
Entry.query(Entry.tags.contains('python'))
Hi. Excuse my English, I'm with the translator.
I need to store in memory the UUID (pk) of the relational database user. This needs to be used to locate the user in the redis inside querys and also to "open" the relational database user.
I was wondering if I use walrus.TextField() or walrus.UUIDField ).
Using UUIDField I'm getting an error when I try to retrieve records.
>>> import walrus
>>> import uuid
>>> db = walrus.Database(host='localhost', port=6379, db=0)
>>> class X(walrus.Model):
database = db
name = walrus.TextField(primary_key=True)
oi = walrus.UUIDField()
>>> X.create(name='Charlie', oi=uuid.UUID('2ad3afaf-cd21-4842-b51d-c8e0bc80860d'))
<X: Charlie>
>>> test = X.load('Charlie')
**Traceback (most recent call last):**
File "<input>", line 1, in <module>
File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/models.py", line 887, in load
data[name] = field.python_value(raw_data[encode(name)])
File "/Volumes/MacHD/Users/Git/projectXYZ/.env/lib/python3.6/site-packages/walrus/models.py", line 259, in python_value
return uuid.UUID(value)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/uuid.py", line 137, in __init__
hex = hex.replace('urn:', '').replace('uuid:', '')
TypeError: a bytes-like object is required, not 'str'
Some details:
I'm in Python 3.6
There will actually be 3 UUID fields, none of them will be primary_key and everyone will need to include in query.
At the time of writing runtests.py
produces
FAILED (failures=28, errors=13)
when run under Python 3.6.
Most of these errors look like
======================================================================
FAIL: test_basic_apis (walrus.tests.TestList)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vaultah/walrus/walrus/tests.py", line 890, in test_basic_apis
self.assertList(['ix', 'i1', 'i2', 'i3'])
File "/home/vaultah/walrus/walrus/tests.py", line 884, in assertList
self.assertEqual(list(self.lst), expected)
AssertionError: Lists differ: [b'ix', b'i1', b'i2', b'i3'] != ['ix', 'i1', 'i2', 'i3']
First differing element 0:
b'ix'
ix
- [b'ix', b'i1', b'i2', b'i3']
? - - - -
+ ['ix', 'i1', 'i2', 'i3']
We should definitely make Walrus more Py3k friendly. Why don't we use six for that?
I want to at least try to initiate a general discussion of the collaborative efforts needed.
Using the following code I've tried to do a basic benchmark on updating instances of Models. Using the following code I get between 1400 and 1900 updates per second. I've also run benchmarks without as many function calls but see similar results.
Given the advertised Redis numbers around SETs and GETs this seems a couple of orders of magnitude too low. Perhaps I'm doing something obviously wrong?
# Standard Library
import time
from random import random, randint
# Third Party
from walrus import (Walrus,
Model,
TextField,
IntegerField)
class Agent(Model):
database = None
agent_id = IntegerField(primary_key=True)
location = TextField()
def get_random_coordinate():
return randint(0, 325) + random()
def get_random_coordinates():
x = get_random_coordinate()
y = get_random_coordinate()
theta = get_random_coordinate()
return str(x) + str(y) + str(theta)
def main():
redis_host = "localhost"
redis_port = 6379
agent_db = Walrus(host=redis_host, port=redis_port, db=0)
# Provide databases to our models
Agent.database = agent_db
# Create 100 agents
for i in range(1, 101):
agent = Agent.create(agent_id=i,
location=get_random_coordinates())
# Permute their locations as quickly as possible
while True:
start_time = time.time()
counter = 0
while time.time() < (start_time + 1):
aid = randint(1, 100)
agent = Agent.load(aid)
agent.location = get_random_coordinates()
agent.save()
counter += 1
print counter
if __name__ == '__main__':
main()
Models and cache in one package! I have used it in a couple of Python 2.7 projects.
Unfortunately, my next project has to be Python 3, because the other 2 pieces are only Py3 compatible :(
So I was wondering what the current state of the Python 3 compatibility was (what works, what doesn't), or if I should look at a different package for this one.
I saw ticket #8
Thanks again for releasing this, it made my life a lot easier in the past!
Geoff
Have a default argument for name seems like an accident(al namespace collision) waiting to happen. Make it compulsory would at least force you to think about it. Its a design decision, not a bug, but I though I would flag it, I know this is an issue that's come up at work.
Is there an easy way to return all results if no partial phrase is entered? Something like
>>> phrases = ['aa bb', 'aa cc', 'bb cc', 'bb aa cc', 'cc aa bb']
>>> for phrase in phrases:
... ac.store(phrase)
>>> ac.search('')
['aa bb', 'aa cc', 'bb cc', 'bb aa cc', 'cc aa bb']
Currently from what I see, I can only delete a concrete instance for fetched model:
http://walrus.readthedocs.org/en/latest/models.html#creating-updating-and-deleting
It would be nice to make a delete
by query, without fetching documents from the DB.
Updating an attribute of an object does not affect the results of a query for that object. Here is a minimal example demonstrating the unexpected behavior.
python shell
from walrus import *
db = Database(host='127.0.0.1', port=6379, db=0)
class Test(Model):
database = db
myattr = BooleanField(index=True)
# Create an object
test = Test(myattr=True)
# Query for it
list(Test.query(Test.myattr==True)) # [<Test: 1>]
# Update the attribute
test.myattr=False
test.save()
# Query again - UNEXPECTED RESULT
list(Test.query(Test.myattr==True)) # [<Test: 1>]
test.myattr # False
Hi,
I have a question regarding the Model class that walrus offers. I need to keep the state of a small set of python complex data structures for a small period of time. I have thought to use
walrus as a nice way to organize those data for easier manipulation when needed (each python object spans into 3 walrus models) and persist them in redis. Since I only care for those objects for
a specific period of time, I was wondering if there is any way to specify an expiration time for every Model instance? I have tried using cache as database but it doesn't work.
Another idea I have is to handle by specifying a maxmemory and maxmemory-policy in redis instance. Finally another idea is to have a cron like job to delete least recent model instances, which to be honest I want to avoid.
Do you have any suggestion on how to handle this ?
Thank you in advance.
Hi,
Is it possible to create a method so that it is possible to return a random object after a query?
Today there is already the method to get the first one in the queue:
user = User.query(filter1=="1").__next__()
It would be interesting, if possible, to create something like:
user = User.query(filter1=="1").__random__()
I had a need to store JSON encoded data in a Walrus Hash, so I hand coded a class to do that, but I was wondering if there would be more interest for something like this.
Example:
>>> hash = redisconnection.Hash(codec=json)
>>> hash["hello"] = dict(place="world", greeting="hey")
# Dict is stored as a JSON
>>> hash["hello"]
{"place": "world", "greeting": "hey"}
>>>
So my question:
Is there any interest for a PR that would implement that for all containers?
The codec would be any module or object that implements both loads
and dumps
function/method.
First of all - brilliant work. Walrus is one of my favourites tools right now:)
It's not an issue but rather a question / suggestion. Maybe it'd be nice to chave a CHANGELOG file, as I can see walrus is under active development and I'm really interested in how it goes. What do you think ?
re-running with more debugging enabled to see why it only happens for 1 of the entries
Traceback (most recent call last): File "redis_indexer.py", line 21, in <module>
populate_index()
File "redis_indexer.py", line 18, in populate_index
ac.store(obj_id=product['sku'], title=product['name'])
File "/root/open-data-set/.venv/lib/python3.6/site-packages/walrus/autocomplete.py", line 136, in store
clean_title = ' '.join(self.tokenize_title(title))
File "/root/open-data-set/.venv/lib/python3.6/site-packages/walrus/autocomplete.py", line 60, in tokenize_title
phrase = re.sub('[^a-z0-9_\-\s]', '', phrase.lower())
AttributeError: 'int' object has no attribute 'lower'
Hello. I'm new with Walrus and Redis and I don't know how can I set timeout for Publisher/Subscriber.
My current rough solution is monkey patching for redis.Connection.init and set self.socket_timeout there. But, obviously, I don't like this solution.
Thanks in advance and let me know if I have to move this question to SO or so.
Hi,
I'm having a hard time figuring out whether redis-py be a hard dependency or not. What threw me off guard is (database.py):
try:
from redis import Redis
except ImportError:
Redis = None
My problem occurs when installing walrus in a brand new env. Quoting the doc:
>>> git clone https://github.com/coleifer/walrus.git
>>> cd walrus
>>> python setup.py install
Traceback (most recent call last):
File "setup.py", line 17, in <module>
version=__import__('walrus').__version__,
File ".../walrus/walrus/__init__.py", line 34, in <module>
from walrus.database import Database
File ".../walrus/walrus/database.py", line 42, in <module>
class Database(Redis):
TypeError: Error when calling the metaclass bases
cannot create 'NoneType' instances
When the setup conf looks for the version in the init.py, Database is imported and since redis-py isn't installed (yet), the Database class definition fails.
So, here come my two cents on suggestions:
ImportError
, set Redis to object
rather than None
install_requires=["redis"]
in the setup conf?I'd be happy to PR if you let me know what you think!
And, just for the record, I've been a happy user of redis-completion for a few years, and I am looking forward to getting my hands dirty with all the new stuff you concocted for us.
Off topic: since the autocomplete appears to rely on HSCAN, you might want to mention that it's only compatible with Redis >= 2.8.0 (I couldn't set it up on 2.4.14, which is Debian 7 stable version).
Cheers
What principial difference between walrus and https://github.com/lethain/Redis-Python-Datastructures ?
Is it feasible to provide a bust() method to a property used by the cached_property decorator?
I'm trying to update a boolean, but when I pull the data again it returns the old information and default.
# Presence REDIS DB (no-SQL)
class Presence(walrus.Model):
__database__ = redis_db
channel_name = walrus.TextField(primary_key=True)
user = walrus.UUIDField(index=True, default=uuid.uuid4())
is_accessible = walrus.BooleanField(index=True, default=False)
>>> p = Presence.get(Presence.user == '4fd22924-3a45-479b-90f0-ee49bb89caad')
>>> p.is_accessible
False
>>> p.is_accessible = True
>>> p.is_accessible
True
>>> p.save()
>>> p = Presence.get(Presence.user == '4fd22924-3a45-479b-90f0-ee49bb89caad')
>>> **p.is_accessible**
False
PS: The problem is actually in the Boolean object type. I solved by changing the type to Integer.
is_accessible = walrus.IntegerField(index=True, default=0)
Thanks for the fantastic work! Quick question about autocomplete: the docs say: "results are scored by the proximity of the match to the front of the string, then alphabetically." Trying the following, however:
database = Database()
ac = database.autocomplete()
ac.store("aa bb cc")
ac.store("tt cc")
ac.search("cc")
gives:
[u'aa bb cc', u'tt cc']
I was wondering if this was intentional--the behavior described in the docs seems preferable.
model.py
uses basestring
and unicode
, which are gone in Python 3.
Also, there are a few issues stemming form the unicode-ness of strings in Python 3:
model.TextField.db_value()
converts to encoded strings, which would be bytes
in Python 3. model.Query.get_primary_hash_key
uses db_value
to convert the primary key value into the Redis key suffix. So we end up with keys like user.id.b'joedoe'
.model.Model.load()
iterates over the field names, which are (unicode) strings and looks them up in raw_data
, which is a dict with bytes
keys, since redis.hgetall()
returns all hash field names as bytes
objects. Consequently all field values of the Model instance will be None
.When the model is loads from redis, the model field will be 'None' str instead of the python None type. That will cause some trouble when using the value by expecting the Nonetype value
I can't find a way to prevent the None value from saving into redis.
I need to return all inactive users, searching for more than 60 seconds without updating the information. But the return is always empty.
I've tried to put a simpler example by looking for the smaller (or larger) field than the current date. Both queries (greater equal or less equal) return empty.
>>> for user in User.query(User.last_seen <= datetime.datetime.now()):
... print(user.last_seen)
... (return NULL)
Listing all the records to ensure there is information:
>>> for user in User.all():
... print(user.last_seen)
...
2017-05-04 21:58:12.117880
2017-05-04 21:58:12.372056
2017-05-04 18:59:11.817694
2017-0....+50
Using Walrus' Models feature, I'm getting a Segmentation fault
when I set order_by
on a query and the query returns no results. Example:
>>> q = User.query(User.username=='nobody', order_by=User.first_name)
>>> for user in q: do_something()
Segmentation fault (core dumped)
...and then the whole Python REPL exits!
This only happens if the query does not return any matches. So if there is a user with username nobody
, then it returns the one result, as expected.
>>> u = User(username='nobody')
>>> u.save()
>>> q = User.query(User.username=='nobody', order_by=User.first_name)
>>> for user in q: print user.username # prints 'nobody' and returns to prompt
If I run a query without order_by
and there are no results, that works fine, too.
>>> q = User.query(User.username=='nobody', order_by=User.first_name)
>>> for user in q: print user.username # executes and returns to prompt, as expected
Note that I'm using RLite, not Redis, as the backend. I've pasted a complete example below:
#! /usr/bin/python2
import walrus
from walrus.tusks.rlite import WalrusLite
db = WalrusLite(':memory:')
class User(walrus.Model):
database = db
username = walrus.TextField(primary_key=True)
password = walrus.TextField()
is_active = walrus.BooleanField(default=True)
first_name = walrus.TextField(index=True)
last_name = walrus.TextField(index=True)
email = walrus.TextField()
addresses = walrus.HashField()
q = User.query(User.username=='nobody', order_by=User.first_name)
# Crashes somewhere here:
for user in q:
print user.username
Any ideas? I don't have Redis, so maybe somebody who does could check that it's not an RLite-specific problem.
I have a model:
class WithListModel(Model):
name = TextField(index=True)
values = ListField()
When I do
model.delete()
the keys associated with values
, i.e. keys of form withlistmodel:container.values.withlistmodel:id.1
are not deleted.
If we create an instance of a Model that doesn't specify a field (and that field is not part of a secondary index), should the behavior be for that attribute to return None on access?
Getting KeyError in this case is a bit unexpected.
>>> from walrus import *
>>> db = Database()
>>> class Foo(Model):
... name = TextField(primary_key=True)
... keyerror = TextField()
...
test>>> test = Foo(name='test')
>>> test.name
'test'
>>> test.keyerror
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "...\lib\site-packages\walrus\models.py", line 123, in __get__
return instance._data[self.name]
KeyError: 'keyerror'
It seems that the fix from #25 broke the save()
method.
Here's what I do:
class WithListModel(Model):
name = TextField(index=True)
values = ListField()
c = WithListModel.create(name='x')
c.values.append('x')
list(c.values) # ['x']
c.save()
list(c.values) # []
This is because of this part: https://github.com/coleifer/walrus/blob/master/walrus/models.py#L931 -- deleting now also deletes associated list...
I have a class
class Post(BaseModel):
title = TextField(index=True)
slug = TextField()
body = TextField()
created = DateTimeField()
edited = DateTimeField()
tags = SetField()
draft = IntegerField()
When I run
>>> [i.draft for i in Post.all()]
[0, 0, 1]
The results are as expected
However
>>> list(Post.query(Post.draft==0))
[]
There seems to be a bug somewhere, or am I missing someting
Hi,
I love this project, thanks!
Hash
is dict
-like, but it could be more dict
-like.
Is there a design decision preventing the implementation of a dict.get
analogue for Hash
? The implementation could be as simple as
def get(self, key, fallback=None):
return self.as_dict().get(key, fallback)
Hello coleifer thank you for good library.
I am trying to use it but I failed to import walrus.graph.
When digging I also found that Lua scripts are not installed during pip or setup.py install.
'pip2 install redis-py
pip2 install redis
pip2 install --upgrade walrus
pip2 install -U --force walrus
python2.7 setup.py build
python2.7 setup.py install
python2.7 setup.py test
'
`test_sort (walrus.tests.TestZSet) ... ok
Traceback (most recent call last):
File "/Users/alexandermikhalev/py_code_local/walrus/walrus/tests.py", line 1057, in test_basic_apis
self.assertEquivalent(self.arr.pop(1), 'i2')
File "/Users/alexandermikhalev/py_code_local/walrus/walrus/containers.py", line 803, in pop
args=[idx])
File "/Users/alexandermikhalev/py_code_local/walrus/walrus/database.py", line 131, in run_script
return self._scripts[script_name](keys, args)
File "/usr/local/lib/python2.7/site-packages/redis/client.py", line 2699, in call
return client.evalsha(self.sha, len(keys), _args)
File "/usr/local/lib/python2.7/site-packages/redis/client.py", line 1944, in evalsha
return self.execute_command('EVALSHA', sha, numkeys, *keys_and_args)
File "/usr/local/lib/python2.7/site-packages/redis/client.py", line 573, in execute_command
return self.parse_response(connection, command_name, *_options)
File "/usr/local/lib/python2.7/site-packages/redis/client.py", line 585, in parse_response
response = connection.read_response()
File "/usr/local/lib/python2.7/site-packages/redis/connection.py", line 582, in read_response
raise response
ResponseError: Error running script (call to f_59fe925e1dd307a32fa4168556f6891e67bfad0f): @user_script:12: @user_script: 12: Lua redis() command arguments must be strings or integers
`
How can I prevent an error when I try to query an object that is being saved at the same time? I noticed that at the moment the object is being saved it is not visible to the GET method, then there is error, since I try to get the object and it is not found for a few seconds.
Hello sir, thanks for the walrus. Truely helpful.
I'm a staruper. and I get a question. Please help.
here it is.
in runtime.py i make things like
runtimedb = Database(host='localhost', port=6379, db=0)
runtimedb['walrus'] = 'tusk'
print runtimedb
print runtimedb['walrus']
h = runtimedb.Hash('flowinfo')
h.update(name='testinfo', testtime='0')
and in another app.py I need get that database.(for test, just print h['testtime'])
how do I make it ?
I tried import everthing, not work.
I guss, maybe I should connect to the database.
how do I make it ?
Ps: I run this script in tornado with django.
Hello,
I'm using your module for my apps and it's awesome. Unfortunately I can't install latest version :(
Error when installing from source:
Traceback (most recent call last):
File "setup.py", line 17, in <module>
version=__import__('walrus').__version__,
File "/root/walrus/walrus/__init__.py", line 34, in <module>
from walrus.database import Database
File "/root/walrus/walrus/database.py", line 42, in <module>
class Database(Redis):
TypeError: Error when calling the metaclass bases
cannot create 'NoneType' instances
Error when installing from pip:
Collecting walrus==0.2.0 (from -r common/requirements.txt (line 42))
Downloading walrus-0.2.0.tar.gz (42kB)
100% |################################| 45kB 502kB/s
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/tmp/pip-build-Q3v_pi/walrus/setup.py", line 17, in <module>
version=__import__('walrus').__version__,
File "walrus/__init__.py", line 34, in <module>
from walrus.database import Database
File "walrus/database.py", line 42, in <module>
class Database(Redis):
TypeError: Error when calling the metaclass bases
cannot create 'NoneType' instances
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/tmp/pip-build-Q3v_pi/walrus/setup.py", line 17, in <module>
version=__import__('walrus').__version__,
File "walrus/__init__.py", line 34, in <module>
from walrus.database import Database
File "walrus/database.py", line 42, in <module>
class Database(Redis):
TypeError: Error when calling the metaclass bases
cannot create 'NoneType' instances
Tested in:
Mac OS X 10.10.2
Debian Testing
Ubuntu 14.04.1
EDIT:
After installing hiredis everything works like a charm. ๐
There is some moment that it is not possible to return an object and everything spoils. When I try to pull a list with all objects nothing else works. I could not detect exactly when this occurs, sometimes it looks like it was after using delete(). But I can say that it is unstable.
# Presence REDIS DB (no-SQL)
class Presence(walrus.Model):
__database__ = redis_db
channel_name = walrus.TextField(primary_key=True)
user = walrus.UUIDField(index=True, default='00000000-0000-0000-0000-000000000000')
last_seen = walrus.DateTimeField(default=datetime.datetime.now, index=True)
is_accessible = walrus.IntegerField(index=True, default=0)
connection_state = walrus.IntegerField(default=1, index=True)
>>> for t in Presence.all():
print(t.user, t.is_accessible, t.connection_state)
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 789, in all
yield cls.load(result, convert_key=False)
File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 889, in load
raise KeyError('Object not found.')
KeyError: 'Object not found.'
See that in the object list I have an object that is marked as indexable, when I do a query looking specifically for that object it also bursts the same error.
>>> for presence in Presence.all():
print("pint user ID: {}".format(presence.user))
pint user ID: 46981222-d147-48de-aba2-e63133d2491d
>>> for t in Presence.query(Presence.user == '46981222-d147-48de-aba2-e63133d2491d'):
print("pint user ID: {}".format(presence.user))
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 839, in query
yield cls.load(hash_id, convert_key=False)
File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 889, in load
raise KeyError('Object not found.')
KeyError: 'Object not found.'
or: uuid.UUID('46981222-d147-48de-aba2-e63133d2491d')):
After the error starts happening, not even the method to delete all records of that type of object does:
>>> Presence.query_delete()
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Volumes/MacHD/Users/Git/webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 857, in query_delete
cls.load(hash_id, convert_key=False).delete()
File "/Volumes/MacHD/Users/Git/-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 889, in load
raise KeyError('Object not found.')
KeyError: 'Object not found.'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.