aaugustin / django-resto Goto Github PK

File "/home/api/venv/lib/python3.5/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/api/venv/lib/python3.5/site-packages/django/db/models/query.py", line 1045, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/home/api/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1053, in execute_sql
for sql, params in self.as_sql():
File "/home/api/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1006, in as_sql
for obj in self.query.objs
File "/home/api/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1006, in
for obj in self.query.objs
File "/home/api/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1005, in
[self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
File "/home/api/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 955, in pre_save_val
return field.pre_save(obj, add=True)
File "/home/api/venv/lib/python3.5/site-packages/django/db/models/fields/files.py", line 292, in pre_save
file.save(file.name, file, save=False)
File "/home/api/venv/lib/python3.5/site-packages/django/db/models/fields/files.py", line 91, in save
self.name = self.storage.save(name, content, max_length=self.field.max_length)
File "/home/api/venv/lib/python3.5/site-packages/django/core/files/storage.py", line 53, in save
name = self.get_available_name(name, max_length=max_length)
File "/home/api/venv/lib/python3.5/site-packages/django/core/files/storage.py", line 77, in get_available_name
while self.exists(name) or (max_length and len(name) > max_length):
File "/home/api/venv/lib/python3.5/site-packages/django_resto/storage.py", line 277, in exists
return self.transport.exists(host, name)
File "/home/api/venv/lib/python3.5/site-packages/django_resto/storage.py", line 122, in exists
resp = self._http_request(HeadRequest(url))
File "/home/api/venv/lib/python3.5/site-packages/django_resto/storage.py", line 94, in _http_request
return urlopen(request, timeout=self.timeout)
File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.5/urllib/request.py", line 466, in open
response = self._open(req, data)
File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
'_open', req)
File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
result = func(*args)
File "/usr/lib/python3.5/urllib/request.py", line 1282, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "/usr/lib/python3.5/urllib/request.py", line 1256, in do_open
raise URLError(err)
urllib.error.URLError:
[02/Jan/2018 12:51:23] "POST /en/mymodel/add/ HTTP/1.1" 500 302151

Greetings.

My name is Gustavo, from colombia. first of all I wanted to say thanks because we are using django_resto for our repository and we think is an impressive and easy to use tool..

However, recently we had to include some new features to django admin interface, letting adminstrator to upload images to specific models.. This is causing the need to change the media url and the default storage in the settings..

In your Readme document, you specify "You can also enable a backend only for selected fields in your models.", but I've been trying to do it without success.

I put in my model this parameter:
from django_resto.storage import DistributedStorage

models.FileField(..., storage=DistributedStorage...)

We don't know what are we missing.

Thanks in advance.

Note: error displayed when we try to upload a new instance of the model...

unbound method save() must be called with DistributedStorage instance as first argument (got unicode instance instead)

In testing django-resto's HybridStorage option where a worker server writes to the media server, it's working as expected both saving the local copy on worker and writing remote copy on media.

However, when trying to load/read in the worker, it has to be explicit whether to look at path (local) or url (on media server).

Is it possible to make it do the same process in reverse for all instances (i.e. when accessing myobj.file.read() which is a FileStroage field that uses the HybridStorage approach, it will check locally first, if not there, will fetch from the media server)?

Hi Aymeric!

Could you please upload a new release to pypi?

HEAD request is sent twice to webdav server upon file upload. I believe this is due to DistributedStorage._save method:

def _save(self, name, content):
    # This is really prone to race conditions - see README.
    name = self.get_available_name(name)
    DistributedStorageMixin._save(self, name, content)
    return name

The line "name = self.get_available_name(name)" is probably not necessary, because django default Storage.save() method already calls it (https://docs.djangoproject.com/en/1.3/howto/custom-file-storage/#save-name-content ):

def save(self, name, content):
    """
    Saves new content to the file specified by name. The content should be a
    proper File object, ready to be read from the beginning.
    """
    # Get the proper name for the file, as it will actually be saved.
    if name is None:
        name = content.name

    name = self.get_available_name(name)
    name = self._save(name, content)

    # Store filenames with forward slashes, even on Windows
    return force_unicode(name.replace('\\', '/'))

Is this a bug?

I have some model with field

image = models.ImageField(_('image'), upload_to='photos')

in settings.py

DEFAULT_FILE_STORAGE = 'django_resto.storage.DistributedStorage'

Error appears then I save a model instance through django admin (and in django request-response loop, too).

<<outer django traceback stripped as irrelevant>>

File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/files.py", line 252, in pre_save
    file.save(file.name, file, save=False)
  File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/files.py", line 86, in save
    self.name = self.storage.save(name, content)
  File "/usr/local/lib/python2.7/dist-packages/django/core/files/storage.py", line 49, in save
    name = self._save(name, content)
  File "/usr/local/lib/python2.7/dist-packages/django_resto/storage.py", line 263, in _save
    self.execute(self.transport.create, name, content.read())
  File "/usr/local/lib/python2.7/dist-packages/django_resto/storage.py", line 228, in execute
    raise exceptions.popitem()[1][1]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)

I've unpack my debugger and got this: in file /usr/lib/python2.7/httplib.py, line 814

def _send_output(self, message_body=None):
    """Send the currently buffered request and clear the buffer.

    Appends an extra \\r\\n to the buffer.
    A message_body may be specified, to be appended to the request.
    """
    self._buffer.extend(("", ""))
    msg = "\r\n".join(self._buffer)
    del self._buffer[:]
    # If msg and message_body are sent in a single send() call,
    # it will avoid performance problems caused by the interaction
    # between delayed ack and the Nagle algorithm.
    if isinstance(message_body, str):
        msg += message_body
        message_body = None
    self.send(msg)
    if message_body is not None:
        #message_body was not a string (i.e. it is a file) and
        #we must run the risk of Nagle
        self.send(message_body)

so this string raised an error: msg += message_body
because msg is type unicode, and message_body is type basestring with bytes (image file)

any ideas?

I delete a file in Admin Panel, but in the media server doesn't do :(