eldarion / django-gapc-storage Goto Github PK

By default, GCS caches any publicly accessible objects for 3600 seconds.

Especially if a developer is trying to update an existing object (see #4), it may not be desirable to have objects with that long of a cache value.

Either add an option to GoogleCloudStorage to specify cache headers or an additional backend that supports custom cache headers.

GCS_PUBLIC_READ_CACHE_DEFAULT = "public, max-age=3600"

...

class CustomCacheMediaStorage(GoogleCloudStorage):
    """
    Custom subclass of GoogleCloudStorage that allows customization of GCS's
    cache-control header
    """
    cache_control = GCS_PUBLIC_READ_CACHE_DEFAULT

    def _save(self, name, content):
        mimetype, _ = mimetypes.guess_type(os.path.basename(name))
        if mimetype is None:
            mimetype = "application/octet-stream"
        media = MediaIoBaseUpload(content, mimetype)
        req = self.client.objects().insert(
            bucket=self.bucket,
            name=self._prefixed_name(name),
            body={
                "cacheControl": self.cache_control
            },
            media_body=media
        )
        req.execute()
        return name

...

class NoCacheMediaStorage(CustomCacheMediaStorage):
    # never cache
    cache_control = "private, max-age=0"

Configure per backend, set a retry for the following operations:

• execute
• media.next_chunk

https://docs.djangoproject.com/en/1.10/howto/custom-file-storage/#django.core.files.storage._open

Called by Storage.open(), this is the actual mechanism the storage class uses to open the file. This must return a File object, though in most cases, you’ll want to return some subclass here that implements logic specific to the backend storage system.

In particular, File.open should support opening or reopening the underlying file:

https://docs.djangoproject.com/en/1.10/ref/files/file/#django.core.files.File.open

Am beginning some work in reopen-files to handle file reopening.

Add an option or additional storage backend that allows overwrites.

This is especially useful if the storage backend is being used to update existing objects (similar to what you might do on a local file system opening and writing to an existing file).

Roughly:

    def get_available_name(self, name):
        """
        Always assume a name is available.
        """
        return name

In trying to use this storage backend with a file upload module that contains a browse view, I found that the GoogleCloudStorage class doesn't have an override for the needed listdir method.

Would this be possible to add? Thanks!

https://github.com/GoogleCloudPlatform/google-cloud-python/tree/master/storage/google/cloud/storage

See https://cloud.google.com/storage/docs/access-control/signed-urls for details.

Prior examples to draw from:

• https://github.com/GoogleCloudPlatform/storage-signedurls-python
• googleapis/google-cloud-python#56

see #1

While the project has always required Django>=1.8, most of the backend has been compatible with earlier versions of Django.

Changes introduced in 148e910 (released in 0.3.0) added the max_length kwarg, which does not exist in earlier versions of Django.

To continue using the backend on earlier versions of Django, override the get_available_name method to remove the max_length kwarg:

def get_available_name(self, name):
    if self.allow_overwrite:
        return name
    return super(GoogleCloudStorage, self).get_available_name(name)

need to update six import location in order to make it compatible with django 3.

replace from django.utils.six.moves.urllib import parse as urlparse with from six.moves.urllib import parse as urlparse at storage file

line from django 3 release page:

django.utils.six - Remove usage of this vendored library or switch to six.