odashi / cloud-storage-mocker Goto Github PK
View Code? Open in Web Editor NEWMocker library of Google Cloud Storage with local filesystem mounting.
License: MIT License
Mocker library of Google Cloud Storage with local filesystem mounting.
License: MIT License
In the current version, when creating a blob using Blob.from_string()
and specifying the gs://bucket/path
URL as well as the client, the following error is thrown once we want to retrieve the object (e.g. using blob.download_to_file()
):
Traceback (most recent call last):
File "/workspace/project/foo3.py", line 28, in <module>
blob.download_to_file(stream)
File "/workspace/venv/lib/python3.12/site-packages/google/cloud/storage/blob.py", line 1160, in download_to_file
self._prep_and_do_download(
File "/workspace/venv/lib/python3.12/site-packages/google/cloud/storage/blob.py", line 4338, in _prep_and_do_download
download_url = self._get_download_url(
^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/venv/lib/python3.12/site-packages/google/cloud/storage/blob.py", line 862, in _get_download_url
hostname = _get_host_name(client._connection)
^^^^^^^^^^^^^^^^^^
File "/usr/local/python/current/lib/python3.12/unittest/mock.py", line 678, in __getattr__
result = self._get_child_mock(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/python/current/lib/python3.12/unittest/mock.py", line 1055, in _get_child_mock
return klass(**kw)
^^^^^^^^^^^
TypeError: Client.__init__() got an unexpected keyword argument 'parent'
Example code:
import io
import google.cloud.storage
import google.cloud.storage.blob
from cloud_storage_mocker import Mount
from cloud_storage_mocker import patch as gcs_patch
storage_dir = "."
bucket_name = "b"
with gcs_patch([Mount(bucket_name, storage_dir, readable=True, writable=True)]):
client = google.cloud.storage.Client()
blob_name = "hello.txt"
# Write object
blob = client.bucket(bucket_name).blob(blob_name)
blob.upload_from_string("world")
# Make object using `Blob.from_string()` and `gs://` URL
blob = google.cloud.storage.blob.Blob.from_string(f"gs://{bucket_name}/{blob_name}", client=client)
# Workaround: Recreate blob using `client.bucket().blob()`
# blob = client.bucket(blob.bucket.name).blob(blob.name)
# Read object
stream = io.BytesIO()
blob.download_to_file(stream) # <-- throws exception
stream.seek(0)
There's a workaround though: after parsing the gs://
URL using Blob.from_string()
, we can create a new blob using client.bucket(bucket).blob(blob)
, which works.
The Python requirement in ./pyproject.toml
is very restrictive, preventing installation on recent versions of Python.
ERROR: Ignored the following versions that require a different python version: 0.0.1a1 Requires-Python <3.12,>=3.10; 0.0.1a2 Requires-Python <3.12,>=3.10; 0.0.1a3 Requires-Python <3.12,>=3.10; 0.0.1a4 Requires-Python <3.12,>=3.10; 0.0.1a5 Requires-Python <3.12,>=3.10; 0.1.0 Requires-Python <3.12,>=3.10; 0.2.0 Requires-Python <3.12,>=3.10; 0.2.1 Requires-Python <3.12,>=3.10; 0.3.0 Requires-Python <3.12,>=3.10; 0.3.1 Requires-Python <3.12,>=3.10
Mocked methods raise exception if passed kwargs.
I'm uploading blob with content_type like below, but this lib looks not considered.
blob.upload_from_file(
file_to_upload.stream, content_type=file_to_upload.mimetype, rewind=True
)
The args are ignored, but the kwargs do not seem to be expected.
cloud-storage-mocker/src/cloud_storage_mocker/_core.py
Lines 345 to 349 in a8d89a6
So, this resolve this issue.
def upload_from_file(
self,
file_obj: io.BufferedReader,
*args: Any, # Not supported
+ **kwargs: Any, # Not supported
) -> None:
The same would be necessary for other methods.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.