jacobsa / gcloud Goto Github PK
View Code? Open in Web Editor NEW(Unofficial) Google cloud platform stuff
License: Apache License 2.0
(Unofficial) Google cloud platform stuff
License: Apache License 2.0
bucket_tests.go currently contains a workaround for Google-internal bug 19717210 in the GCS XML API. We switched to the XML API because of Google-internal bug 19700548 in URL handling in the JSON API.
So:
In order to allow for custom user agents (cf. GoogleCloudPlatform/gcsfuse#26), we need to be able to generate our own HTTP requests. (We do control the http.RoundTripper, but that interface is not supposed to modify the request. Anyway, it's probably better to just continue the trend of cutting dependencies.)
So, we want to stop relying on other people's code for all operations. I believe these are left to do:
This is blocked by #3.
google.golang.org/cloud/storage is used for less and less with every change like e309733, motivated by bugs and missing features. It would be a weird wart to have some of its types left over if we use it for nothing in implementation anymore. Plus having our own version of storage.Object
etc. let's us document better and narrow to just the fields that we have tested.
So ensure there are no more mentions of that package in the public interface. Also take this opportunity to update the GitHub project page to be a bit less glum about how it is "just a wrapper", which is not really true anymore.
As discovered in GoogleCloudPlatform/gcsfuse#52 and as documented here, composite objects in GCS do not advertise MD5 hash fields, but this package assumes they are always there. Oops.
Plan:
This library was created before Go 1.13 where error wrapping was introduced. So, it did not wrap errors, which made error handling at the caller side a bit tricky.
For instance, when opening an unknown bucket, the HTTP 404 error was replaced by a custom "Unknown bucket" error.
https://github.com/jacobsa/gcloud/blob/master/gcs/conn.go#L191-L192
case http.StatusNotFound:
err = fmt.Errorf("Unknown bucket %q", b.Name())
This error is hard to be handled by the caller. However, if we wrap the internal http error like this:
case http.StatusNotFound:
err = fmt.Errorf("Unknown bucket %q: %w", b.Name(), err)
On the user side, such error can be casted back to an 404, and properly handled:
if errors.As(err, &apiErr) {
switch apiErr.Code {
case http.StatusNotFound:
return syscall.EINVAL
}
}
The Go JSON package maps invalid UTF-8 to the Unicode replacement rune (cf. json.Marshal). Other similar packages do the same. So it's not insane to think that somehow we may wind up with an object name containing the rune.
So, make sure the "interesting names" test case in the integration test covers this character. It may or may not already be part of the control character generation code.
Allow the user to set a custom User-Agent header in calls to GCS (as desired in e.g. GoogleCloudPlatform/gcsfuse#26).
Do this by adding a ConnConfig
struct with fields for name and project ID, and making NewConn
take that. Then add a user agent field.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.