Revisit deprecation of GCS storage backend about bb-storage

[EdSchouten]

I think 'raw performance' was never the reason for us to deprecate the GCS/S3 storage backend. It was always the inability to do batch existence queries. This is something that needs to be fast for Buildbarn to work efficiently.

Please see #128 for more discussion in this area.

[Jonpez2]

Thank you, that's a super helpful link, sorry for blindly reopening the conversation.

[Jonpez2]

So I assume that a parallel call - one per object - over the gcp storage grpc list api was considered and discarded? I see you mention 'milliseconds' as the latency budget; how many objects are generally checked for in a single batch?

Thank you!

[EdSchouten]

It may be as many as tens of thousands of objects. It's thus not realistic to spawn one goroutine per object.

[Jonpez2]

Ack thanks. So then maybe I could do something with a cloud native database behind the api. I’m pretty allergic to running my own stateful services - that’s what I pay cloud providers for! Thanks for engaging, and thanks for making such a useful bit of kit.

…

On Wed, 19 Oct 2022 at 18:27, Ed Schouten ***@***.***> wrote: It may be as much as tens of thousands. It's thus not realistic to spawn one goroutine per object. — Reply to this email directly, view it on GitHub <#147 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABN425JHHXPV3VV2UC4DOHLWEAVRLANCNFSM6AAAAAARJDHBLI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Jonpez2]

So I've started messing around with the codebase, and the first thing I needed to do was to add a dependency on the Spanner API, after which I ran go mod tidy on the root of the repo. This errored out with:

go: finding module for package github.com/buildbarn/bb-storage/internal/mock
github.com/buildbarn/bb-storage/pkg/auth tested by
github.com/buildbarn/bb-storage/pkg/auth.test imports
github.com/buildbarn/bb-storage/internal/mock: no matching versions for query "latest"

Is that expected or am I doing it wrong?

Thanks!

[Jonpez2]

The reason I think I have to run go mod tidy is because a call to go get cloud.google.com/go/spanner seems to give me a version skew in envoy between protoc-gen-validate and envoyproxy-go-control-plane. I'm wondering whether the right approach here would actually be to make a grpc api skin out of the BlobAccess API, and then I can implement the spanner backend without touching the BB codebase...?

[moroten]

Does go mod tidy -e work?

[Jonpez2]

Seems to give exactly the same error. I think the problem is that that directory is quite weird in go space, and only defined in bazel space (right?)

[moroten]

Yes, the problem is the generated files not available in the repository. Does anyone know a good way to deal with it?

For me, go get -u ./... && go mod tidy -e has worked. That avoids the test packages and does not feel ideal.

[EdSchouten]

I don't think I use go mod tidy at all. I generally just:

• Remove go.sum
• Remove all the dependencies in go.mod
• Run go get -d ./...

And live with that.

[EdSchouten]

Closing this, as I'm still not convinced this is feasible. GCS may well be faster, but because it still lacks facilities for doing bulk existence checks, it's still not suitable. If I'm wrong, I'd like to be persuaded otherwise in the form of a PR containing benchmark results.