Comments (6)
Is this deletion by prefix or multiple objects.
For deletion by prefix we can implement a separate func, however, for deleting multiple objects we may want to consider a batching API that also supports renames, deletes, downloads etc.
from uplink.
I don't remember exactly for which functionality we added this TODO point while doing MPU API. I agree with what you wrote also I don't plan to work on this actively until we will have real use cases e.g. performance issues.
from uplink.
Given storj/storj#4256, I think delete by prefix is a reasonable feature.
from uplink.
Sounds good. I changed the title to reflect intention.
from uplink.
Such an API would help in a few use cases:
- Implement
rclone purge
. - Improve
uplink rm --recursive
. Currently, it lists the prefix and deletes the object one by one. - Any app that emulates folders with prefixes.
from uplink.
One thing to consider is that we probably want to give feedback to the caller on the progress. Otherwise, it might seem like the API call has gotten stuck.
There are two approaches that come to mind:
A. Add DeleteByPrefix
with a Limit
. The Limit (e.g. max 1000) is enforced by the satellite, and it will respond with the number of deleted objects. This has the added benefit that there's some uplink rate limiting happening for the calls.
B. Add streaming rpc DeleteByPrefix
which every N
deletes sends an update to the satellite.
from uplink.
Related Issues (20)
- Memory usage with large uploads on large bandwidth links HOT 4
- Error "too many requests" when uploading files now common HOT 8
- Create more integration tests
- Link sharing feature HOT 6
- New uplink release HOT 1
- Race when upload is canceled HOT 1
- New upload code path is dropping satellite errors HOT 4
- Implement support for enabling bucket-level versioning in uplink HOT 1
- Regression: "unaligned 64-bit atomic operation" on 32-bit systems HOT 3
- Existing download codepath acceleration HOT 4
- Create specific instrumentation to help debug edge intermittent failures HOT 2
- Undefined `net` functions HOT 5
- Flaky test storj.io/uplink/private/eestream.TestRSRandomData HOT 2
- Flaky test storj.io/storj/satellite/reputation/suspension_test.TestAuditSuspendDQDisabled HOT 2
- [QUESTION] Delete method HOT 3
- Flaky test storj.io/storj/satellite/orders/orders_test.TestProjectBandwidthDailyRollups HOT 2
- Bad upload performance / no long tail cancelation in retry situation like too many offline nodes HOT 1
- Test support for keys with trailing slash
- Release uplink and uplink-c
- Uplink stuck during download HOT 16
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from uplink.