whysobad / abwart Goto Github PK
View Code? Open in Web Editor NEWHousekeeper for your docker registry which adds support for retention policies
License: GNU General Public License v3.0
Housekeeper for your docker registry which adds support for retention policies
License: GNU General Public License v3.0
Currently, abwart uses the garbage collector of the registry binary which is shipped with the registry
image to clean up unused blobs.
It turns out the --delete-untagged
flag also deletes manifests of mutli-arch images as described in distribution/distribution#3178.
This makes the garbage collector pretty much unusable since after a cleanup most images will end up with a manifest unknown
error when being pulled.
The idea is now to come up with some kind of custom "garbage collector" which deletes the unneeded layers from the registry. This implementation should be a temporary solution until the PR from the mentioned issue is merged and the bug in the garbage collector is fixed. Below are some ideas how this could be achieved:
I would implement a "garbage collector" over the rest api. This implementation is resource costly as it would fetch every repository, all tags of all repositories and for every tag all layers.
Note
This implementation wouldn't work since we're only fetching all layers which are currently used. Our goal is the opposite: fetch all layers which are currently not used. I still wanted to bring this idea up since it could may be used in another implementation
All blobs are stored in the registry at /var/lib/registry/docker/registry/v2/blobs/**
(or in directory specified in /etc/docker/registry/config.yml
). This implementation would index all blobs from this directory and filter out the layers from the manifests. Once all layers are fetched we could filter out the unneeded layers (maybe by fetching all needed layers using idea 1) and then delete them either over the api or directly in the filesystem. I don't know how reliable modifications to the filesystem of a running registry are but this would need to be tested.
In this implementation we would call the garbage collector with the dry run flag in a first step. From this output we identify all manifests which would be deleted falsly. A way to do this is by fetching all manifests for all repositories and compare them to the identified manifests. Overlapping manifests are the backed up and recreated after the garbage collector was run.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.