GithubHelp home page GithubHelp logo

dataaxiom / ghcr-cleanup-action Goto Github PK

View Code? Open in Web Editor NEW
12.0 2.0 5.0 1.85 MB

GitHub Container Registry Cleanup Action

License: BSD 3-Clause "New" or "Revised" License

Shell 0.10% TypeScript 2.84% JavaScript 97.06%

ghcr-cleanup-action's Introduction

GitHub Container Registry Image Cleanup Action

GitHub Super-Linter CI Check dist/ CodeQL Coverage

A workflow action that cleans up images in the GitHub Container Registry (ghcr.io)

It includes the following features:

  • Automatic GitHub user/organization repository support
  • Removing by tags, including untagging multi tagged images
  • Multi architecture image support
  • Referrers/Attestation support (OCIv1 tag approach)
  • Keeping a number of untagged images
  • Keeping a number of tagged images
  • Supports wildcard syntax for tag/exclude tag options
  • Multi architecture & Referrers image validation mode

Setup

Do a dry-run

Test the cleanup action first by setting the "dry-run: true" option on the action and then reviewing the workflow log. This mode will simulate the cleanup action but will not delete any images/packages. This is especially important when using a wildcard syntax to select images.

Token Permissions

To allow the injected GITHUB_TOKEN to have access to delete the images/packages ensure it's permissions have been setup correctly, either by:

  1. In project Settings > Actions > General set the Workflow permissions option to "Read and write permissions"

  2. Set the permissions directly in the workflow by setting the packages value to write.

    jobs:
      delete-untagged-images:
        name: Delete Untagged Images
        runs-on: ubuntu-latest
        permissions:
          packages: write

Action Options

Option Required Defaults Description
token yes Token used to connect with ghcr.io and the package API
tags no Comma separated list of tags to delete (supports wildcard syntax)
exclude-tags no Comma separated list of tags to exclude (supports wildcard syntax)
keep-n-untagged no Number of untagged images to keep, sorted by date
keep-n-tagged no Number of tagged images to keep, sorted by date
dry-run no false Simulate cleanup action, does not make changes (true/false)
validate no false Validate all multi architecture images in the registry after cleanup
owner no project owner The repository owner, can be organization or user type
repository no repository name The repository name
package no repository name The package name

If the tags, keep-n-untagged or keep-n-tagged options are not set then all untagged images will be deleted.

The keep-n-untagged and keep-n-tagged options can not be set at the same time.

Main Execution Modes

Delete all untagged images

To cleanup all untagged images in a image repository only the token is required to be set. It will use the current repository information to setup the owner and package name.

jobs:
  - name: ghcr.io cleanup action
    runs-on: ubuntu-latest
    steps:
      - uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}

Delete specific tagged images

Set the tags option to delete specific tags

jobs:
  - name: ghcr.io cleanup action
    runs-on: ubuntu-latest
    steps:
      - uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          tags: mytag,mytag2
          token: ${{ secrets.GITHUB_TOKEN }}

If the tag links to an image with multiple tags, the action will unlink the tag before is deleted, effetively untagging the image, but the underlying image will not be deleted unless all tags are deleted.

Keep 'n' untagged images cleanup (keep-n-untagged)

Keeps all tagged images and removes all untagged images except for the number of "keep-n-untagged" images (sorted by date). It supports multi-architecture images so the number of untagged images showing after running the action may be higher then the keep-n-untagged value set. May be combined with tags option to delete those tags too.

jobs:
  - name: ghcr cleanup action
    runs-on: ubuntu-latest
    steps:
      - uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          keep-n-untagged: 3
          token: ${{ secrets.GITHUB_TOKEN }}

Keep 'n' tagged images cleanup (keep-n-tagged)

Keeps a number (keep-n-tagged) of tagged images and then deletes the rest. Tags are sorted by date. Additional exclude-tags values are not include the total count.

jobs:
  - name: ghcr cleanup action
    runs-on: ubuntu-latest
    steps:
      - uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          keep-n-tagged: 3
          token: ${{ secrets.GITHUB_TOKEN }}

Extra Samples

Delete image when pull request is closed

name: Cleanup Pull Request Images
on:
  pull_request:
    types: [closed]
jobs:
  ghcr-cleanup-image:
    name: ghcr cleanup action
    runs-on: ubuntu-latest
    steps:
      - name: Delete image
        uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          tags: pr-${{github.event.pull_request.number}}
          token: ${{ secrets.GITHUB_TOKEN }}

Daily image cleanup

name: Daily Image Cleanup
on:
  # every day at 01:30am
  schedule:
    - cron: '30 1 * * *'
  # or manually
  workflow_dispatch:
jobs:
  ghcr-cleanup-image:
    name: ghcr cleanup action
    runs-on: ubuntu-latest
    steps:
      - uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}

Override default owner/repository/package

The default settings will use the current project to determine the owner, repository and package name but for cross project and multiple package support these can be overriden by setting owner, repository and package values.

jobs:
  - name: ghcr cleanup action
    runs-on: ubuntu-latest
    steps:
      - uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          tags: mytag,mytag2
          owner: dataaxiom
          repository: tiecd
          package: tiecd
          token: ${{ secrets.GITHUB_TOKEN }}

Tag Wildcard

The tags and exclude-tags options can use a wildcard syntax, using the ?, * and ** characters. (Utilizes the wildcard-match library)

jobs:
  - name: ghcr cleanup action
    runs-on: ubuntu-latest
    steps:
      - uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          keep-n-tagged: 3
          exclude-tags: 'v*,dev,latest'
          token: ${{ secrets.GITHUB_TOKEN }}

Keep 10 tagged images and "dev" image, and dry-run

Simulates how to keep 10 tagged images as well as the dev-image.

jobs:
  - name: ghcr cleanup action
    runs-on: ubuntu-latest
    steps:
      - name: 'Clean up docker images'
        uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          keep-n-tagged: 10
          exclude-tags: dev
          dry-run: true
          token: ${{ secrets.GITHUB_TOKEN }}

Notes

Package Restoration

GitHub has a package restoration API capability. The package IDs are printed in the workflow log where the ghcr-cleanup-action is run.

Restore Organization Package

Restore User Package

Ghost Images

Multi architecture images which have no underlying platform packages are automatically removed for the keep-n-untagged and keep-n-tagged modes and not included in their count. Partially corrupt images are not removed by default, use the validate option to be able to identify and then fix them.

Validate Option

Set the validate option to true to enable a full scan of the image repository at the end of the execution to check that all multi architecture images have no missing platform images. Warnings will be outputted if there are missing packages.

ghcr-cleanup-action's People

Contributors

rohanmars avatar dependabot[bot] avatar manimatter avatar nosfistis avatar erates avatar

Stargazers

Felix Moessbauer avatar  avatar Arne Jørgensen avatar Matt Foxx avatar Gabe Cook avatar  avatar Yumin Zhou avatar Phil Stevenson avatar Guillermo Caracuel avatar  avatar Elegant avatar Xuan (Sean) Hu avatar

Watchers

 avatar  avatar

ghcr-cleanup-action's Issues

operating on attestation bundles not supported

The github provenance attestations (e.g. from the attest-build-provenance action) are uploaded to the container registry as well (following the Cosign bundle specification). The attestation manifests are tagged as sha256-<digest>, whereby the digest refers to the container the attestation belongs to.

These attestation manifests have no platform set, which makes the cleanup action fail.

 in dry run mode - no packages will be deleted
deleting images by tags sha256-*
 deleting package id: 220688938 digest:sha256:060981ae539b31b773484a3b4030c9805b2d82899a5395517d884475372727ca tag:sha256-84d0ca4f9a5975c7fe229bb08c2340be1a4f8a1b7fcd65b0a43f3ea93589942f
Error: Cannot read properties of undefined (reading 'architecture')

Inspect this tag:

regctl manifest get --format raw-body ghcr.io/siemens/kas/kas:sha256-84d0ca4f9a5975c7fe229bb08c2340be1a4f8a1b7fcd65b0a43f3ea93589942f | jq
{
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:7520ac63a6f1545cddcf33bb243980e15cc1e0ec3a6ff935f99f0aac90cda3e9",
      "size": 812,
      "artifactType": "application/vnd.dev.sigstore.bundle.v0.3+json",
      "annotations": {
        "org.opencontainers.image.created": "2024-05-24T14:58:36.963Z",
        "dev.sigstore.bundle.content": "dsse-envelope",
        "dev.sigstore.bundle.predicateType": "https://slsa.dev/provenance/v1"
      }
    }
  ]
}

[Bugfix] Input fields "repository" and "package" are ignored

The readme describes that there are 2 input fields available repository and package.
However, when testing this in our workflows, the job gave me an error:

Warning: Unexpected input(s) 'repository', 'package', valid inputs are ['token', 'owner', 'name', 'keep-n-untagged', 'keep-n-tagged', 'tags', 'exclude-tags', 'validate', 'dry-run']

Looking through the code a bit, I noticed that the fields are available in the configuration file and it is used. But it's not requested from the action.yml.

Can this please be added?

Not an issue - just a question: On your new testing suite, how do you generate the images? What's the role of busybox?

Hi,

I am trying to understand your newly added test suite. Where I am stuck is how you create the images that you need to perform your tests on.

I understand that in this code, you copy your images and push them to git (before you run your workflow over it and test the outcome):

let command = `skopeo copy docker://${srcImage} docker://${destImage} --dest-creds=token:${token}`

For the test "multi-untagged", my understanding is that this code translates into this command:

skopeo copy docker://busybox@sha256:97d85ff9630b634ddff3e3ff69fd02bc3b69de8dba0c5002eb0ad6915d1bf4c0 docker://ghcr.io/${owner}/${packageName}:uclibc --dest-creds=token:${token} --all

My reading: skopeo takes the image from busybox and pushes it to git. What I don't get is where that "busybox" repository is located, and how the image got in there in the first place.

Probably a very simple question, would much appreciate your pointer.

It's great work you are putting into this, and as written in the other thread - I am learning a ton ;)

Thanks again.

Error: couldn't find package id undefined in repository, skipping

Trying to implement this into my workflow, but it can't seem to locate the package.

Package: https://github.com/mill1000/midea-msmart/pkgs/container/msmart-ng

Full log output

in dry run mode - no packages will be deleted
deleting all untagged images
Error: couldn't find package id undefined in repository, skipping

Relevant chunk of workflow

  prune-old-images:
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest
    permissions:
      packages: write
    steps:
      - uses: dataaxiom/ghcr-cleanup-action@v1
        with:
          package: msmart-ng
          token: ${{ secrets.GITHUB_TOKEN }}
          dry-run: true
          validate: true

How to operate on packages with "/" in the name

In the kas project, we have two packages (containers), named kas/kas and kas/kas-isar [1]. When using the GitHub API, I can access these packages by HTML escaping the slash as %2f. However, in the ghcr-cleanup-action, I found no way to operate on these packages.

I tried the following "names", but all resulted in an error Package not found. - https://docs.github.com/rest/packages/packages#list-package-versions-for-a-package-owned-by-a-user.

  • kas
  • kas/kas
  • kas%2fkas

Example job:

cleanup_ghcr_containers:
  name: cleanup untagged containers
  runs-on: ubuntu-latest
  permissions:
    packages: write
  strategy:
    matrix:
      image-name: ["kas", "kas-isar"]
  steps:
    - uses: dataaxiom/[email protected]
      with:
        validate: true
        dry-run: true
        name: "kas%2f${{ matrix.image-name }}"
        token: ${{ secrets.GITHUB_TOKEN }}

Successful curl query against the API:

curl -L \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <redacted>" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/users/fmoessbauer/packages/container/kas%2fkas/versions

[1] https://github.com/siemens/kas/pkgs/container/kas%2Fkas

Fails as it tries to delete the same package twice

Hi,

we tried to apply this action to the siemens/kas repository, but there it fails as it tries to delete the same package twice (in the same run):

[...]
 deleting package id: 183706397 digest: sha256:44871a3c4628a192347a09e39e7e6f5bbdb7e68f95fa8ab3914be3d2301aca7d architecture: amd64
[...]
 deleting package id: 183706397 digest: sha256:44871a3c4628a192347a09e39e7e6f5bbdb7e68f95fa8ab3914be3d2301aca7d architecture: amd64
Error: Package version not found. - https://docs.github.com/rest/packages/packages#delete-package-version-for-an-organization

Full CI output: https://github.com/siemens/kas/actions/runs/10011297384/job/27675632510

[Feature request] Keep only n tagged images (and ignore specific tag)

Hi,

Fantastic work, thank you so much for solving this headache.
I have a feature request to keep only n-tagged images, and to always keep a specific tag as well.

Background:
I have a setup where I have two types of containers:

  1. One "dev" container
  2. Ongoing versioned containers (ie. latest)

Whenever I push to "dev" branch, it will create a new container tagged "dev", and untag the old "dev" container; this helps me to test the image before releasing it.

When I then merge to the "main" branch, a new versioned (e.g. 4.2.1) container is created.

Proposal
What I would love to do is the following:

  • When I run your github action, it should keep maximum n (say, 10) versioned images (e.g. 4.1.0, 4.1.1, 4.1.2, etc)
  • In addition, it should always keep "dev" (never remove it)
  • All other packages (all untagged, any older than 10 versions, and not being "dev") should be removed

Error: deletes layers still needed

Hi!

First of all, thanks for this great action. It does so much more what one needs than the corresponding GitHub action.

However, if nothing has changes in a rebuilt container image, then it reuses all of the previous layers. In this case the action does remove layers that are still needed by remaining images.

How to Reproduce

Functioning Part

The container image will be built.

  • Run the workflow two more times.

You will have three container instances which all run well:

# set the owner of the fork
OWNER=XXXXX

# clean local images
docker rmi --force $(docker images -q ghcr.io/"$OWNER"/debug-ghcr-cleanup-action)
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.

Usage:  docker rmi [OPTIONS] IMAGE [IMAGE...]

Remove one or more images
docker pull ghcr.io/"$OWNER"/debug-ghcr-cleanup-action:run-1
run-1: Pulling from XXXXX/debug-ghcr-cleanup-action
a258b2a6b59a: Already exists 
6499a07b7301: Already exists 
ef51b21b47ac: Already exists 
5155adf0195a: Already exists 
5397266c869c: Pull complete 
fadf02fa3d9f: Pull complete 
b2dae9647df9: Pull complete 
92f4c20d5423: Pull complete 
09732b23169c: Pull complete 
1d1cc2f9f75f: Pull complete 
Digest: sha256:82ceb35e1b3f5783fb5e42a1bb519e7cd30a2507bb0f2701dfe5eaa841a52d00
Status: Downloaded newer image for ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-1
ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-1
docker pull ghcr.io/"$OWNER"/debug-ghcr-cleanup-action:run-2
run-2: Pulling from XXXXX/debug-ghcr-cleanup-action
Digest: sha256:a2fd4991974967b8363f7e5b7f47682fce8c76a252cc0e062106b6544601812d
Status: Downloaded newer image for ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-2
ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-2
docker pull ghcr.io/"$OWNER"/debug-ghcr-cleanup-action:run-3
run-3: Pulling from XXXXX/debug-ghcr-cleanup-action
Digest: sha256:5ad4d7e8f7857dd931267f57b4d03576af4a718986d8c3352fafe67264ff98d0
Status: Downloaded newer image for ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-3
ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-3

Malfunction

  • Run the workflow one final time.
  • The first image will be removed, discarding required layers.
  • You will receive three non-functional container images:
# clean local images
docker rmi --force $(docker images -q ghcr.io/"$OWNER"/debug-ghcr-cleanup-action)
Untagged: ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-1
Untagged: ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-2
Untagged: ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-3
Untagged: ghcr.io/XXXXX/debug-ghcr-cleanup-action:run-4
Untagged: ghcr.io/XXXXX/debug-ghcr-cleanup-action@sha256:5ad4d7e8f7857dd931267f57b4d03576af4a718986d8c3352fafe67264ff98d0
Untagged: ghcr.io/XXXXX/debug-ghcr-cleanup-action@sha256:82ceb35e1b3f5783fb5e42a1bb519e7cd30a2507bb0f2701dfe5eaa841a52d00
Untagged: ghcr.io/XXXXX/debug-ghcr-cleanup-action@sha256:a2fd4991974967b8363f7e5b7f47682fce8c76a252cc0e062106b6544601812d
Untagged: ghcr.io/XXXXX/debug-ghcr-cleanup-action@sha256:e9c13fd460e48420d459c627421da730e9f03c745e67c36a61a1a71ccb44e659
Deleted: sha256:b26081def499a7305ce74810cc7713e3a08b75d95f9c807f45d9405ad26a6961
Deleted: sha256:20f845eee9b84310de796ca767cff30e655415953f6da8e9afcd80a7d3990186
Deleted: sha256:13ab21f8ae66f9ce59d5acca985fab664252e04546bf599d7308d992ae555b97
Deleted: sha256:e4f1e450cdb22794fffaa6b744884b4ae09aa81ba0389875b4630bd172d7dd58
Deleted: sha256:32e1ca3124fb2818b9a90102ddd8adf987e9316d774f354336b2d67186874170
Deleted: sha256:f96442d073203d983b9c7d07d9c14202bc8d774cee575bbccd4d7ff56ed6800d
Deleted: sha256:8498a7fc792818c63b125f3a1ee3177812eb1b73491f7c90c5bf3782237c7bb2
Error response from daemon: No such image: b26081def499:latest
Error response from daemon: No such image: b26081def499:latest
Error response from daemon: No such image: b26081def499:latest
docker pull ghcr.io/"$OWNER"/debug-ghcr-cleanup-action:run-4
run-4: Pulling from XXXXX/debug-ghcr-cleanup-action
manifest unknown
docker pull ghcr.io/"$OWNER"/debug-ghcr-cleanup-action:run-3
run-3: Pulling from XXXXX/debug-ghcr-cleanup-action
manifest unknown
docker pull ghcr.io/"$OWNER"/debug-ghcr-cleanup-action:run-2
run-2: Pulling from XXXXX/debug-ghcr-cleanup-action
manifest unknown

The last run will also contain Annotations:
Ohne Titel

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.