opensourcevdi / disk-image

Generating disk images of (desktop) operating systems

While most Linux distributions provide pre-built cloud disk images (e.g., Debian, Rocky Linux) for server installations, pre-built disk images for desktop installation are lacking¹. Various open-source tools exist to automatically generate (and customize, e.g., with pre-installed packages) such disk images:

• virt-install (example in this repository) (potentially combined with Ansible; see virssh on how to get the IP address of a VM running with virt-install)
• ubuntu-image (example in this repository)
• Packer 1.9.5²
• mkosi
• virt-builder
• debos
• oz-install
• openstack-debian-image
• arch-boxes (and build_in_archiso_vm.sh)

How to download the images generated in this repository

Use oras:

curl -L "https://github.com/$(curl -L https://github.com/oras-project/oras/releases/ | grep -Eom1 '/[^"]*/oras_[^"]+_linux_amd64.tar.gz')" | tar xz -C ~/.local/bin oras

oras pull ghcr.io/opensourcevdi/disk-images:main-virt-install
oras pull ghcr.io/opensourcevdi/disk-images:main-ubuntu-image 

or the get-oci-blob-url from this repository to get a (time-limited) download URL:

./get-oci-blob-url ghcr.io/opensourcevdi/disk-images:main-virt-install
./get-oci-blob-url ghcr.io/opensourcevdi/disk-images:main-ubuntu-image

which you can, e.g., use with a web browser or combine with curl:

curl -Lo image.qcow2 "$(./get-oci-blob-url ghcr.io/opensourcevdi/disk-images:main-virt-install)"
curl -Lo image.qcow2 "$(./get-oci-blob-url ghcr.io/opensourcevdi/disk-images:main-ubuntu-image)"

Saving large files from GitLab CI/CD/GitHub Actions

An ideal storage for providing large public artifacts, e.g., disk images, produced by GitLab CI/CD or GitHub Actions would

• allow to host files with an unlimited file size
• for an indefinite time,
• provide the raw file/blob (as octet-stream),
• provide a digest (i.e., a cryptographic hash, e.g., SHA-256) of the file's content,
• be directly based on HTTP(S) (such that files can be linked and downloaded, e.g., via a web browser or curl),
• not require authentication for downloading (for uploading, it should be integrated into GitLab CI/CD or GitHub Actions),
• and allow for easy mirroring of the provided files to other locations.

The following public or self-hosted services fulfill these requirements to various degrees.

	Service	Maximum artifact size	Retention time	Container format	Provides digest of content	Access	Unauthenti­cated access	Mirroring easily possible
	GitLab CI/CD job artifacts	1000 MiB (on gitlab.com)	90 days or latest for branch	.zip (custom URLs to download individual files available)	No	HTTP redirect	Yes	No
	GitHub Actions job artifacts	unlimited	90 days	.zip	No	HTTP redirect or GitHub API	No, requires GitHub account both via web site and via GitHub API; workarounds exist but can be brittle	No
⭐	GitLab generic package registry	unlimited (on gitlab.com or with correct settings)	indefinite	octet-stream	Yes, SHA-256	HTTP redirect	Yes	Somewhat (custom API)
	GitHub releases (associated with a Git tag)	2 GiB	indefinite	octet-stream	No	HTTP redirect	Yes	No (custom API with rate limits)
	GitLab pages	1000 MiB (on gitlab.com)	indefinite	octet-stream	No	HTTP(S)	Yes	No
	GitHub pages	1 GB	indefinite	octet-stream	No	HTTP(S)	Yes	No
	GitLab Git repository	100 MiB (on gitlab.com)	indefinite	octet-stream	Yes, SHA-1 (SHA-256 planned)	HTTP(S)	Yes	Yes
	GitHub Git repository	100 MiB	indefinite	octet-stream	Yes, SHA-1 (SHA-256 planned)	HTTP(S)	Yes	Yes
(⭐)	GitLab Git LFS	10 GB (on gitlab.com)	indefinite	octet-stream	Yes, SHA-256	HTTP(S) or Git LFS API	Yes	Yes
	GitHub Git LFS	2 GB	indefinite	octet-stream	Yes, SHA-256	HTTP(S) or Git LFS API	Yes	Yes
⭐	GitLab container registry	unlimited (on registry.​gitlab.com)	indefinite (unless cleanup policy active)	octet-stream (or .tar.gz when not using ORAS)	Yes, SHA-256	OCI distribution spec3	Yes	Yes
⭐	GitHub container registry (ghcr.io)	unlimited	indefinite	octet-stream (or .tar.gz when not using ORAS)	Yes, SHA-256	OCI distribution spec3	Yes	Yes
(⭐)	Quay container registry (quay.io)	unlimited	indefinite	octet-stream (or .tar.gz when not using ORAS)	Yes, SHA-256	OCI distribution spec3	Yes	Yes
	Docker Hub container registry (registry-1.docker.io)	unlimited	indefinite	octet-stream (or .tar.gz when not using ORAS)	Yes, SHA-256	OCI distribution spec3	Yes (with rate limits)	Yes
(⭐)	External S3 (object/blob) storage	unlimited	indefinite	octet-stream	Yes but usually MD5-based (SHA-256 depending on provider)	HTTP(S) or S3 API	Yes	Yes

For all listed providers, unlimited storage is provided for free for public projects. However, GitLab is planning to introduce a (total) storage limit of 5 GiB in future.

Footnotes

1. For Windows, Microsoft is providing a pre-built development image. ↩

2. Later Packer versions are licensed as proprietary "source-available software", which is worse than providing no source code at all because the source code's availability is used to threaten open-source projects with claims of copyright infringement. Using Packer's non-cost version to "compete" with paid versions is forbidden by the proprietary license. Using Packer in GitLab CI/CD or GitHub Actions to generate publicly available disk images could potentially be characterized as competing with Packer's paid cloud version. ↩

3. The get-oci-blob-url script provided in this repository can generate an (often time-limited) HTTP(S) URL to download a file/blob from an (OCI) container registry. ↩ ↩² ↩³ ↩⁴