GithubHelp home page GithubHelp logo

ovirt / imgbased Goto Github PK

View Code? Open in Web Editor NEW
8.0 9.0 5.0 1.19 MB

imgbased provides a specific management method to derive writeable filesystem layers from read-only base images.

License: GNU General Public License v2.0

Shell 6.63% Python 90.89% Makefile 1.45% M4 1.02%

imgbased's Introduction

imgbased

Copr build status

Welcome to the oVirt imgbased source repository. This repository is hosted on GitHub:imgbased.

imgbased provides a specific management method to derive writeable filesystem layers from read-only base images. It also takes care that the layer which shall be used can be selected at boot time.

In a nutshell this works by:

  • having a boot partition
  • and having a default LVM volume group (HostVG)
  • which has a thinpool
  • each base is kept in a read-only thin logical volume in the thinpool
  • for each base at least one writable layer, which is a thin logical volume, is created in the thinpool
  • for each layer a boot entry is created which can be used to boot in a specific layer

For more details see below.

How to contribute

All contributions are welcome - patches, bug reports, and documentation issues.

Submitting patches

Please submit patches to GitHub:imgbased. If you are not familiar with the process, you can read about collaborating with pull requests on the GitHub website.

Found a bug or documentation issue?

To submit a bug or suggest an enhancement for oVirt imgbased please report issues. If you find a documentation issue on the oVirt website, please navigate to the page footer and click "Report an issue on GitHub".

Still need help?

If you have any other questions or suggestions, you can join and contact us on the oVirt Users forum / mailing list.

Build the tool

How to build the tools.

sudo yum install -y make autoconf automake python
git clean -fdx
./autogen.sh
make dist
rpmbuild -ta imgbased-*.tar.xz

Using imgbase

To use the imgbase tool you need a Fedora or CentOS image, which was installed using the autopart --type=thinp directive. imgbased can be used to create new layers and install new bases.

# To create the assumed LVM layout
imgbase layout --init Base-1-0 --from /

# List existing layers and bases
imgbase layout

# Add a new base
# The `--size` argument specifies the size of the underlying
# logical volume. It must be at least the size of the filesystem
# contained in `$IMGFILE`.
imgbase base --add Base-2-0 --size 1G

# Get the latest base (which will be used for subsequent layers)
imgbase base --latest

# Add a new layer on the latest base or latest layer of the latest base
imgbase layer --add Base-1-0

# And with more infos
imgbase --debug layer --add

Purpose

Provide a more flexible solution for oVirt Node. Mainly a way where we can re-use existing technologies like anaconda.

High-Level Things

  • Read-Only bases (see also Drawbacks)
  • Write-able layers
  • Boot into layers
  • No inheritance between bases
  • Persistence between bases
    • Copy files between Base-N to Base-(N+1)

Features

  • Based on stable things (LVM, ext4)
  • A real filesystem (the layer) is modified and is used for boot
    • This solves all sort of problems with early boot
  • Sparseness everywhere
    • LVM with thinpool and thinvolume (frees space on discard operation)
    • ext4 FS with discard option (frees space after file removal)
  • LiveCD is only the delivery method
    • rootfs image is used at runtime
  • More distro agnostic than LiveCD
    • dracut, lvm (with thin volumes) and ext4 are the requirements

Drawbacks

  • Not as read-only as LiveCD-everywhere approach
    • The rootfs is kept on a ro LV, if this is changed to rw then the original base lv can be changed. This wasn't possible when the LiveCD was stored, because the rootfs was in a (as by limitation) read-only squashfs.
  • The persistence is a copy
    • When a new base image is installed, the persistence of configuration (and other) files from the previous base happens by copying the files. Previously bind mounts were used to achieve this.
  • Runtime space requirements are higher compared to LiveCD runtime
    • The LiveCD based delivery will be comparable in size.

LVM Structure

Assumptions about the host:

+ VG
|
+--+ Config (LV)
|
+--+ Base-1-0 (LV, ro)
|   \
|    \
|     + Base-1-0+1 (LV, rw)
|     |
|     + Base-1-0+2 (LV, rw)
|
|
+--+ Base-2-0 (LV, ro)
|   \
|    \
|     + Base-2-0+1 (LV, rw)
:     :

With a boot entry for each Base-* this allows the user to boot into each Base-LV. Changes are inherited from each Base-* and can also be persisted using the Config LV.

LiveCD Payload

The image is the (in future) intended to be also used as a paylod for LiveCD ISOs and to be deployed via PXE. Because of this we want to minimize the image even further. For this virt-sparsify and squashfs can be used to simulate the size of the image when it is used as a payload.

# To get an idea of the minimized size use
make runtime-layout.squash

The LiveCD will be created the livemedia-creator which is part of lorax.

imgbased's People

Contributors

arachmani avatar didib avatar dougsland avatar dupondje avatar fabiand avatar lveyde avatar michalskrivanek avatar mrkev-gh avatar nijinashok avatar sandrobonazzola avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

imgbased's Issues

Improve test coverage

Is your feature request related to a problem? Please describe.

The automation we had in Jenkins did deep integration testing using virtualization.

We should do something similar here.

Describe the solution you'd like
I'd like the runtime tests executed in https://github.com/oVirt/imgbased/blob/master/automation/check-patch.sh to be covered by test also within GitHub actions workflow.

Describe alternatives you've considered
As an alternative we can try to cover with unit testing and mocking but it may not show up issues tipical to upgrades like selinux denials and config errors.

Additional context
No additional context.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.