rainbowhackerhorse / vzvol

vzvol is a general use ZFS zvol management tool, that handles creation, destruction, listing, and formatting with various FSes, in an easy to use single program

Testing Status:

vzvol

vzvol is a tool to assist in the creation of ZFS zvols as storage for various virtualization providers.

What

Creates a ZFS zvol, and configures permissions, and creates and registers a VirtualBox VMDK shim for the zvol if you ask nicely.

Why

This allows you to use the zvol to back a disk for VirtualBox, bhyve, or other virtualization providers. vzvol also allows you to format your zvol with many filesystems, including:

• zfs
• ufs2
• fat32
• ext2,3,4
• xfs

Dependencies

vzvol is written to be a 0-dependency program for its main functionality. However, to enable the use of certain functions, some additional packages are required:

FreeBSD

• The -p flag requires the installation of sysutils/pv. This port is not needed if you don't use -p
• XFS filesystem support requires the installation of sysutils/xfsprogs
• ext2, ext3, and ext4 require the installation of sysutils/e2fsprogs
• vzvol no longer requires sudo, however now, it must either be run as root, or with sudo.

By default, on FreeBSD, these options are enabled. They can be disabled if installing fron ports.

Linux

• dialog
• dosfstools
• xfsprogs
• pv
• zfsutils

Contributing

Fork and open a PR with your changes. If you've contributed, please ensure you edit CONTRIBUTORS and add your GitHub username to the bottom if it isn't already listed!

Package Status

vzvol is currently supported in the following OS pkg systems:

FreeBSD Ports

1. cd /usr/ports/sysutils/vzvol && make install clean or pkg install vzvol

Debian (Using my repo)

1. Ensure apt-transport-https is installed.
2. Add deb https://repo-hackerhorse-io.nyc3.digitaloceanspaces.com/deb/ debian main to /etc/apt/sources.list.d/vzvol.list (Once DO allows for CNAMEs on Spaces, this will change to https://repo.hackerhorse.io/deb)
3. Run sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys FF07F6AE
4. Run apt-get update
5. Run apt-get install vzvol

Unsupported

• Debian Linux - If you know a Debian pkg maintainer who would like to sponser us getting a pkg made officially, get us in touch!

Building Linux Packages

I made building Linux packages easy here, thanks to FPM.

Dependencies: FPM

1. Clone the git repo
2. cd to where you cloned the repo to
3. cd build
4. ./vzvol_build.sh "version" "deb/rpm" "vzvol git directory" "maintainer email"

Creating a repo

I'm lazy and found a thing called aptly. This is how I created the Debian repo.

1. aptly repo create vzvol
2. aptly repo add vzvol vzvol_VERSION_all.deb
3. aptly snapshot create vzvol from repo vzvol
4. aptly publish -distribution=debian -architectures=amd64 snapshot vzvol
5. aptly publish -distribution=ubuntu -architectures=amd64 snapshot vzvol

Your repo will be in ~/.aptly/public Publish where you please, or just mirror mine!

vzvol's Issues

FreeBSD 11.1: check for no-zvols fails

On FreeBSD 11.1, the message "no datasets available" from zfs list -t volume goes to stdout, not stderr, so the check in vzvol_pre_list will never detect that error message, and always returns 0 -- which leads to funny results in vzvol_list.

Shell scripts contain errors

Hello RainbowHackerHorse,

Thank you for submitting the FreeBSD port.

Shell scrips in this project contains a considerable amount of errors.
Please use shellcheck to check your scripts.
ex: for ss in `find . -name "*.sh"`; do shellcheck $ss; done;

All scripts that have the shebang #!/bin/sh are expected to be Bourne shell scripts.

Could you please correct the errors and create a release here? No need to update the FreeBSD patch. I will take it from here.

Thanks!
Yuri

This shouldn't use logname to determine ZUSER

logname tells you the user who originally opened the terminal, not the actual current user.

For example, the following user is a user named "fred" I "su'd" to under "sean". I also specified the user with the -u flag. But it still creates a vmdk in /home/sean.

fred@debian:~$ sudo vzvol -s 10m -t virtualbox -v virttest5 -u fred
creating a vzol of size
10m
Testing to ensure zvol was created
/dev/zd928
DIE
Creating /home/sean/VBoxdisks/virttest5.vmdk
RAW host disk access VMDK file /home/sean/VBoxdisks/virttest5.vmdk created successfully.
Please use /home/sean/VBoxdisks/virttest5.vmdk as your VM Disk
fred@debian:~$ logname
sean

vzvol can't be automated

Options that require y/N should be able to use -y to automatically select Yes

FreeBSD 11.1: tries to su when it doesn't need to

When running vzvol as root, the su command should probably be left blank / do nothing, since there's no need to su. Something like this, perhaps:

if [ 0 = $( id -u ) ] ; then
    VZVOL_SU_CMD=sudo_do_nothing
    # Execute the command passed in, with no wrapping
    sudo_do_nothing() {
        "$@"
    }
fi

FreeBSD 11.1: missing error checks when run as user

Running vzvol as a regular user on FreeBSD 11.1 with no special permissions, no error checking seems to be done and no error messages are shown:

[user@beastie ~/src/]$ vzvol -v zscratch -p zarray
[user@beastie ~/src/]$

I don't even have a pool called zarray in the system.

FreeBSD 11.1: inconsistent use of $HOME (for root)

I can create a volume with a virtualbox shin:

root@beastie:/home/user # vzvol -s 1G -t virtualbox -p zarray -v whatevs
Testing to ensure zvol was created
Creating /home/root/VBoxdisks/whatevs.vmdk
RAW host disk access VMDK file /home/root/VBoxdisks/whatevs.vmdk created successfully.
Please use /home/root/VBoxdisks/whatevs.vmdk as your VM Disk

Root's home directory is /root, though. Running vzvol --list doesn't show me the associated VMDK, because that code is looking in the actual home-dir, so it's looking for /root/VBoxdisks/ which doesn't exist.

vzvol fails with spaces in import filename

When doing:

root@nvidion:~# vzvol -t virtualbox -s 120GB -u hcw -v Win7_64bitVBox --import ~hcw/VirtualBox\ 

i get

VMs/Win7/Win7\ 64\ bit.vdi 
Testing to ensure zvol was created
Testing to ensure zvol exists
Now importing /home/hcw/VirtualBox VMs/Win7/Win7 64 bit.vdi to /dev/zd0
This will DESTROY all data on /dev/zd0
Do you want to continue? [y/N]?y
Beginning import...
dd: nicht erkannter Operand »VMs/Win7/Win7“
„dd --help“ liefert weitere Informationen.
Error occurred in function vzvol_import_img
Exiting

This seems to be caused by spaces in the path to my .vdi file.

Can be fixed in vzvol_import.sh by quoting the dd arguments:

VZVOL_IMPORT_CMD="dd if=\"${IMPORTIMG}\" of=${ZVOL_IMPORT} status=progress"

--user no longer writes .vmdk in that user's directory

On FreeBSD -- so maybe this should be a downstream PR first -- the .vmdk files are no (longer, at least in my memory) created in the VBoxdisks directory in the user's home-dir, and are also not given permissions suitable for that user. E.g., on my home machine running 11-STABLE and port version 0.6.1,:

[moi@beastie ~]$ sudo vzvol --user moi -s 1G -t virtualbox -p zpool -v example
Testing to ensure zvol was created
Creating /root/VBoxdisks/example.vmdk
RAW host disk access VMDK file /root/VBoxdisks/example.vmdk created successfully.
Please use /root/VBoxdisks/example.vmdk as your VM Disk
[moi@beastie ~]$ ls -la /root/VBoxdisks/example.vmdk
-rw-------  1 root  wheel  551 May 24 15:11 /root/VBoxdisks/example.vmdk

This is (partly) because sudo in the default configuration drops $HOME from the environment, so it gets re-initialized to root's home. Can this be made smarter in vzvol? Or (downstream) a pkg-message could be added explaining how to configure sudo to preserve $HOME.

In addition, the permissions are not set on the .vmdk file to allow the user for whom the volume was created (and who has permissions from ZFS for the volume) to read the file in order to use it in VirtualBox.

FreeBSD 11.1: bad check for presence of sudo

Using pkg -N info | grep sudo to check for the presence of sudo is likely to hit false positives, e.g. with various sudoku programs installed. pkg -N info -e sudo is probably a better bet.

Remnants of development left in release

A wild user appears! Filing an upstream bug, rather than a packaging bug in FreeBSD. In vzvol 0.7.0, near the top of the script is this:

vzvol_gitdir=$( find ./ -name "vzvol" | grep bin | sed 's/.\{10\}$//' )

Which, when I run vzvol --list to find what is out there, does a find in my cwd -- which is the top of 3 GiB of source checkouts :( There might even be a vzvol clone in there somewhere, but that's not what I'm waiting for.

FreeBSD 11.1: doesn't work when run with cwd /root (or /tmp, or /usr/local/bin)

Congratulations on getting into the ports tree! vzvol sounds like it'd be awesome to use.

Which is a way of saying "but ..". I'll file issues here instead of FreeBSD's bugtracker, so there are fewer intermediates. Some of these problems might come down to packaging, rather than upstream -- in which case I'll re-report as a PR for olgeni.

I've installed vzvol from ports (built in poudriere locally), version 0.5.5. Running the script as root produces only an error message:

root@beastie:~ # which vzvol
/usr/local/bin/vzvol
root@beastie:~ # /usr/local/bin/vzvol
.: cannot open ../lib/shared/*: No such file or directory

FreeBSD 11.1: doesn't create volumes

vzvol doesn't create any volumes on my system, but doesn't produce an error message either:

root@beastie:/home/user # vzvol -s 1G -v whatevs 
root@beastie:/home/user # vzvol --list
ZVOL    TYPE  VMDK   USED        SIZE  FS
Error,  No    zvols  Configured

Adding -p and a pool name to specify a pool doesn't change behavior: no zvol is created.

FreeBSD 11.1: incorrect use of su -c

When run as a user, without sudo, VZVOL_SU_CMD is set to "su - root -c"; there's another issue filed about how this expands badly. But su is different from sudo in that it expects one parameter as the command, so a command like this one:

$ su - root -c zfs set custom:fs=none zbigone/whatevs3

goes on to fail. Using a shell function does the trick:

VZVOL_SU_CMD=sudo_do_su
sudo_do_su() {
su - root -c "$*"
}

Using vzvol in combination with su is a pain in the butt, though, since it'll run su four times during creation, so I get asked for my root password four times (ugh, but not your problem -- this is one reason why sudo was invented).

Anyway, this is my last GH issue: knowing that -t is mandatory, knowing to run it from somewhere where there isn't a ../lib in the way, with this shell function for su purposes, vzvol ticks an awful lot of boxes for me. Thanks. (I feel bad for dogpiling on you like this, please take this as an overenthusiastic "oh boy, cool tool that I really want to use" combined with me being a numbskull when using the tool.)

--delete doesn't

Admittedly, this is with vzvol 0.5, but I've looked and the code is unchanged.

root@beastie:/zpool/vm/iso # vzvol --list
ZVOL                         TYPE  VMDK  USED   SIZE   FS
zpool/bhyve-test             RAW   none  56K    20.6G  none
zpool/manjaro                RAW   none  13.1G  20.6G  none
zpool/netrunner              RAW   none  9.68G  12.4G  none
root@beastie:/zpool/vm/iso/pine64 # vzvol --delete zpool/bhyve-test
Error, zvol zpool/bhyve-test does not exist.
Try running vzvol --list or zfs list -t volume to see the available zvols on the system.
Error occurred in function getargz
Exiting

The code to check existence of a volume is different between --delete and --format; --delete only works if there's exactly one zvol, not when there's more. The problem is vzvol_getargz.sh line 97, in if ... | grep -vq "${2}" ; then .. this gives a successful exit if there are other zvols. You probably want the same code as in --format, which reverses the logic and drops the -v flag to grep.