bsdci / libioc

A Python library to manage jails with ioc{age,ell}

libioc

Python Library to manage FreeBSD jails with ioc{age,ell}.

iocage is a jail/container manager fusioning some of the best features and technologies the FreeBSD operating system has to offer. It is geared for ease of use with a simple and easy to understand command syntax.

This library provides programmatic access to iocage features and jails, while aiming to be compatible with iocage_legacy, iocell and the Python 3 version of iocage (< 1.0).

Install

git clone https://github.com/bsdci/libioc
cd libioc
make install

The default Python version is 3.7. If you intend to run libioc from another version, please specify it during the installation:

make PYTHON=python3.8 install

At the current time libioc is not packaged or available in FreeBSD ports.

Documentation

• Ioc Handbook: https://bsdci.github.io/handbook
• Reference Documentation: https://bsdci.github.io/libioc
• Gitter Chat: https://gitter.im/libioc/community

Configuration

Active ZFS pool

libiocage iterates over existing ZFS pools and stops at the first one with ZFS property org.freebsd.ioc:active set to yes. This behavior is the default used by other iocage variants and is restricted to one pool managed by iocage

Root Datasets configured in /etc/rc.conf

When iocage datasets are specified in the jail hosts /etc/rc.conf, libiocage prefers them over activated pool lookups. Every ZFS filesystem that iocage should use as root dataset has a distinct name and is configured as ioc_dataset_<NAME>="zroot/some-dataset/iocage", for example:

$ cat /etc/rc.conf | grep ^ioc_dataset
ioc_dataset_mysource="zroot/mysource/iocage"
ioc_dataset_othersource="zroot/iocage"

iocage commands default to the first root data source specified in the file. Operations can be pointed to an alternative root by prefixing the subject with the source name followed by a slash.

import ioc
release = libioc.Release("12.0-RELEASE")

jail_a = libioc.Jail(new=True, {})
ioc create othersource/myjail
ioc rename othersource/myjail myjail2

When othersource is the only datasource with a jail named myjail the above operation would have worked without explicitly stating the dataset name.

Legacy Support

With upcoming releases existing and future legacy / compatibility features will be disabled by default. Setting the sysrc ioc_legacy_support="YES" these compatibility features:

• ZFS Basejail Support (iocage_legacy)

On initialization libioc detects the hosts sysrc setting ioc_legacy_support that can be enabled to unlock features liste above.

sysrc ioc_legacy_support="YES"

Usage

Library

import ioc

jail = libioc.Jail()
jail.create("11.1-RELEASE")

CLI

libioc has a CLI tool called ioc that is no longer bundled with the library, but can be installed individually. It is inspired by the command line interface of iocage but meant to be developed along with the library and to spike on new features.

Documentation

The API Reference (html) documenting all public interfaces of libioc is updated with every release. The information found in the reference is compiled from Python docstrings and MyPy typings using Sphinx.

Development

Unit Tests

Unit tests may run on FreeBSD or HardenedBSD and require an activated ioc pool.

ZPOOL=zroot make test

Static Code Analysis

The project enforces PEP-8 code style and MyPy strong typing via flake8, that is required to pass before merging any changes. Together with Bandit checks for common security issues the static code analysis can be ran on Linux and BSD as both do not require py-libzfs or code execution.

make install-dev
make check

---

Project Status (Archive)

2018-09-22

Progress towards the transition of python-iocage using libiocage has been made. Recent changes to both projects ensure compatibility running on the same host, so that it is now possible to partially utilize libiocage in iocage until a full migration is performed. Because some changes to the command line arguments and the script output will occur, @skarekrow will continue to maintain the current implementation until users had time to follow the deprecation warnings and suggestions.

In terms of the "Advanced container management with libiocage" tutorial at EuroBSDCon 2018 the Handbook was published.

2018-08-07

libiocage is making small but continuous steps to stabilize the interfaces and become used in iocage/iocage. The project was first presented in the talk "Imprisoning software with libiocage" at BSDCan 2018 (Video Recording on YouTube). There will be a Tutorial about Advanced container management with libiocage on September 20th, 2018 at EuroBSDCon in Bucharest.

Ongoing preparations at this repository and iocage ensure that the transition to using libiocage under the hood of iocage go as smooth as possible for users. Features that exist in iocage will be further improved and tested or announced to be replaced or deprecated shortly. iXsystems let one imagine that libiocage once finds its way into FreeNAS where it can play its full strength behind a Web GUI.

2017-11-14

As of November 2017 this project is working towards an alpha release. This means stabilization of command-line and library interfaces, so that proper integration tests can be built. This phase requires manual verification and testing until reaching feature-completion and compatibility with Python iocage and prior iocage_legacy versions with ZFS property and UCL file config storage.

libioc's Issues

Update release basejail datasets after updating a release

When a release that was already copied to the collection of base datasets (/iocage/base/<RELEASE>), the datasets representing the datasets need to get updated as well.

I wonder what the most efficient way to do that is. Here are some options:

Run freebsd-update a second time

+ does not need to write/touch all files
- HardenedBSD currently does not cache assets on the host (yet)
- It it another source of errors that require our action (e.g. roll back changes from the previous release update, so that the release and the base do not diverge)

Copy over the files

+ simple
- slow
- reads and writes back from disk
- errors like full disks may still require our action to handle rollbacks

Suggestions welcome

Feature: Resource Limits

FreeBSD allows to define resource limits for jails, as seen in the documentation:
https://www.freebsd.org/doc/handbook/security-resourcelimits.html

See also:

• rctl man page
• rctl.conf man page
• expand_number man page

Requirements

• Resource limits can be defined via JailConfig
• Memory can be limited via limit_memoryuse option
• When limiting memory
  • a jail may be running
  • values can be provided in human readable format according to expand_number
• CPU usage on a single CPU core in percent can be limited via limit_cputime option
• Other limits can be set via limit_<RESOURCE> according to the rctl man page RESOURCES section

code disagrees with comments disagrees worth types

in working on #86 / #88, i stumbled across this piece of art in Resource.py:

    @property
    def dataset(self) -> libzfs.ZFSDataset:
        """
        The jail's base ZFS dataset
        """
        if self._dataset_name is not None:
            # sets self._dataset_name to None and memoize the dataset
            self._dataset = self.zfs.get_dataset(self.dataset_name)

        return self._dataset

ioc start does not work for basejail and have issues with clonejail

• Supply iocage --version
  latest master ioc, version 0.2.11 08/29/2017

Using ioc start UUID on basejail type of legacy jail just return with no info also jail is not running. Using same command on clonejail type starts the jail but with some errors

Command exited with 1: /usr/sbin/jexec ioc-9048b2dc-91ac-11e7-b8c6-7071bca7c826 /bin/sh /etc/rc
    jexec: execvp: /bin/sh: No such file or directory
Command exited with 1

I managed to get it work for a while but I was unable to replicate fully working start with latest master.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Feature: Rename jail

A jails name also defines the jails dataset location relative to the activated zpool. That's why it's important to do more than just renaming the property. Certainly we must force a jail to be stopped for renaming it.

Requirements

• When renaming a Jail
  • an exception is raised when the jail is started
  • moves the dataset immediately
  • changes the host_hostname when it was manually set (available in config.data) and matches the old jail name

Properly handle KeyboardInterrupt

Users should have the ability to abort the process at any time.

In case of half-finished progress steps, we should warn and provide a way to cleanup. For example an aborted stop-command should say "Stopping jail 'ABC' was not complete. Please run 'ioc stop ABC --force' to ensure the jail was stopped."

Stop fails with TypeError: string indices must be integers

# python3.6 . -d spam stop myjail
[+] JailDestroy: OK [0.005s]: 1
[-] JailNetworkTeardown: ...
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "./__main__.py", line 51, in <module>
    main()
  File "./__main__.py", line 45, in main
    cli(prog_name="iocage")
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "./libiocage/cli/stop.py", line 57, in cli
    ctx.parent.print_events(jail.stop(force=force))
  File "./libiocage/cli/__init__.py", line 64, in print_events
    for event in generator:
  File "./libiocage/lib/Jail.py", line 377, in stop
    self._stop_vimage_network()
  File "./libiocage/lib/Jail.py", line 788, in _stop_vimage_network
    for network in self.networks:
  File "./libiocage/lib/Jail.py", line 1068, in __getattribute__
    return object.__getattribute__(self, key)
  File "./libiocage/lib/Jail.py", line 758, in networks
    bridges = list(self.config["interfaces"][nic])
TypeError: string indices must be integers

allow_chflags:0
allow_mount:0
allow_mount_devfs:0
allow_mount_nullfs:0
allow_mount_procfs:0
allow_mount_tmpfs:0
allow_mount_zfs:0
allow_quotas:0
allow_raw_sockets:0
allow_set_hostname:1
allow_socket_af:0
allow_sysvipc:0
basejail:yes
boot:no
children_max:0
clonejail:yes
defaultrouter:-
defaultrouter6:-
devfs_ruleset:4
enforce_statfs:2
exec_clean:1
exec_fib:1
exec_poststart:/usr/bin/true
exec_poststop:/usr/bin/true
exec_prestart:/usr/bin/true
exec_prestop:/usr/bin/true
exec_start:/bin/sh /etc/rc
exec_stop:/bin/sh /etc/rc.shutdown
exec_timeout:60
host_domainname:-
id:myjail
interfaces:vnet0:bridge1
ip4:new
ip4_addr:-
ip4_saddrsel:1
ip6:new
ip6_saddrsel:1
jail_zfs:no
legacy:-
mac_prefix:02ff60
mount_devfs:1
mount_fdescfs:1
priority:0
release:11.1-RELEASE
resolver:/etc/resolv.conf
securelevel:2
stop_timeout:30
sysvmsg:new
sysvsem:new
sysvshm:new
tags:-
vnet:yes

Refactor cross-dependencies between Resource, Config and Jail/Release

Currently the order of imports influences the ability to load modules and classes. This becomes a problem when using mypy to do strict type checking.

The classes should be refactored, so that each class may be loaded individually and order independent.

Check for optional return types

As @igalic noted in #83 (comment) the library returns None in functions that do not return an optional type.

def something() -> str:
    # ....
    return None

should be instead

import typing
def something() -> typing.Optional[str]:
    # ...
    return None

Docs: Development Environment

We introduced code checking with flake8 and mypy, but did not document how to configure and use the development environment.

At least developers need to know a list of dependencies and some helper to install them on FreeBSD and HardenedBSD.

• Makefile
  • make travis runs the same tasks as they would on TravisCI
  • make test runs the code style checks and unit tests
  • make dev-dependencies installs developer dependencies (and general dependencies)
  • make install installs libiocage to the system
• Readme
  • lists general iocage dependencies
  • lists iocage development dependencies
  • explains how to setup the development environment using make dev-dependencies

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

KeyError: 'Config variable clonejail not found'

root@localhost:/usr/local/src/libiocage # ioc list
...
  File "/usr/local/bin/ioc", line 12, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/libiocage/cli/get.py", line 95, in cli
    value = jail.config["get_string"](key)
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 491, in get_string
    return self.__getitem__(key, string=True)
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 528, in __getitem__
    raise KeyError(f"Config variable {key} not found")
KeyError: 'Config variable clonejail not found'

if a jail has no config, ioc throws a stacktrace

ioc version is head

~ # ioc list -o jid -o name -o release                              
Traceback (most recent call last):                                  
  File "/usr/local/bin/ioc", line 12, in <module>                   
    sys.exit(cli())               
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__                                                     
    return self.main(*args, **kwargs)                               
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main                                                         
    rv = self.invoke(ctx)         
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke                                                      
    return _process_result(sub_ctx.command.invoke(sub_ctx))         
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke                                                       
    return ctx.invoke(self.callback, **ctx.params)                  
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke                                                       
    return callback(*args, **kwargs)                                
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func                                                
    return f(get_current_context(), *args, **kwargs)                
  File "/root/libiocage/libiocage/cli/list.py", line 104, in cli    
    _print_table(jails, columns, header, _sort)                     
  File "/root/libiocage/libiocage/cli/list.py", line 126, in _print_table                                                                
    table_data.append(_lookup_jail_values(jail, columns))           
  File "/root/libiocage/libiocage/cli/list.py", line 176, in _lookup_jail_values                                                         
    columns                       
  File "/root/libiocage/libiocage/cli/list.py", line 175, in <lambda>                                                                    
    lambda column: jail.getstring(column),                          
  File "/root/libiocage/libiocage/lib/Jail.py", line 926, in getstring                                                                   
    return libiocage.lib.helpers.to_string(self.__getattr__(key))   
  File "/root/libiocage/libiocage/lib/Jail.py", line 893, in __getattr__                                                                 
    return object.__getattribute__(self, key)                       
  File "/root/libiocage/libiocage/lib/Jail.py", line 849, in release                                                                     
    name=self.config["release"],  
  File "/root/libiocage/libiocage/lib/JailConfig.py", line 567, in __getitem__                                                           
    return self.defaults[key]     
KeyError: 'release'               

this error is specific to the release column, since that has no defined default.

Release updates of 10.3-RELEASE failed

root@zwei:/usr/local/src/libiocage # ioc fetch -r 10.3-RELEASE
[+] FetchRelease: OK [21.26s]: 4
  [+] ReleasePrepareStorage: OK [0.201s]: 1
  [+] ReleaseDownload: OK [3.355s]: 1
  [+] ReleaseExtraction: OK [17.703s]: 1
[+] ReleaseConfiguration: OK [0.002s]: 1
[-] ReleaseUpdateDownload: ...
Command exited with 1: /zroot/iocage/releases/10.3-RELEASE/updates/freebsd-update.sh -d /zroot/iocage/releases/10.3-RELEASE/updates -f /zroot/iocage/releases/10.3-RELEASE/updates/freebsd-update.conf --not-running-from-cron fetch
Command exited with 1
Traceback (most recent call last):
  File "/usr/local/bin/ioc", line 12, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/libiocage/cli/fetch.py", line 98, in cli
    fetch_updates=fetch_updates
  File "/usr/local/lib/python3.6/site-packages/libiocage/cli/__init__.py", line 64, in print_events
    for event in generator:
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/Release.py", line 393, in fetch
    for event in ReleaseGenerator.fetch_updates(self):
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/Release.py", line 526, in fetch_updates
    ], logger=self.logger)
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/helpers.py", line 139, in exec
    logger=logger
libiocage.lib.errors.CommandFailure: Command exited with 1

Use `--libxo=json` to parse jls output

@igalic mentioned in a PR review that we should consider using --libxo=json to get parsable output of jls.

Personally I would prefer to get the jail state from a python library, but until then the formatted output is a clean alternative to the way we currently parse the output.

Validation for `list`'s `output`

We need it ;)

https://github.com/iocage/libiocage/blob/6068a5c4ec7c4c2e7fdacaad8740e7f2d27cc3e3/libiocage/cli/list.py#L77

Feature: support for resolving runtime variables [$50]

As discussed through IRC adding option to use variables on specific attributes that are resolved on runtime would make using exec.pre/post_start/stop a lot easier. Native jail with jail.conf supports this out of the box (jail.conf, but I think it's not possible via cmd line using jail -c ...

A workaround I did on legacy jail was to add jail name while calling exec scripts urosgruber/iocage@ca766a7

Example of an old script we used post_start script that is using jail name as an argument but could be improved by providing JID or some other parameter

#!/bin/sh

# get current jid
_jid=`jls -j $1 jid 2>/dev/null`
_ip=`jls -j $1 ip4.addr 2>/dev/null`
_name=$1

# cpuset
# cpuset -c -l 0 -j ${_jid}

# mem limit
rctl -a jail:$1:memoryuse:deny=256M

# pf
( pfctl -a jails -sr ; echo "pass on lo0 from ${_ip} to ${_ip}" ) | pfctl -a jails -f -

# update started time and status
mysql -ss -D db -e "UPDATE service SET started=NOW(), status='running', jid=${_jid} WHERE name='$1';" 2> /dev/null

Similarly for post_stop script and pre_ scripts.

Nice to have would be to specify a folder pre_start.d where all executable files would be executed with same parameters. example: ioc set exec_prestart="/my/prestart.d/ ${jid}"

---

Did you help close this issue? Go claim the $50 bounty on Bountysource.

Exclude releases newer than the host release

It is unsupported to run jails with releases newer than the host's major version. We should not offer to fetch those releases when prompting the user and properly warn when attempting to create from an unsupported release.

Proper warning messages are important and I'd love to hear ideas how to inform that there are newer releases available that are unsupported. Maybe "There are 2 more releases available newer than your host release"?

Allow dataset from different pool in jail_zfs_dataset

@rexpo opened this on iocage/iocage#345
iocage is activated on one (for example small, SSD-backed) zpool, and I want to be able to mount datasets from a second (for example massive, but slow HDD-backed) zpool.

This isn't currently possible, as iocage can only mount a dataset from the same pool that iocage is activated on.

I know that I should just shut up, activate the big slow pool, throw lots of RAM at the host, and take comfort in the fact that the ZFS ARC will eventually speed things up. There are, however, other use cases that benefit from the ability to mount a dataset from another pool, especially when it comes to maintenance and migration.

As far as I can tell, work was done on iocage_legacy to implement just such a feature, but that it never made it to prime time. I'm sure this has already been discussed before, but I couldn't find any closed issues about it. Has this already been considered?

Feature: Destroy

The current implementation lacks the ability to destroy jails.

Requirements

• ioc destroy <JAIL_GLOB> destroys stopped jails
• ioc destroy -r <RELEASE_GLOB> destroys a release

Securely create jails

iocage/iocage#347 was the original issue, iocage/iocage@54583ed commit that fixed it in iocage.

Makefile using invalid pip executable

Doing a fresh make gives

python3.6 -m ensurepip
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages
Requirement already satisfied: pip in /usr/local/lib/python3.6/site-packages
pkg install -q -y libgit2 libucl cython3
The most recent version of packages are already installed
pip3.6 install -U .
make: exec(pip3.6) failed (No such file or directory)
*** Error code 1

Stop.

Renaming pip3.6 to pip-3.6 fixes make process. I'm not sure if something is installed on a system differently but I installed pip through pkg.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

HardenedBSD Support

Libiocage is not able to fetch and update HardenedBSD releases yet.

libiocage/ioc list -r prints the same output as ioc list

on libiocage#HEAD, ioc list -r should print the releases,

root@container-host1 ~/libiocage# ioc list -r
+-----+------+---------+---------+----------+
| JID | NAME | RUNNING | RELEASE | IP4.ADDR |
+=====+======+=========+=========+==========+
+-----+------+---------+---------+----------+

however, it prints the same thing as ioc list

root@container-host1 ~/libiocage# ioc list 
+-----+------+---------+---------+----------+
| JID | NAME | RUNNING | RELEASE | IP4.ADDR |
+=====+======+=========+=========+==========+
+-----+------+---------+---------+----------+

make install affects files in /usr/src

...
   A cddl/boot/zfs
   A cddl/dev/fbt/powerpc/fbt_isa.c
   A cddl/dev/fbt/fbt.c
   A cddl/dev/fbt/powerpc/fbt_isa.h
   A cddl/dev/fbt/riscv/fbt_isa.h
   A cddl/dev/fbt/aarch64/fbt_isa.h
   A cddl/contrib/opensolaris/common/zfs/zpool_prop.c
   A cddl/contrib/opensolaris/common/zfs/zfs_namecheck.h
   A cddl/dev/fbt/x86/fbt_isa.c
   A cddl/dev/fbt/mips/fbt_isa.c
   A cddl/contrib/opensolaris/common/zfs/zfs_prop.h
   A cddl/contrib/opensolaris/common/atomic/amd64/opensolaris_atomic.S
   A cddl/contrib/opensolaris/common/avl/avl.c
   A cddl/dev/dtrace/powerpc/dtrace_isa.c
   A cddl/dev/dtrace/dtrace_debug.c
   A cddl/dev/dtrace/amd64/instr_size.c
   A cddl/dev/dtrace/i386/instr_size.c
   A cddl/dev/dtrace/dtrace_load.c
   A cddl/dev/dtrace/aarch64/dtrace_subr.c
   A cddl/dev/dtrace/riscv/dtrace_subr.c
   A cddl/dev/dtrace/mips/dtrace_isa.c
   A cddl/dev/dtrace/arm/dtrace_asm.S
   A cddl/dev/dtrace/dtrace_hacks.c
   A cddl/dev/dtrace/dtrace_modevent.c
   A cddl/dev/fbt/arm/fbt_isa.c
   A cddl/dev/fbt/x86/fbt_isa.h
   A cddl/dev/fbt/mips/fbt_isa.h
   A cddl/dev/systrace/systrace.c
   A cddl/dev/prototype.c
   A cddl/boot/zfs/blkptr.c
   A cddl/boot/zfs/lzjb.c
   A cddl/dev/dtrace/dtrace_vtime.c
   A cddl/dev/fbt/arm/fbt_isa.h
   A cddl/dev/fbt/riscv/fbt_isa.c
   A cddl/dev/fbt/aarch64/fbt_isa.c
   A cddl/dev/sdt/sdt.c
   A cddl/dev/fbt/fbt.h
   A cddl/dev/dtmalloc/dtmalloc.c
   A cddl/dev/profile/profile.c
   A cddl/boot/zfs/lz4.c
   A cddl/boot/zfs/README
   A cddl/boot/zfs/zle.c
   A cddl/boot/zfs/zfssubr.c
   A cddl/boot/zfs/sha256.c
   A cddl/boot/zfs/zfsimpl.h
   A cddl/boot/zfs/fletcher.c
Checked out revision 323616.
Summary of conflicts:
  Tree conflicts: 4
Tree conflict on 'cddl/boot'
   > local dir unversioned, incoming dir add upon update
Select: (r) mark resolved, (p) postpone, (q) quit resolution, (h) help: q

Feature: Tags

In addition to a resource name, jails may have one or more tags that makes them selectable.

The general functionality to provide tags is embedded in libiocage, but we need to expose the feature via the command line interface to make it usable.

For compatibility with previous versions the tags should be stored as comma separated string. Another thing to consider is the previously used jail tag configuration option which should be seamlessly merged into the tags when it exists.

Requirements

• a jail can have one or more tags
• a jails tags can be shown in list command output
• jail filters can match individual tags
• jail config property tags can be set from a comma separated string
• jail config property tags can be set from a list of strings
• When no user defined tags property was found in a jail's configuration
  • setting a tag JailConfig item sets the deprecated tag property
  • setting tags removed any existing tag property from the configuration
• When user defined tags are already existing
  • setting a tag that already exists in tags makes it the first item in tags
  • setting a tag that does not exist in tags yet, adds it as first item

Feature: DHCP

Jails should be allowed to set ip4_addr=vnet0|dhcp to obtain an IPv4 address via DHCP on the target interface vnet0.

DHCP requires access to /dev/bpf, which needs to be granted via udev rule on the host.

Requirements

An acceptable solution:

• creates a set of udev rules per jail
• removes jail udev rules when stopping a jail
• when dhcp is set on any interface in ip4_addr
  • enables BPF in the jail's udev rule before starting the ip4_addr
• when dhcp is not set on any interface in ip4_addr
  • disables BPF in the jail's udev rule before starting the jail

Download release checksums via trusted connection

The manifest containing the release asset signatures is downloaded from the same untrusted http resource as the assets. To be an effective security feature we need to verify this signatures from trusted sources.

• downloads Hardened BSD signatures via encrypted connection
• downloads FreeBSD signatures via encrypted connection
• allows to specify a custom signature source

CLI

• allows the user to skip verification (currently --verify/--no-verify option
• allows to manually specify signatures (e.g. --signatures base=62acaee7e... or similar)

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Feature: Internationalize CLI output

Introduced with #37 there will be signaling between the library and the cli, which is responsible for printing event to the user. All events are listed in libiocage/lib/events.py. Another list of event-like items is libiocage/lib/errors.py. Both are inviting for internationalization!

To generate similar output to previous iocage versions, we should begin with adding translations for English language.

We may wait with translation iocage to further languages until we come close to a stable release, so that we don't have update translations among changing interfaces.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Add License information to .py files

This is a non-commercial project and the credit belongs to the "iocage project", which is a group of independent contributors. We should put proper license information (BSD 2-clause) to every .py file in the library.

See also: https://github.com/iocage/libiocage/blob/master/LICENSE.txt

`make install` is broken: 'dependency_links' must be a list of strings

on the current HEAD (3f184d9), when running make install i get the following error:

vagrant@ ~/libiocage> sudo -H make install
python3.6 -m ensurepip
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages
Requirement already satisfied: pip in /usr/local/lib/python3.6/site-packages
pkg install -q -y libgit2
The most recent version of packages are already installed
pip3.6 install -U .
Processing /usr/home/vagrant/libiocage
    Complete output from command python setup.py egg_info:
    error in libiocage setup command: 'dependency_links' must be a list of strings (got <map object at 0x8080cdb00>)
    fastentrypoints
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-esegjs7v-build/
*** Error code 1

Stop.
make: stopped in /usr/home/vagrant/libiocage
vagrant@ ~/libiocage>

Flake 8 errors are not printed in TravisCI

The output of TravisCI build logs leaves a hint that there are 67 code style issues, but does not print them or cause a failed build status.

From the Travis CI build logs:

...
$ cd iocage/libiocage
0.41s$ git fetch origin +refs/pull/33/merge:
remote: Counting objects: 45, done.
remote: Compressing objects: 100% (15/15), done.
Unpacking objects: 100% (45/45), done.
remote: Total 45 (delta 30), reused 44 (delta 30), pack-reused 0
From https://github.com/iocage/libiocage
 * branch            refs/pull/33/merge -> FETCH_HEAD
$ git checkout -qf FETCH_HEAD
0.01s$ source ~/virtualenv/python3.6/bin/activate
$ python --version
Python 3.6.2
$ pip --version
pip 9.0.1 from /home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages (python 3.6)
install
1.16s$ pip3 install flake8
Collecting flake8
  Downloading flake8-3.4.1-py2.py3-none-any.whl (68kB)
    100% |████████████████████████████████| 71kB 2.6MB/s 
Collecting pyflakes<1.6.0,>=1.5.0 (from flake8)
  Downloading pyflakes-1.5.0-py2.py3-none-any.whl (225kB)
    100% |████████████████████████████████| 225kB 3.5MB/s 
Collecting mccabe<0.7.0,>=0.6.0 (from flake8)
  Downloading mccabe-0.6.1-py2.py3-none-any.whl
Collecting pycodestyle<2.4.0,>=2.0.0 (from flake8)
  Downloading pycodestyle-2.3.1-py2.py3-none-any.whl (45kB)
    100% |████████████████████████████████| 51kB 7.8MB/s 
Installing collected packages: pyflakes, mccabe, pycodestyle, flake8
Successfully installed flake8-3.4.1 mccabe-0.6.1 pycodestyle-2.3.1 pyflakes-1.5.0
4.69s$ .travis/flake8.sh
Current Error Count: 67
usage: git rev-list [OPTION] <commit-id>... [ -- paths... ]
  limiting output:
    --max-count=<n>
    --max-age=<epoch>
    --min-age=<epoch>
    --sparse
    --no-merges
    --min-parents=<n>
    --no-min-parents
    --max-parents=<n>
    --no-max-parents
    --remove-empty
    --all
    --branches
    --tags
    --remotes
    --stdin
    --quiet
  ordering output:
    --topo-order
    --date-order
    --reverse
  formatting output:
    --parents
    --children
    --objects | --objects-edge
    --unpacked
    --header | --pretty
    --abbrev=<n> | --no-abbrev
    --abbrev-commit
    --left-right
    --count
  special purpose:
    --bisect
    --bisect-vars
    --bisect-all
fatal: No names found, cannot describe anything.
Comparing with last stable release: 
's Error Count: 67
The command ".travis/flake8.sh" exited with 0.
Done. Your build exited with 0.

Feature: Plugins

Iocage implements a mechanism to automatically install jails from a template called "plugin". Various plugins are listed in the freenas plugin registry.

The way of installing packages and applying post-install hooks can be abstracted as discussed in iocage/iocage#163 about Ansible (or other provisioning tools) integration. Running a hook script read from a GitHub source would be sufficient to support the current plugin system.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Refactor Storage.py

Jails were undergoing some fundamental changes towards Resources. The Storage class should be refactored and utilize new classes and helpers, specially ZFS.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Have the same user output as iocage

start: Little output, should look the same. JID can be included in the same style.

stop: Little output, should look the same.

There are likely more, but just creating this as a placeholder so when this becomes iocage, the user will not notice a difference. This is important for scripting and other things we aren't necessarily privy of.

Bug: Do not **unintentionally** modify function arguments

We have a handful of functions that take non-None arguments, and then modify them.
This leads to python directly modifying the passed argument, and is generally considered bad form.

One such example is JailConfigAddresses#add()

    def add(self, nic, addresses=[], notify=True):

        if isinstance(addresses, str):
            addresses = [addresses]

maybe there are linter params that will warn us of such code, until then we might need human eyes.

Writing to files in jail's root path might compromise the host

During jail start we write to a bunch of files within a jail's root dataset. When those assets are a symlink, the host system will write to the symlink target instead of a (safe) file only in the jails dataset.

Due to the nature of these helpers, the attacker would not be controlling the full content of the files, even partial control over the file content might allow to compromise the host (e.g. .ssh/authorized_keys will be read even if a few lines are broken)

This was discovered by @igalic in #72 (comment)

Feature: Resource Limits

In previous versions of iocage it was possible to set resource limits on jails.

Sort Python imports according to style guidelines

According to CONTRIBUTING.md imports should be ordered with isort.

Names for creation of a number of jails

When creating a number of jails with --count flag, names cannot be set, because the second creation attempt will fail because the jail with this name already exists.

It is required to introduce a wildcard character that represents the index of the currently created jail.

My suggestion is: ioc create --count 128 --name "my-%i-jail-of-%n" where i is the index and n is the total number of jails to create.

Allow to set base dataset other than iocage

I think iocage dataset is what is currently hardcoded, right? It would be great if iocage activate would allow to specify base dataset other than iocage. This would allow easier migration from legacy jails. Right now only single dataset can exist and after installing new iocage old legacy jails will not properly start/stop.

I created iocage_legacy port with renamed commands but I'm not sure how to get around this dataset naming clash.

Feature: EOL warnings [$25]

When fetching releases EOL warnings should appear next to outdated releases.

---

Did you help close this issue? Go claim the $25 bounty on Bountysource.

Using ioc start creates iocage/log dataset and iocage_legacy can't handle this

Creating jail with iocage_legacy iocage create tag=myjail and then starting this jail with ioc start UUID creates dataset zpool/iocage/log

If using iocage list after dataset created makes iocage legacy outputing a lot of errors

cannot open 'sys/iocage/jails/': invalid dataset name
cannot open 'sys/iocage/jails/': invalid dataset name
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
cannot open 'sys/iocage/jails/': invalid dataset name
cannot open 'sys/iocage/jails/': invalid dataset name
cannot open 'sys/iocage/jails/': invalid dataset name
cannot open 'sys/iocage/jails/': invalid dataset name
  ERROR: missing property or UUID/TAG!

Although it outputs list of jails after this errors as usual.

Destroying sys/iocage/jails manually fixes errors above but after running ioc start again recreates dataset.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

ioc create is broken

Using latest master ioc create creates jail half way with and error

9f24cf51 could not be created!
Traceback (most recent call last):
  File "/usr/local/bin/ioc", line 12, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/libiocage/cli/create.py", line 156, in cli
    jail.create(release.name)
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/Jail.py", line 402, in create
    self.config.save()
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 209, in save
    self.jail.rc_conf.save()
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/Jail.py", line 162, in rc_conf
    logger=self.logger
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/RCConf.py", line 41, in __init__
    self.path = path
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/RCConf.py", line 52, in path
    self._read_file()
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/RCConf.py", line 63, in _read_file
    new_keys = set(data.keys())
UnboundLocalError: local variable 'data' referenced before assignment

But ioc list show newly created jail

+-----+------------+--------------------------------------+---------+--------------+----------------+-----------+--------+
| JID |    TAG     |                 NAME                 | RUNNING |   RELEASE    |    IP4.ADDR    |   TYPE    | LEGACY |
+=====+============+======================================+=========+==============+================+===========+========+
| -   | -          | 9f24cf51-9ae4-4f28-88f8-a71dabadcf39 | no      | 11.1-RELEASE | -              | -         | no     |
+-----+------------+--------------------------------------+---------+--------------+----------------+-----------+--------+

And then ioc get all 9f24cf51-9ae4-4f28-88f8-a71dabadcf39 throws exception as well

basejail:no
Traceback (most recent call last):
  File "/usr/local/bin/ioc", line 12, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/libiocage/cli/get.py", line 95, in cli
    value = jail.config.get_string(key)
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 504, in get_string
    return self.stringify(self.__getitem__(key))
  File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 546, in __getitem__
    return self.defaults[key]
KeyError: 'clonejail'

And funny thing, jail can be started with ioc start 9f24cf51-9ae4-4f28-88f8-a71dabadcf39

Note: activate was done by iocage legacy.

Makefile support for HardenedBSD

The makefile downloads sources (if they are not available yet) for the current host distribution. The URL points hardcoded to a FreeBSD resource. To enable installation via make on HardenedBSD the Makefile needs to be extended.

validate names helper

almost all of our name arguments have the same general format:

/^[a-z0-9][a-z0-9-_\.]*/i

given that many of our names are used as part of a path, we can further restrict the *

/^[a-z0-9][a-z0-9-_\.]{0,254}/i

to FreeBSD's NAME_MAX (255).

---

edit with conclusions from the comments

HOWEVER, until FreeBSD 12, fstab's mount paths are further restricted to 88, and not 255. Since we're supporting freebsd all the way down to 9.3, we should also limit our names.

As we've seen from experience from using UUIDs as names, 32 is a length that can stretch it. I suggest only using half of that (16)

/^[a-z0-9][a-z0-9-_\.]{0,15}/i

add ability to select which columns to list

currently, iocage list has a number of different formats --long, and --quick (and… well, default…)

i propose to simply add --column X,Y,Z or else, the same way openstack server list does it: -c x -c y -c z, so users (or completion scripts!) can slice exactly the information that they need.

Feature: Autostart

Libiocage should provide a service that allows to start jails at host boot.

Previous iocage versions used the boot (bool) configuration property to enable a jail for system boot. Boot priorities saved as priority jail configuration property should be obeyed.

Feature: Default config

Currently libiocage has hardcoded default config properties. In iocage it should be possible to define global defaults by executing iocage set foo=bar default.

By providing a full list of defaults, each jails configuration only needs to contain the delta. When initializing the defaults, the global defaults should be read and override the hardcoded ones.

Even though it does not have influence on the way user defined globals work, it seems cleaner to export the hardcoded values to a json file instead as well.

Use type-hints for all function arguments & return values

Since we're already using Python 3.6, we should consider also specifying all our input parameters with input & return types.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Feature: Templates

currently, libiocage doesn't implement Templates!

mostly, because, conceptually, it's really hard to grasp what they are, and how to implement them in a way that's intuitive, performant, and convenient.

@gronke and me discussed mapping it to the existing jail types, and it seems the most sensible thing would be to make them akin to NullFS BaseJails.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.