bsdci / libioc Goto Github PK
View Code? Open in Web Editor NEWA Python library to manage jails with ioc{age,ell}
Home Page: https://bsd.ci/libioc
License: Other
A Python library to manage jails with ioc{age,ell}
Home Page: https://bsd.ci/libioc
License: Other
iocage/iocage#347 was the original issue, iocage/iocage@54583ed commit that fixed it in iocage.
The manifest containing the release asset signatures is downloaded from the same untrusted http resource as the assets. To be an effective security feature we need to verify this signatures from trusted sources.
--verify/--no-verify
option--signatures base=62acaee7e...
or similar)Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
It is unsupported to run jails with releases newer than the host's major version. We should not offer to fetch those releases when prompting the user and properly warn when attempting to create from an unsupported release.
Proper warning messages are important and I'd love to hear ideas how to inform that there are newer releases available that are unsupported. Maybe "There are 2 more releases available newer than your host release"?
This is a non-commercial project and the credit belongs to the "iocage project", which is a group of independent contributors. We should put proper license information (BSD 2-clause) to every .py
file in the library.
See also: https://github.com/iocage/libiocage/blob/master/LICENSE.txt
ioc version is head
~ # ioc list -o jid -o name -o release
Traceback (most recent call last):
File "/usr/local/bin/ioc", line 12, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/root/libiocage/libiocage/cli/list.py", line 104, in cli
_print_table(jails, columns, header, _sort)
File "/root/libiocage/libiocage/cli/list.py", line 126, in _print_table
table_data.append(_lookup_jail_values(jail, columns))
File "/root/libiocage/libiocage/cli/list.py", line 176, in _lookup_jail_values
columns
File "/root/libiocage/libiocage/cli/list.py", line 175, in <lambda>
lambda column: jail.getstring(column),
File "/root/libiocage/libiocage/lib/Jail.py", line 926, in getstring
return libiocage.lib.helpers.to_string(self.__getattr__(key))
File "/root/libiocage/libiocage/lib/Jail.py", line 893, in __getattr__
return object.__getattribute__(self, key)
File "/root/libiocage/libiocage/lib/Jail.py", line 849, in release
name=self.config["release"],
File "/root/libiocage/libiocage/lib/JailConfig.py", line 567, in __getitem__
return self.defaults[key]
KeyError: 'release'
this error is specific to the release
column, since that has no defined default.
The current implementation lacks the ability to destroy jails.
ioc destroy <JAIL_GLOB>
destroys stopped jailsioc destroy -r <RELEASE_GLOB>
destroys a releasein working on #86 / #88, i stumbled across this piece of art in Resource.py:
@property
def dataset(self) -> libzfs.ZFSDataset:
"""
The jail's base ZFS dataset
"""
if self._dataset_name is not None:
# sets self._dataset_name to None and memoize the dataset
self._dataset = self.zfs.get_dataset(self.dataset_name)
return self._dataset
According to CONTRIBUTING.md imports should be ordered with isort.
Iocage implements a mechanism to automatically install jails from a template called "plugin". Various plugins are listed in the freenas plugin registry.
The way of installing packages and applying post-install hooks can be abstracted as discussed in iocage/iocage#163 about Ansible (or other provisioning tools) integration. Running a hook script read from a GitHub source would be sufficient to support the current plugin system.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
When creating a number of jails with --count
flag, names cannot be set, because the second creation attempt will fail because the jail with this name already exists.
It is required to introduce a wildcard character that represents the index of the currently created jail.
My suggestion is: ioc create --count 128 --name "my-%i-jail-of-%n"
where i
is the index and n
is the total number of jails to create.
When fetching releases EOL warnings should appear next to outdated releases.
Did you help close this issue? Go claim the $25 bounty on Bountysource.
Creating jail with iocage_legacy iocage create tag=myjail
and then starting this jail with ioc start UUID
creates dataset zpool/iocage/log
If using iocage list
after dataset created makes iocage legacy outputing a lot of errors
cannot open 'sys/iocage/jails/': invalid dataset name
cannot open 'sys/iocage/jails/': invalid dataset name
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
Error occured: cannot open file /iocage/jails//config: No such file or directory
cannot open 'sys/iocage/jails/': invalid dataset name
cannot open 'sys/iocage/jails/': invalid dataset name
cannot open 'sys/iocage/jails/': invalid dataset name
cannot open 'sys/iocage/jails/': invalid dataset name
ERROR: missing property or UUID/TAG!
Although it outputs list of jails after this errors as usual.
Destroying sys/iocage/jails manually fixes errors above but after running ioc start
again recreates dataset.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
root@zwei:/usr/local/src/libiocage # ioc fetch -r 10.3-RELEASE
[+] FetchRelease: OK [21.26s]: 4
[+] ReleasePrepareStorage: OK [0.201s]: 1
[+] ReleaseDownload: OK [3.355s]: 1
[+] ReleaseExtraction: OK [17.703s]: 1
[+] ReleaseConfiguration: OK [0.002s]: 1
[-] ReleaseUpdateDownload: ...
Command exited with 1: /zroot/iocage/releases/10.3-RELEASE/updates/freebsd-update.sh -d /zroot/iocage/releases/10.3-RELEASE/updates -f /zroot/iocage/releases/10.3-RELEASE/updates/freebsd-update.conf --not-running-from-cron fetch
Command exited with 1
Traceback (most recent call last):
File "/usr/local/bin/ioc", line 12, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/libiocage/cli/fetch.py", line 98, in cli
fetch_updates=fetch_updates
File "/usr/local/lib/python3.6/site-packages/libiocage/cli/__init__.py", line 64, in print_events
for event in generator:
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/Release.py", line 393, in fetch
for event in ReleaseGenerator.fetch_updates(self):
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/Release.py", line 526, in fetch_updates
], logger=self.logger)
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/helpers.py", line 139, in exec
logger=logger
libiocage.lib.errors.CommandFailure: Command exited with 1
on the current HEAD (3f184d9), when running make install
i get the following error:
vagrant@ ~/libiocage> sudo -H make install
python3.6 -m ensurepip
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages
Requirement already satisfied: pip in /usr/local/lib/python3.6/site-packages
pkg install -q -y libgit2
The most recent version of packages are already installed
pip3.6 install -U .
Processing /usr/home/vagrant/libiocage
Complete output from command python setup.py egg_info:
error in libiocage setup command: 'dependency_links' must be a list of strings (got <map object at 0x8080cdb00>)
fastentrypoints
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-esegjs7v-build/
*** Error code 1
Stop.
make: stopped in /usr/home/vagrant/libiocage
vagrant@ ~/libiocage>
Since we're already using Python 3.6, we should consider also specifying all our input parameters with input & return types.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
# python3.6 . -d spam stop myjail
[+] JailDestroy: OK [0.005s]: 1
[-] JailNetworkTeardown: ...
Traceback (most recent call last):
File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "./__main__.py", line 51, in <module>
main()
File "./__main__.py", line 45, in main
cli(prog_name="iocage")
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "./libiocage/cli/stop.py", line 57, in cli
ctx.parent.print_events(jail.stop(force=force))
File "./libiocage/cli/__init__.py", line 64, in print_events
for event in generator:
File "./libiocage/lib/Jail.py", line 377, in stop
self._stop_vimage_network()
File "./libiocage/lib/Jail.py", line 788, in _stop_vimage_network
for network in self.networks:
File "./libiocage/lib/Jail.py", line 1068, in __getattribute__
return object.__getattribute__(self, key)
File "./libiocage/lib/Jail.py", line 758, in networks
bridges = list(self.config["interfaces"][nic])
TypeError: string indices must be integers
allow_chflags:0
allow_mount:0
allow_mount_devfs:0
allow_mount_nullfs:0
allow_mount_procfs:0
allow_mount_tmpfs:0
allow_mount_zfs:0
allow_quotas:0
allow_raw_sockets:0
allow_set_hostname:1
allow_socket_af:0
allow_sysvipc:0
basejail:yes
boot:no
children_max:0
clonejail:yes
defaultrouter:-
defaultrouter6:-
devfs_ruleset:4
enforce_statfs:2
exec_clean:1
exec_fib:1
exec_poststart:/usr/bin/true
exec_poststop:/usr/bin/true
exec_prestart:/usr/bin/true
exec_prestop:/usr/bin/true
exec_start:/bin/sh /etc/rc
exec_stop:/bin/sh /etc/rc.shutdown
exec_timeout:60
host_domainname:-
id:myjail
interfaces:vnet0:bridge1
ip4:new
ip4_addr:-
ip4_saddrsel:1
ip6:new
ip6_saddrsel:1
jail_zfs:no
legacy:-
mac_prefix:02ff60
mount_devfs:1
mount_fdescfs:1
priority:0
release:11.1-RELEASE
resolver:/etc/resolv.conf
securelevel:2
stop_timeout:30
sysvmsg:new
sysvsem:new
sysvshm:new
tags:-
vnet:yes
Libiocage should provide a service that allows to start jails at host boot.
Previous iocage versions used the boot
(bool) configuration property to enable a jail for system boot. Boot priorities saved as priority
jail configuration property should be obeyed.
...
A cddl/boot/zfs
A cddl/dev/fbt/powerpc/fbt_isa.c
A cddl/dev/fbt/fbt.c
A cddl/dev/fbt/powerpc/fbt_isa.h
A cddl/dev/fbt/riscv/fbt_isa.h
A cddl/dev/fbt/aarch64/fbt_isa.h
A cddl/contrib/opensolaris/common/zfs/zpool_prop.c
A cddl/contrib/opensolaris/common/zfs/zfs_namecheck.h
A cddl/dev/fbt/x86/fbt_isa.c
A cddl/dev/fbt/mips/fbt_isa.c
A cddl/contrib/opensolaris/common/zfs/zfs_prop.h
A cddl/contrib/opensolaris/common/atomic/amd64/opensolaris_atomic.S
A cddl/contrib/opensolaris/common/avl/avl.c
A cddl/dev/dtrace/powerpc/dtrace_isa.c
A cddl/dev/dtrace/dtrace_debug.c
A cddl/dev/dtrace/amd64/instr_size.c
A cddl/dev/dtrace/i386/instr_size.c
A cddl/dev/dtrace/dtrace_load.c
A cddl/dev/dtrace/aarch64/dtrace_subr.c
A cddl/dev/dtrace/riscv/dtrace_subr.c
A cddl/dev/dtrace/mips/dtrace_isa.c
A cddl/dev/dtrace/arm/dtrace_asm.S
A cddl/dev/dtrace/dtrace_hacks.c
A cddl/dev/dtrace/dtrace_modevent.c
A cddl/dev/fbt/arm/fbt_isa.c
A cddl/dev/fbt/x86/fbt_isa.h
A cddl/dev/fbt/mips/fbt_isa.h
A cddl/dev/systrace/systrace.c
A cddl/dev/prototype.c
A cddl/boot/zfs/blkptr.c
A cddl/boot/zfs/lzjb.c
A cddl/dev/dtrace/dtrace_vtime.c
A cddl/dev/fbt/arm/fbt_isa.h
A cddl/dev/fbt/riscv/fbt_isa.c
A cddl/dev/fbt/aarch64/fbt_isa.c
A cddl/dev/sdt/sdt.c
A cddl/dev/fbt/fbt.h
A cddl/dev/dtmalloc/dtmalloc.c
A cddl/dev/profile/profile.c
A cddl/boot/zfs/lz4.c
A cddl/boot/zfs/README
A cddl/boot/zfs/zle.c
A cddl/boot/zfs/zfssubr.c
A cddl/boot/zfs/sha256.c
A cddl/boot/zfs/zfsimpl.h
A cddl/boot/zfs/fletcher.c
Checked out revision 323616.
Summary of conflicts:
Tree conflicts: 4
Tree conflict on 'cddl/boot'
> local dir unversioned, incoming dir add upon update
Select: (r) mark resolved, (p) postpone, (q) quit resolution, (h) help: q
@igalic mentioned in a PR review that we should consider using --libxo=json
to get parsable output of jls.
Personally I would prefer to get the jail state from a python library, but until then the formatted output is a clean alternative to the way we currently parse the output.
almost all of our name
arguments have the same general format:
/^[a-z0-9][a-z0-9-_\.]*/i
given that many of our names are used as part of a path, we can further restrict the *
/^[a-z0-9][a-z0-9-_\.]{0,254}/i
to FreeBSD's NAME_MAX
(255).
edit with conclusions from the comments
HOWEVER, until FreeBSD 12, fstab
's mount paths are further restricted to 88, and not 255. Since we're supporting freebsd all the way down to 9.3, we should also limit our names.
As we've seen from experience from using UUIDs as names, 32 is a length that can stretch it. I suggest only using half of that (16)
/^[a-z0-9][a-z0-9-_\.]{0,15}/i
As discussed through IRC adding option to use variables on specific attributes that are resolved on runtime would make using exec.pre/post_start/stop a lot easier. Native jail with jail.conf supports this out of the box (jail.conf, but I think it's not possible via cmd line using jail -c ...
A workaround I did on legacy jail was to add jail name while calling exec scripts urosgruber/iocage@ca766a7
Example of an old script we used post_start script that is using jail name as an argument but could be improved by providing JID or some other parameter
#!/bin/sh
# get current jid
_jid=`jls -j $1 jid 2>/dev/null`
_ip=`jls -j $1 ip4.addr 2>/dev/null`
_name=$1
# cpuset
# cpuset -c -l 0 -j ${_jid}
# mem limit
rctl -a jail:$1:memoryuse:deny=256M
# pf
( pfctl -a jails -sr ; echo "pass on lo0 from ${_ip} to ${_ip}" ) | pfctl -a jails -f -
# update started time and status
mysql -ss -D db -e "UPDATE service SET started=NOW(), status='running', jid=${_jid} WHERE name='$1';" 2> /dev/null
Similarly for post_stop script and pre_ scripts.
Nice to have would be to specify a folder pre_start.d where all executable files would be executed with same parameters. example: ioc set exec_prestart="/my/prestart.d/ ${jid}"
Did you help close this issue? Go claim the $50 bounty on Bountysource.
We have a handful of functions that take non-None
arguments, and then modify them.
This leads to python directly modifying the passed argument, and is generally considered bad form.
One such example is JailConfigAddresses#add()
def add(self, nic, addresses=[], notify=True):
if isinstance(addresses, str):
addresses = [addresses]
maybe there are linter params that will warn us of such code, until then we might need human eyes.
@rexpo opened this on iocage/iocage#345
iocage is activated on one (for example small, SSD-backed) zpool, and I want to be able to mount datasets from a second (for example massive, but slow HDD-backed) zpool.
This isn't currently possible, as iocage can only mount a dataset from the same pool that iocage is activated on.
I know that I should just shut up, activate the big slow pool, throw lots of RAM at the host, and take comfort in the fact that the ZFS ARC will eventually speed things up. There are, however, other use cases that benefit from the ability to mount a dataset from another pool, especially when it comes to maintenance and migration.
As far as I can tell, work was done on iocage_legacy to implement just such a feature, but that it never made it to prime time. I'm sure this has already been discussed before, but I couldn't find any closed issues about it. Has this already been considered?
iocage --version
Using ioc start UUID
on basejail type of legacy jail just return with no info also jail is not running. Using same command on clonejail type starts the jail but with some errors
Command exited with 1: /usr/sbin/jexec ioc-9048b2dc-91ac-11e7-b8c6-7071bca7c826 /bin/sh /etc/rc
jexec: execvp: /bin/sh: No such file or directory
Command exited with 1
I managed to get it work for a while but I was unable to replicate fully working start with latest master.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
I think iocage dataset is what is currently hardcoded, right? It would be great if iocage activate
would allow to specify base dataset other than iocage. This would allow easier migration from legacy jails. Right now only single dataset can exist and after installing new iocage old legacy jails will not properly start/stop.
I created iocage_legacy port with renamed commands but I'm not sure how to get around this dataset naming clash.
on libiocage#HEAD, ioc list -r
should print the releases,
root@container-host1 ~/libiocage# ioc list -r
+-----+------+---------+---------+----------+
| JID | NAME | RUNNING | RELEASE | IP4.ADDR |
+=====+======+=========+=========+==========+
+-----+------+---------+---------+----------+
however, it prints the same thing as ioc list
root@container-host1 ~/libiocage# ioc list
+-----+------+---------+---------+----------+
| JID | NAME | RUNNING | RELEASE | IP4.ADDR |
+=====+======+=========+=========+==========+
+-----+------+---------+---------+----------+
start: Little output, should look the same. JID can be included in the same style.
stop: Little output, should look the same.
There are likely more, but just creating this as a placeholder so when this becomes iocage, the user will not notice a difference. This is important for scripting and other things we aren't necessarily privy of.
Currently libiocage has hardcoded default config properties. In iocage it should be possible to define global defaults by executing iocage set foo=bar default
.
By providing a full list of defaults, each jails configuration only needs to contain the delta. When initializing the defaults, the global defaults should be read and override the hardcoded ones.
Even though it does not have influence on the way user defined globals work, it seems cleaner to export the hardcoded values to a json file instead as well.
The makefile downloads sources (if they are not available yet) for the current host distribution. The URL points hardcoded to a FreeBSD resource. To enable installation via make on HardenedBSD the Makefile needs to be extended.
Jails should be allowed to set ip4_addr=vnet0|dhcp
to obtain an IPv4 address via DHCP on the target interface vnet0
.
DHCP requires access to /dev/bpf, which needs to be granted via udev rule on the host.
An acceptable solution:
dhcp
is set on any interface in ip4_addr
dhcp
is not set on any interface in ip4_addr
Using latest master ioc create
creates jail half way with and error
9f24cf51 could not be created!
Traceback (most recent call last):
File "/usr/local/bin/ioc", line 12, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/libiocage/cli/create.py", line 156, in cli
jail.create(release.name)
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/Jail.py", line 402, in create
self.config.save()
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 209, in save
self.jail.rc_conf.save()
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/Jail.py", line 162, in rc_conf
logger=self.logger
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/RCConf.py", line 41, in __init__
self.path = path
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/RCConf.py", line 52, in path
self._read_file()
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/RCConf.py", line 63, in _read_file
new_keys = set(data.keys())
UnboundLocalError: local variable 'data' referenced before assignment
But ioc list
show newly created jail
+-----+------------+--------------------------------------+---------+--------------+----------------+-----------+--------+
| JID | TAG | NAME | RUNNING | RELEASE | IP4.ADDR | TYPE | LEGACY |
+=====+============+======================================+=========+==============+================+===========+========+
| - | - | 9f24cf51-9ae4-4f28-88f8-a71dabadcf39 | no | 11.1-RELEASE | - | - | no |
+-----+------------+--------------------------------------+---------+--------------+----------------+-----------+--------+
And then ioc get all 9f24cf51-9ae4-4f28-88f8-a71dabadcf39
throws exception as well
basejail:no
Traceback (most recent call last):
File "/usr/local/bin/ioc", line 12, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/libiocage/cli/get.py", line 95, in cli
value = jail.config.get_string(key)
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 504, in get_string
return self.stringify(self.__getitem__(key))
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 546, in __getitem__
return self.defaults[key]
KeyError: 'clonejail'
And funny thing, jail can be started with ioc start 9f24cf51-9ae4-4f28-88f8-a71dabadcf39
Note: activate was done by iocage legacy.
Libiocage is not able to fetch and update HardenedBSD releases yet.
currently, libiocage doesn't implement Templates!
mostly, because, conceptually, it's really hard to grasp what they are, and how to implement them in a way that's intuitive, performant, and convenient.
@gronke and me discussed mapping it to the existing jail types, and it seems the most sensible thing would be to make them akin to NullFS BaseJails.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Jails were undergoing some fundamental changes towards Resources. The Storage class should be refactored and utilize new classes and helpers, specially ZFS.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
In previous versions of iocage it was possible to set resource limits on jails.
root@localhost:/usr/local/src/libiocage # ioc list
...
File "/usr/local/bin/ioc", line 12, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/libiocage/cli/get.py", line 95, in cli
value = jail.config["get_string"](key)
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 491, in get_string
return self.__getitem__(key, string=True)
File "/usr/local/lib/python3.6/site-packages/libiocage/lib/JailConfig.py", line 528, in __getitem__
raise KeyError(f"Config variable {key} not found")
KeyError: 'Config variable clonejail not found'
Doing a fresh make
gives
python3.6 -m ensurepip
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages
Requirement already satisfied: pip in /usr/local/lib/python3.6/site-packages
pkg install -q -y libgit2 libucl cython3
The most recent version of packages are already installed
pip3.6 install -U .
make: exec(pip3.6) failed (No such file or directory)
*** Error code 1
Stop.
Renaming pip3.6
to pip-3.6
fixes make process. I'm not sure if something is installed on a system differently but I installed pip through pkg.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
The output of TravisCI build logs leaves a hint that there are 67 code style issues, but does not print them or cause a failed build status.
From the Travis CI build logs:
...
$ cd iocage/libiocage
0.41s$ git fetch origin +refs/pull/33/merge:
remote: Counting objects: 45, done.
remote: Compressing objects: 100% (15/15), done.
Unpacking objects: 100% (45/45), done.
remote: Total 45 (delta 30), reused 44 (delta 30), pack-reused 0
From https://github.com/iocage/libiocage
* branch refs/pull/33/merge -> FETCH_HEAD
$ git checkout -qf FETCH_HEAD
0.01s$ source ~/virtualenv/python3.6/bin/activate
$ python --version
Python 3.6.2
$ pip --version
pip 9.0.1 from /home/travis/virtualenv/python3.6.2/lib/python3.6/site-packages (python 3.6)
install
1.16s$ pip3 install flake8
Collecting flake8
Downloading flake8-3.4.1-py2.py3-none-any.whl (68kB)
100% |████████████████████████████████| 71kB 2.6MB/s
Collecting pyflakes<1.6.0,>=1.5.0 (from flake8)
Downloading pyflakes-1.5.0-py2.py3-none-any.whl (225kB)
100% |████████████████████████████████| 225kB 3.5MB/s
Collecting mccabe<0.7.0,>=0.6.0 (from flake8)
Downloading mccabe-0.6.1-py2.py3-none-any.whl
Collecting pycodestyle<2.4.0,>=2.0.0 (from flake8)
Downloading pycodestyle-2.3.1-py2.py3-none-any.whl (45kB)
100% |████████████████████████████████| 51kB 7.8MB/s
Installing collected packages: pyflakes, mccabe, pycodestyle, flake8
Successfully installed flake8-3.4.1 mccabe-0.6.1 pycodestyle-2.3.1 pyflakes-1.5.0
4.69s$ .travis/flake8.sh
Current Error Count: 67
usage: git rev-list [OPTION] <commit-id>... [ -- paths... ]
limiting output:
--max-count=<n>
--max-age=<epoch>
--min-age=<epoch>
--sparse
--no-merges
--min-parents=<n>
--no-min-parents
--max-parents=<n>
--no-max-parents
--remove-empty
--all
--branches
--tags
--remotes
--stdin
--quiet
ordering output:
--topo-order
--date-order
--reverse
formatting output:
--parents
--children
--objects | --objects-edge
--unpacked
--header | --pretty
--abbrev=<n> | --no-abbrev
--abbrev-commit
--left-right
--count
special purpose:
--bisect
--bisect-vars
--bisect-all
fatal: No names found, cannot describe anything.
Comparing with last stable release:
's Error Count: 67
The command ".travis/flake8.sh" exited with 0.
Done. Your build exited with 0.
In addition to a resource name, jails may have one or more tags that makes them selectable.
The general functionality to provide tags is embedded in libiocage, but we need to expose the feature via the command line interface to make it usable.
For compatibility with previous versions the tags should be stored as comma separated string. Another thing to consider is the previously used jail tag
configuration option which should be seamlessly merged into the tags when it exists.
tags
can be set from a comma separated stringtags
can be set from a list of stringstags
property was found in a jail's configuration
tag
JailConfig item sets the deprecated tag
propertytags
removed any existing tag
property from the configurationtags
are already existing
tag
that already exists in tags
makes it the first item in tags
tag
that does not exist in tags
yet, adds it as first itemDuring jail start we write to a bunch of files within a jail's root dataset. When those assets are a symlink, the host system will write to the symlink target instead of a (safe) file only in the jails dataset.
Due to the nature of these helpers, the attacker would not be controlling the full content of the files, even partial control over the file content might allow to compromise the host (e.g. .ssh/authorized_keys
will be read even if a few lines are broken)
This was discovered by @igalic in #72 (comment)
We introduced code checking with flake8 and mypy, but did not document how to configure and use the development environment.
At least developers need to know a list of dependencies and some helper to install them on FreeBSD and HardenedBSD.
make travis
runs the same tasks as they would on TravisCImake test
runs the code style checks and unit testsmake dev-dependencies
installs developer dependencies (and general dependencies)make install
installs libiocage to the systemmake dev-dependencies
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Introduced with #37 there will be signaling between the library and the cli, which is responsible for printing event to the user. All events are listed in libiocage/lib/events.py
. Another list of event-like items is libiocage/lib/errors.py
. Both are inviting for internationalization!
To generate similar output to previous iocage versions, we should begin with adding translations for English language.
We may wait with translation iocage to further languages until we come close to a stable release, so that we don't have update translations among changing interfaces.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
When a release that was already copied to the collection of base datasets (/iocage/base/<RELEASE>
), the datasets representing the datasets need to get updated as well.
I wonder what the most efficient way to do that is. Here are some options:
+ does not need to write/touch all files
- HardenedBSD currently does not cache assets on the host (yet)
- It it another source of errors that require our action (e.g. roll back changes from the previous release update, so that the release and the base do not diverge)
+ simple
- slow
- reads and writes back from disk
- errors like full disks may still require our action to handle rollbacks
Suggestions welcome
FreeBSD allows to define resource limits for jails, as seen in the documentation:
https://www.freebsd.org/doc/handbook/security-resourcelimits.html
See also:
limit_memoryuse
optionlimit_cputime
optionlimit_<RESOURCE>
according to the rctl man page RESOURCES sectionA jails name also defines the jails dataset location relative to the activated zpool. That's why it's important to do more than just renaming the property. Certainly we must force a jail to be stopped for renaming it.
host_hostname
when it was manually set (available in config.data) and matches the old jail nameUsers should have the ability to abort the process at any time.
In case of half-finished progress steps, we should warn and provide a way to cleanup. For example an aborted stop-command should say "Stopping jail 'ABC' was not complete. Please run 'ioc stop ABC --force' to ensure the jail was stopped."
currently, iocage list has a number of different formats --long
, and --quick
(and… well, default…)
i propose to simply add --column X,Y,Z
or else, the same way openstack server list
does it: -c x -c y -c z
, so users (or completion scripts!) can slice exactly the information that they need.
Currently the order of imports influences the ability to load modules and classes. This becomes a problem when using mypy to do strict type checking.
The classes should be refactored, so that each class may be loaded individually and order independent.
As @igalic noted in #83 (comment) the library returns None in functions that do not return an optional type.
def something() -> str:
# ....
return None
should be instead
import typing
def something() -> typing.Optional[str]:
# ...
return None
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.