canonical / operator-libs-linux

Linux helper libraries for the operator framework

Linux Libraries for Operator Framework Charms

Description

The operator-libs-linux charm provides a set of charm libraries which can be used for managing and manipulating Debian packages, snap packages, system repositories, users and groups, and other operations that charm authors may need to carry out when authoring machine charms.

This charm is not meant to be deployed itself, and is used as a mechanism for hosting libraries only.

Usage

Each library contains information on usage and code examples. They are meant to be complete as standalone libraries, and should be managed as charm libraries, with installation via fetch-lib, after which they may be imported and used as normal charms.

• apt - a library that enables the installation of Debian packages and management of system package repositories.
• snap - a library for installing and working with snap packages.
• systemd - a library for manipulating services managed by the systemd init system.
• passwd - a library for manipulating Linux users and groups.
• grub - a library for managing Linux kernel configuration via GRUB.
• sysctl - a library for managing kernel parameters at runtime via sysctl.
• juju-systemd-notices - a library for utilizing systemd to observe and emit notices when services change state.

Contributing

Please see the Juju SDK docs for guidelines on enhancements to this charm following best practice guidelines, and CONTRIBUTING.md for developer guidance.

operator-libs-linux's Issues

Snap Get - Error on no value set

Problem

When performing snap get:

        snap_cache = snap.SnapCache()
        mongodb_snap = snap_cache["charmed-mongodb"]
        current_uri = mongodb_snap.get("monitor-uri")

this will cause an error if monitor-uri is not set

Ideal usage

        snap_cache = snap.SnapCache()
        mongodb_snap = snap_cache["charmed-mongodb"]
        if mongodb_snap.get("monitor-uri") == self.monitor_config.uri:
             return 
       
        # do something

Current usage

        snap_cache = snap.SnapCache()
        mongodb_snap = snap_cache["charmed-mongodb"]
        try:
            if mongodb_snap.get("monitor-uri") == self.monitor_config.uri:
                return
        except snap.SnapError:
            # SnapError occurs when the config option `monitor-uri` is not set.
            pass
 
        # do something

Desired Behavior

Return None if value is not set

DebianRepository.import_key() should be a staticmethod so it can be called without a DebianRepository instance

My usecase is, that I want to import a GPG key for apt (eg. https://repos.influxdata.com/influxdata-archive_compat.key). For that, I only need that file and a filename how this key should be named. But that is currently not possible because I first need to create a DebianRepository instance which requires a couple of arguments which are not needed at all for my usecase.
So what I would like to do is:

key = """
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBGPIEycBEACpG4qSjhxA6fh4QJVJxFVBvCFt9tVx/hDbKH0Ryy9iilyMeReC
...
-----END PGP PUBLIC KEY BLOCK-----
"""
DebianRepository.import_key(key)

snap.ensure raises SnapError if channel is defined but confinement isn't

snap.ensure(snap.SnapState.Latest, channel='edge')
fails with:

charms.operator_libs_linux.v1.snap.SnapError: ('Could not %s snap [%s]: %s', ['snap', 'install', 'microsample', '', '--channel="edge"'], 'microsample', '')

on install hook.
note the empty string passed as optargs to _snap; that is the confinement.
Proposed solution: before _install calls _snap, we should filter out the '' from the args list.

Have a branch fixing this bug, will open a PR as soon as I have time.

`snap.set` should correctly handle nested dicts (or raise an exception)

Currently, snap.set({"foo": {"bar": "baz"}}) will result in the snap being configured with
foo = '{"bar": "baz"}' (a repr-dump of the dict object, as a string).

The snap lib should provide means to format this correctly, i.e. something like test.charms.test_main.lib.ops.model._format_action_result_dict: a flattened dict using dots as key separator.

Ideally, snap.set would iterate through the values, attempting to flatten any nested dicts it find, and raise an exception if this fails or if any value is not a string or a dict.

`apt.add_package` hangs when apt encounters a user prompt

for instance... if apt prompts for the users' response -- this method blocks forever and never returns

See this [commit][https://github.com/juju/charm-helpers/commit/ef8a1febc4430239654b11f5de53e0700b82eb8c] from charmhelpers which uses --assume-yes to make the apt-get command non-interactive

snap: `install_local()` fails for some kwarg combinations

The module level function install_local() is failing to install the snap file when given dangerous=True only. This is due to an empty string being passed as an argument to the underlying subprocess.check_output() call, which the snap process does not really appreciate.

operator-libs-linux/lib/charms/operator_libs_linux/v1/snap.py

Lines 900 to 909 in 9d197d7

	_cmd = [
	"snap",
	"install",
	filename,
	"--classic" if classic else "",
	"--dangerous" if dangerous else "",
	]
	try:
	result = subprocess.check_output(_cmd, universal_newlines=True).splitlines()[0]
	snap_name, _ = result.split(" ", 1)

Here is a similar scenario:

miles@t440:/tmp$ python3
Python 3.8.10 (default, Jun 22 2022, 20:18:18) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import check_output
>>> out = check_output(["sudo", "snap", "install", "/tmp/curl_1093.snap", "", "--dangerous"])
error: cannot install snap with empty name
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sudo', 'snap', 'install', '/tmp/curl_1093.snap', '', '--dangerous']' returned non-zero exit status 1.

Integration tests are very inconsistent

There are a few integration tests that often fail, but eventually pass. We should look into this -- why the inconsistency? Should they be passing or failing?

Support setting multiple snap config keys at once

With charm-lxd, our ad-hoc snap set function supports setting multiple config keys at once effectively calling:

snap set lxd ceph.external=true criu.enable=true

It would be nice for the snap implementation to grow support for this.

Should `snap.hold_refresh` change it's functionality to allow for infinite holds?

Currently setting snap.hold_refresh(days=0) will remove the hold on the snaps. What value would be reasonable to allow for indefinite holds?

snap ensure hangs

Problem

On occasion running snap_package.ensure(snap.SnapState.Latest, channel=snap_channel), this line of code will hang for anywhere between several minutes to several hours before erroring out.

Context

This behaviour has not been experienced on the commandline only with this library by both @MiaAltieri and @taurus-forever

After looking into the code it appears to be an issue with check_output failing but no useful error message is given:

unit-mongodb-2: 16:12:54 ERROR unit.mongodb/2.juju-log An exception occurred when installing percona-backup-mongodb. Reason: Snap: 'percona-backup-mongodb'; command ['snap', 'install', 'percona-backup-mongodb', '--channel="edge"'] failed with output = ''

Proposed Solution

Add some sort of timeout+retry for ensure that the snap can be installed in a timely manner.

`apt.add_package` ignores packages that are reported by `dpkg -l`, but not really installed.

library: charms.operator-libs-linux.v0.apt
version: current main, hash c2e2f851daa62870239ae82faf40bfd0b7bee296

A package that shows up in the output of dpkg -l is considered installed, even though it might not be. An example would be a situation when a package was removed, but not purged, and left config files behind:

root@juju-cf280c-xenial-0:~# dpkg -l ubuntu-advantage-tools
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                              Version               Architecture          Description
+++-=================================-=====================-=====================-========================================================================
rc  ubuntu-advantage-tools            27.2.2~16.04.1        amd64                 management tools for Ubuntu Advantage

The code uses a regex to process this output, and it does extract a package_status [0] (which would be "rc" in this case), but it is not used. The library should check this field, note that the package is not really installed, and deal with the situation appropriately. Currently it is essentially a no-op as it considers the package already installed, even though it is not.

[0] https://github.com/canonical/operator-libs-linux/blob/main/lib/charms/operator_libs_linux/v0/apt.py#L413

[Systemd] support detection of virt substrate

Can we add support for systemd-detect-virt on our systemd lib?

That is important, e.g. when we want to use sysctl lib and some of the options may be applicable, but others cannot.

Proposal:

import subprocess

...

def running_as_vm():
    try:
        return subprocess.run(["systemd-detect-virt", "--vm"]).returncode == 0
    except FileNotFoundError:
        # No systemd! Either this is a docker container OR a very old distro
        return False

def running_as_lxc():
    try:
        return subprocess.run(["systemd-detect-virt", "--container"]).returncode == 0
    except FileNotFoundError:
        # No systemd! Either this is a docker container OR a very old distro
        return False

def running_as_container():
    try:
        return subprocess.run(["systemd-detect-virt", "--container"]).returncode == 0
    except FileNotFoundError:
        return True

Example charms for integration tests

Once some of the extant PRs are merged, we should consider creating a set of integration tests with a simple charm to ensure that the libraries can be imported, and function as necessary. We probably need just one test charm at the moment that does the following:

• Adds an apt repo and installs a package from it (ideally one with a GPG key, perhaps the Hashicorp deb repo and install Terraform?)
• Installs a simple snap package (like kubectl?)
• Creates and user and group
• Deletes a user and group

These tests should be run on PR/push using the pytest-operator/actions-operator and GH Actions.

apt-get should be called with DEBIAN_FRONTEND=noninteractive

If a hook triggers the installation of an apt package which prompts the user for input, it will remain stuck.
To avoid this, the command should be run with DEBIAN_FRONTEND=noninteractive.

Tidy up docstrings

In most of the libs, specifically the snap lib, the conventions around docstrings and formatting have become disjointed. What made me notice this particularly was around how optional args are annotated in docstrings.

In general, I'd prefer not to include optionality/defaults in the Args: section where that info is already available in the function signature with type annotations, but if we're going to do it, it should at least be consistent.

Triggered from a review in #88

Docstring for systemd method is incorrect

While I was working with the systemd charm library in one of my charms, I noticed that the docstring for service_start is incorrect:

operator-libs-linux/lib/charms/operator_libs_linux/v1/systemd.py

Lines 139 to 145 in 9d197d7

	def service_start(service_name: str) -> bool:
	"""Start a system service.
	Args:
	service_name: the name of the service to stop
	"""
	return _systemctl("start", service_name)

It should be service_name: the name of the service to start instead of service_name: the name of the service to stop. This might confuse someone who is working with the library for the first time and reading the docstrings for more information.

service_resume doesn't raise Exceptions on failure returns false instead

Hi! I'm using service_resume for my current development of a charm and I noticed that service_resume returns True when it succeeds and False otherwise. Would it be possible to implement some exceptions, it would be helpful to know when/why it fails to resume a service. I'm also happy to help implement this. @petevg

RFE: extend the snap module to provide more info eg. on available tracks / risk levels and revisions

The snap module provides a convenient way to interact with snaps, however it's limited in the information it provides from the snap store.

E.g. you can query the SnapCache for a snap name but afaict it will only give you info for the default track and risk level

c = snap.SnapCache()
microceph_snap = c["microceph"]
print(mc)
<Snap: microceph-817.reef/stable -- SnapState.Available>

It would be great to get more info on

• Available tracks
• Risk levels and revisions
• Versions
• Available archs

snap charmlib's `install_local` tries to unpack the wrong line after installing package

I get this error when I use the install_local method in the snap charm library:

Traceback (most recent call last):
  File "/root/./install", line 126, in <module>
    holder._run()
  File "/root/./install", line 87, in _run
    self._handle_snap_install()
  File "/root/./install", line 118, in _handle_snap_install
    snap.install_local(
  File "/usr/local/lib/python3.10/dist-packages/cleantest/pkg/handler/snap_handler.py", line 798, in install_local
    snap_name, _ = result.split(" ", 1)
ValueError: not enough values to unpack (expected 2, got 1)

The the local snap file is successfully installed, but it fails to return the Snap wrapper class. Upon further inspection, line 910 seems to be the culprit:

operator-libs-linux/lib/charms/operator_libs_linux/v1/snap.py

Lines 909 to 917 in 69f7f59

	try:
	result = subprocess.check_output(_cmd, universal_newlines=True).splitlines()[0]
	snap_name, _ = result.split(" ", 1)
	c = SnapCache()
	return c[snap_name]
	except CalledProcessError as e:
	raise SnapError("Could not install snap {}: {}".format(filename, e.output))

Line 910 sets the value of result to an empty string, which causes the unpack to fail on line 911. This is because the first line captured by check_output is a blank line. I can tell that we are trying to retrieve the snap name from the captured output, so the index should be changed to -1. This will pull the last line that contains the snap name and version. It looks like check_output is throwing a wrench in the works because even though snap install ... is printing to the same line in stdout, check_output is registering each emission as a newline of captured output.

Fix flakey integration tests due to logs mismatch

Currently the integration tests sometimes (or always?) fail with the following error. Haven't had a chance to look in in more detail, but we probably shouldn't be testing the logs so closely.

________________________________ test_snap_logs ________________________________
Traceback (most recent call last):
  File "/ops-libs-test/tests/integration/test_snap.py", line 138, in test_snap_logs
    assert len(kp.logs(num_lines=15).splitlines()) == 15
AssertionError: assert 12 == 15
 +  where 12 = len(['2023-01-23T04:15:24Z systemd[1]: Started Service for snap application kube-proxy.daemon.', '2023-01-23T04:15:25Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...', '2023-01-23T04:15:25Z systemd[1]: snap.kube-proxy.daemon.service: Succeeded.', '2023-01-23T04:15:25Z systemd[1]: Stopped Service for snap application kube-proxy.daemon.', '2023-01-23T04:15:26Z systemd[1]: Started Service for snap application kube-proxy.daemon.', '2023-01-23T04:15:26Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...', ...])
 +    where ['2023-01-23T04:15:24Z systemd[1]: Started Service for snap application kube-proxy.daemon.', '2023-01-23T04:15:25Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...', '2023-01-23T04:15:25Z systemd[1]: snap.kube-proxy.daemon.service: Succeeded.', '2023-01-23T04:15:25Z systemd[1]: Stopped Service for snap application kube-proxy.daemon.', '2023-01-23T04:15:26Z systemd[1]: Started Service for snap application kube-proxy.daemon.', '2023-01-23T04:15:26Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...', ...] = <built-in method splitlines of str object at 0x12f1450>()
 +      where <built-in method splitlines of str object at 0x12f1450> = '2023-01-23T04:15:24Z systemd[1]: Started Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:25Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...\n2023-01-23T04:15:25Z systemd[1]: snap.kube-proxy.daemon.service: Succeeded.\n2023-01-23T04:15:25Z systemd[1]: Stopped Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:26Z systemd[1]: Started Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:26Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...\n2023-01-23T04:15:26Z systemd[1]: snap.kube-proxy.daemon.service: Succeeded.\n2023-01-23T04:15:26Z systemd[1]: Stopped Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:26Z systemd[1]: Started Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:27Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...\n2023-01-23T04:15:27Z systemd[1]: snap.kube-proxy.daemon.service: Succeeded.\n2023-01-23T04:15:27Z systemd[1]: Stopped Service for snap application kube-proxy.daemon.\n'.splitlines
 +        where '2023-01-23T04:15:24Z systemd[1]: Started Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:25Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...\n2023-01-23T04:15:25Z systemd[1]: snap.kube-proxy.daemon.service: Succeeded.\n2023-01-23T04:15:25Z systemd[1]: Stopped Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:26Z systemd[1]: Started Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:26Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...\n2023-01-23T04:15:26Z systemd[1]: snap.kube-proxy.daemon.service: Succeeded.\n2023-01-23T04:15:26Z systemd[1]: Stopped Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:26Z systemd[1]: Started Service for snap application kube-proxy.daemon.\n2023-01-23T04:15:27Z systemd[1]: Stopping Service for snap application kube-proxy.daemon...\n2023-01-23T04:15:27Z systemd[1]: snap.kube-proxy.daemon.service: Succeeded.\n2023-01-23T04:15:27Z systemd[1]: Stopped Service for snap application kube-proxy.daemon.\n' = <bound method Snap.logs of <charms.operator_libs_linux.v1.snap.Snap: {'_name': 'kube-proxy', '_state': <SnapState.Latest: 'latest'>, '_channel': 'latest/stable', '_revision': '2900', '_confinement': 'classic', '_cohort': '', '_apps': [{'snap': 'kube-proxy', 'name': 'daemon', 'daemon': 'simple', 'daemon-scope': 'system'}], '_snap_client': <charms.operator_libs_linux.v1.snap.SnapClient object at 0x7f1459031460>}>>(num_lines=15)
 +          where <bound method Snap.logs of <charms.operator_libs_linux.v1.snap.Snap: {'_name': 'kube-proxy', '_state': <SnapState.Latest: 'latest'>, '_channel': 'latest/stable', '_revision': '2900', '_confinement': 'classic', '_cohort': '', '_apps': [{'snap': 'kube-proxy', 'name': 'daemon', 'daemon': 'simple', 'daemon-scope': 'system'}], '_snap_client': <charms.operator_libs_linux.v1.snap.SnapClient object at 0x7f1459031460>}>> = <charms.operator_libs_linux.v1.snap.Snap: {'_name': 'kube-proxy', '_state': <SnapState.Latest: 'latest'>, '_channel': 'latest/stable', '_revision': '2900', '_confinement': 'classic', '_cohort': '', '_apps': [{'snap': 'kube-proxy', 'name': 'daemon', 'daemon': 'simple', 'daemon-scope': 'system'}], '_snap_client': <charms.operator_libs_linux.v1.snap.SnapClient object at 0x7f1459031460>}>.logs

Improve unit test coverage of libraries

While unit test coverage of most python source files is near 100% according to coverage, v0/apt.py and v2/snap.py are at 79% and 85% respectively. It would be nice to go through and make sure that all the interesting code is covered. I'll open a PR for v2/snap.py in the near future.

SnapCache does not handle dev installed snaps

I do not know if it's by design but SnapCache does not handle snaps when they are installed from a .snap file.

unit-openstack-hypervisor-0: 10:50:21 ERROR unit.openstack-hypervisor/0.juju-log certificates:4: Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-openstack-hypervisor-0/charm/venv/ops_sunbeam/guard.py", line 91, in guard
    yield
  File "/var/lib/juju/agents/unit-openstack-hypervisor-0/charm/venv/ops_sunbeam/charm.py", line 267, in configure_charm
    self.configure_unit(event)
  File "/var/lib/juju/agents/unit-openstack-hypervisor-0/charm/./src/charm.py", line 181, in configure_unit
    self.ensure_snap_present()
  File "/var/lib/juju/agents/unit-openstack-hypervisor-0/charm/./src/charm.py", line 168, in ensure_snap_present
    cache = snap.SnapCache()
  File "/var/lib/juju/agents/unit-openstack-hypervisor-0/charm/lib/charms/operator_libs_linux/v1/snap.py", line 774, in __init__
    self._load_installed_snaps()
  File "/var/lib/juju/agents/unit-openstack-hypervisor-0/charm/lib/charms/operator_libs_linux/v1/snap.py", line 831, in _load_installed_snaps
    revision=int(i["revision"]),
ValueError: invalid literal for int() with base 10: 'x1'

https://github.com/canonical/operator-libs-linux/blob/8f1b16899d12acec9d8cc876a1eca7731e802f17/lib/charms/operator_libs_linux/v1/snap.py#LL831C21-L831C21

Function visibility in systemd library

This didn't strike me until I saw the library page on Charmhub, but I think systemd.service and systemd.popen_kwargs should be "private"?

Noting that the comments in the code state that users of the library shouldn't use service directly, makes sense to me that it should be _service, and _popen_kwargs would convey that it is mostly an internal implementation detail?

Add CI to automatically release libraries

Probably use the release-libs workflow, like https://github.com/canonical/alertmanager-k8s-operator/pull/116/files

See also https://discourse.charmhub.io/t/charming-ci-improvements-for-a-smooth-life/8543?u=lucabello

Add functional test for grub lib

The grub lib created in #98 is missing functional tests that should be added. Plan:

• create a list of test cases
• covered all relevant test cases not covered by unit tests

Hold refresh for specific snaps

Following conversation on #80 and #81. We should be able to hold a refresh per snap, not only system wide.

Request: Option for apt lib to install not backports version package

freeipmi-tools failed to install on focal

Relate: canonical/hardware-observer-operator#128

Reproduce

install.py

apt.add_package("freeipmi-tools", update_cache=False)

$ python install.py
Traceback (most recent call last):
  File "/home/ubuntu/operator-libs-linux/./install.py", line 3, in <module>
    apt.add_package("freeipmi-tools", update_cache=False)
  File "/home/ubuntu/operator-libs-linux/lib/charms/operator_libs_linux/v0/apt.py", line 761, in add_package
    pkg, success = _add(p, version, arch)
                   ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/operator-libs-linux/lib/charms/operator_libs_linux/v0/apt.py", line 802, in _add
    pkg.ensure(state=PackageState.Present)
  File "/home/ubuntu/operator-libs-linux/lib/charms/operator_libs_linux/v0/apt.py", line 289, in ensure
    self._add()
  File "/home/ubuntu/operator-libs-linux/lib/charms/operator_libs_linux/v0/apt.py", line 261, in _add
    self._apt(
  File "/home/ubuntu/operator-libs-linux/lib/charms/operator_libs_linux/v0/apt.py", line 255, in _apt
    raise PackageError(
lib.charms.operator_libs_linux.v0.apt.PackageError: Could not install package(s) [['freeipmi-tools=1.6.9-2~bpo20.04.1']]: None

---

The apt lib choose backports version to install instead of focal-updates

$ apt-cache policy freeipmi-tools
freeipmi-tools:
  Installed: (none)
  Candidate: 1.6.4-3ubuntu1.1
  Version table:
     1.6.9-2~bpo20.04.1 100
        100 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages
     1.6.4-3ubuntu1.1 500
        500 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
     1.6.4-3ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages

One way for us to resolve this issue is try to catch the target version from the command, but I feel more nice if apt lib can support it.

Fix failing test_snap_set_and_get_with_typed integration test

This is failing with an error connecting to the snapd socket:

tests/integration/test_snap.py::test_snap_set_and_get_with_typed 
-------------------------------- live log call ---------------------------------
INFO     charms.operator_libs_linux.v2.snap:snap.py:590 Refreshing snap lxd, revision None, tracking latest
error: cannot perform the following tasks:
- Run configure hook of "lxd" snap if present (run hook "configure": error: cannot communicate with server: \
    Post "http://localhost/v2/snapctl": dial unix /run/snapd-snap.socket: connect: no such file or directory)
FAILED

Other similar tests seem to be succeeding.

snap cache not populated in the way that the snap lib seems to expect

When tests/integration/test_snap.py on a 20.04 image, I get a lot of warning messages about the snap cache not being populated. I'd expect those warnings to go away after the first test that uses them.

We should brush up on how snapd currently handles its cache, and ensure that the library (which is based on charmhelpers code) interacts with it appropriately.

Unable to install packages on Xenial/Bionic

To reproduce:

Prepare up a LXD container

lxc launch ubuntu:xenial xenial
lxc exec xenial bash
wget https://raw.githubusercontent.com/canonical/operator-libs-linux/main/lib/charms/operator_libs_linux/v0/apt.py
python3

Try to install a package

import apt
apt.add_package(["python3-pymongo"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/apt.py", line 761, in add_package
    pkg, success = _add(p, version, arch)
  File "/root/apt.py", line 802, in _add
    pkg.ensure(state=PackageState.Present)
  File "/root/apt.py", line 290, in ensure
    self._add()
  File "/root/apt.py", line 265, in _add
    optargs=["--option=Dpkg::Options::=--force-confold"],
  File "/root/apt.py", line 258, in _apt
    ) from None
apt.PackageError: Could not install package(s) [['python3-pymongo=3.2-1build1']]: None

Without env specified installation succeeds:

sed -i s'/check_call(_cmd, env=env, stderr=PIPE, stdout=PIPE)/check_call(_cmd, stderr=PIPE, stdout=PIPE)/' apt.py

python3
Python 3.5.2 (default, Jan 26 2021, 13:30:48) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import apt
>>> apt.add_package(["python3-pymongo"])
<apt.DebianPackage: {'_arch': 'amd64', '_name': 'python3-pymong', '_version': <apt.Version: {'_epoch': '', '_version': '3.2-1build1'}>, '_state': <PackageState.Present: 'present'>}>

The root cause of this seems to be PATH not making its way to the subprocess env, per dpkg: error: PATH is not set in the following output:

root@xenial:~# python3
Python 3.5.2 (default, Jan 26 2021, 13:30:48) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> _cmd = ['apt-get', '-y', '--option=Dpkg::Options::=--force-confold', 'install', 'python3-pymongo=3.2-1build1']
>>> env={"DEBIAN_FRONTEND": "noninteractive"}
>>> subprocess.run(_cmd, env=env)
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
The following additional packages will be installed:
  python3-pymongo-ext
Suggested packages:
  python-pymongo-doc
The following NEW packages will be installed:
  python3-pymongo python3-pymongo-ext
0 upgraded, 2 newly installed, 0 to remove and 1 not upgraded.
Need to get 0 B/109 kB of archives.
After this operation, 674 kB of additional disk space will be used.
debconf: delaying package configuration, since apt-utils is not installed
dpkg: error: PATH is not set
E: Sub-process /usr/bin/dpkg returned an error code (2)
CompletedProcess(args=['apt-get', '-y', '--option=Dpkg::Options::=--force-confold', 'install', 'python3-pymongo=3.2-1build1'], returncode=100)
>>> 

As the default behaviour when env is not specified is to inherit from the parent shell, I propose a change to copy the current os.environ before we specify any new keys.

If env is not None, it must be a mapping that defines the environment variables for the new process; these are used instead of the default behavior of inheriting the current process’ environment.

Placeholder charm should be a machine charm

Currently, the placeholder charm that hosts these libraries has a default metadata.yaml, which references a placeholder container. This leads to the store to believe that the charm is a k8s charm.

And that is a little confusing :-)

This issue can be closed once the placeholder resource is removed from the metadata.

Snap `get` and `set` methods should adhere to Python's built-in types

Currently, Snap.get always returns the value as a string, and Snap.set always set the value as a string. It will be very helpful for these two methods to respect the types. For example, it will be nice that we can do the following

s = snap.SnapCache()["some-snap"]
s.set({"foo": True,  "bar": {"a": 1, "b": 2.0, "c": None, "d": ["x", "y", "z"]}, "bar.greeting": "hello world", "buzz": 123, "fuzz": 123.123, "fizz": None})

x = s.get("foo")
y = s.get("bar")
y_1 = s.get("bar.a")
y_2 = s.get("bar.b")
# y_3 = s.get("bar.c") # None is not set
y_4 = s.get("bar.d")
y_5 = s.get("bar.greeting")
a = s.get("buzz")
b = s.get("fuzz")
# c = s.get("fizz") # None is not set

print(x, type(x))
print(y, type(y))
print(y_1, type(y_1))
print(y_2, type(y_2))
# print(y_3, type(y_3)) 
print(y_4, type(y_4))
print(y_5, type(y_5))
print(a, type(a))
print(b, type(b))
# print(c, type(c))

should prints

True <class 'bool'>
{'a': 1, 'b': 2.0, 'd': ['x', 'y', 'z'], 'greeting': 'hello world'} <class 'dict'>
1 <class 'int'>
2.0 <class 'float'>
['x', 'y', 'z'] <class 'list'>
hello world <class 'str'>
123 <class 'int'>
123.123 <class 'float'>

Example usages:

# In app_1.py
s = snap.SnapCache()["some-snap"]
s.set({"logging.enable": True})

...

# In app_2.py
s = snap.SnapCache()["some-snap"]
# We can do 
if s.get("logging.enable"):
    do_something()
# Instead of
if s.get("logging.enable") == "True":
    do_something()

juju-systemd-notices does not support nested service names like snap.slurm.slurmd

The juju-systemd-notices charm library does not support observing snap service state.

The issue originates from the fact that snap service names are "nested," where the service name is interpolated with .. These snap-specific service names cause the charm library to bork because when you go to define a custom event for the watched service in ops, the charm will fail to define the event as . is an invalid character for event names. Screenshot below shows the error encountered when passing the service name as snap.slurm.slurmd:

Support snap cohorts

The charm-lxd supports deploying clustered LXD units. Those deployments require using snap cohorts to ensure all the cluster members get the same LXD snap. Our adhoc snap install/refresh routine effectively does this:

snap install lxd --channel=latest/stable --cohort=+
# or
snap refresh lxd --channel=latest/stable --cohort=+

It would be nice for the snap implementation to grow support for this.

Calling `Snap.ensure(....)` with the same `revision` shouldn't refresh the snap if the revision is already installed

I didn't expect the ensure(...) method to actually cause a snap-refresh if the revision requested already matched the installed revision. If one calls ensure(...) with a channel, and the revision is unchanged, no refresh is triggered.

So, in my charm where i'm trying to set a specific revision on the snap, I had to add this if clause prevent a constant snap refresh to a revision where it already was.

    cache = snap_lib.SnapCache()
    which = cache[args.name]
    if which.revision != args.revision:
        log.info("Ensuring %s snap revision=%s", args.name, args.revision)
        which.ensure(**args.dict(exclude_none=True))

ensure does not properly support "classic" confinement.

When a charm is already installed, and you refresh to a classic revision using ensure(state=snap_lib.SnapState.Present, revision=N, classic=True), the lib does not add --classic to the command and so it fails.

Snap get None key fails with `TypeError: expected str, bytes or os.PathLike object, not NoneType`

get's interface allows for key to be optional when typed is provided:

operator-libs-linux/lib/charms/operator_libs_linux/v2/snap.py

Line 313 in 522d54e

def get(self, key: Optional[str], *, typed: bool = False) -> Any:

But, if you actually provide None, you get a TypeError:

snap.SnapCache()["openstack-hypervisor"].get(None, typed=True)
*** TypeError: expected str, bytes or os.PathLike object, not NoneType

The key is passed as is to the subprocess call, but subprocess does not support null arguments:

subprocess.check_output(["cat", None], universal_newlines=True)
*** TypeError: expected str, bytes or os.PathLike object, not NoneType

(tested with jammy's python 3.10.12)

Improve testing of charm libraries

Per our discussion on the dnf charm library pull request here #75 (comment), we realized that there is potentially a better way to launch tests without having to use pytest's built-in flag --ignore=... to ignore specific tests. In the discussion thread, it was suggested to use @pytest.mark.<test_suite> with pytest -m <test_suite>.

I also wanted to suggest potentially using cleantest for helping with the overhaul of the tests for the charm libraries. I noticed in the tox.ini file that there is a standalone integration environment for just setting up an LXD test instance:

operator-libs-linux/tox.ini

Lines 70 to 88 in d0ebc9f

	[testenv:integration]
	description = Build a LXD container for integration tests then run the tests
	allowlist_externals =
	lxc
	bash
	commands =
	# Create a LXC container with the relevant packages installed
	bash -c 'lxc launch -qe ubuntu:focal {[vars]lxd_name} -c=user.user-data="$(<{[vars]itst_dir}/test_setup.yaml)"'
	# Wait for the cloud-init process to finish
	lxc exec {[vars]lxd_name} -- bash -c "cloud-init status -w >/dev/null 2>&1"
	# Copy all the files needed for integration testing
	lxc file push -qp {toxinidir}/tox.ini {[vars]lxd_name}/{[vars]lxd_name}/
	lxc file push -qp {toxinidir}/pyproject.toml {[vars]lxd_name}/{[vars]lxd_name}/
	lxc file push -qpr {toxinidir}/lib {[vars]lxd_name}/{[vars]lxd_name}/
	lxc file push -qpr {[vars]tst_dir} {[vars]lxd_name}/{[vars]lxd_name}/
	# Run the tests
	lxc exec {[vars]lxd_name} -- tox -c /{[vars]lxd_name}/tox.ini -e integration-tests {posargs}
	commands_post =
	-lxc stop {[vars]lxd_name}

cleantest can handle both bringing up and tearing down the Ubuntu test instance. This way the LXD logic does not have to live inside of tox.ini and the integration environment can be dedicated for just launching the integration tests.

passwd: not possible to set home directory for system user

Currently it is not possible to create system users with a home directory using the public api of the passwd lib without specifying a password. However I think this is a valid use case. According to the useradd man page -m, --create-home flag must be passed for this to work.

...
-r, --system
           Create a system account.
...

           Note that useradd will not create a home directory for such a user, regardless of the default setting in /etc/login.defs (CREATE_HOME). You have to specify the -m options if you want a home directory for a
           system account to be created.

Upgrade lint tooling in line with styleguide

Other charms and repos have moved away from flake8 / isort to ruff - we should follow suit.

Make deploying the placeholder charm do something useful

A fun idea from @jnsgruk:

A thought I had for the future: it would be nice if deploying the charm did something useful. One idea that I had was to deploy a very simple web server that could render the docs for each lib. Not exactly high priority, but could be interesting - I imagine we could re-use the code from charmhub.io that renders the library docs pages.

If we do this, it'd be nice to make it reusable/shareable, or put it in the charm-lib template (if there is one).

Snap API raises errors when searching the snap cache for locally installed snap package

During my quest to fix #54, I noticed that I would receive HTTP errors when contacting the snapd.socket after I installed the local snap package. This error prevented install_local from returning the Snap wrapper class.

Upon further investigation, I found the issue to be that check_output was also capturing ANSI escape sequences:

>>> import subprocess
>>> res = subprocess.check_output(["snap", "install", "./local_amd64.snap", "--dangerous"], universal_newlines=True).splitlines()[-1]
>>> snap_name, _ = res.split(" ", 1)
>>> snap_name
'\x1b[0m\x1b[?25h\x1b[Khello-world-gtk'

These escape sequences need to be filtered out of the snap_name string for the snapd API call to function correctly.

Support removing users

So far it looks as if there is support for adding users, but there is no easy way to remove users. The underlying function passwd.User._remove is implemented, but there is neither bare method, nor a passwd.UserState that triggers it.