canonical / observability-libs

A collection of charm libraries curated by the Observability team.

Observability Libs

Description

Observability Libs is a placeholder charm that is used to provide a home for charm libraries that are used for the development of charms deployed as part of the COS Lite bundle.

Usage

While it is possible to deploy this charm, it is only a tester charm and is intended to be deployed only for testing purposes.

charmcraft pack
juju deploy ./observability-libs_ubuntu-20.04-amd64.charm \
  --resource placeholder-image=busybox

Where this charm should be used, is to access one of the following libraries during development:

• KubernetesServicePatch - a small library used to patch the Juju auto-created Kubernetes Service during the deployment of a sidecar charm to contain the correct ports for an application.

• KubernetesComputeResourcesPatch - a small library used to patch the Juju auto-created statefulset with custom resource limits.

• JujuTopology - Used to create and output Juju topologies either from charms, relation data, or parts.

Contributing

Please see the Juju SDK docs for guidelines on enhancements to this charm following best practice guidelines, and CONTRIBUTING.md for developer guidance.

observability-libs's Issues

Fix static check in libs.

Enhancement Proposal

While working in PR #66 I saw we have a lot of errors in our libs static check :

$ tox -e static-lib                                                                                                                     
static-lib installed: anyio==4.2.0,certifi==2023.11.17,exceptiongroup==1.2.0,h11==0.14.0,httpcore==1.0.2,httpx==0.26.0,idna==3.6,lightkube==0.15.0,lightkube-models==1.29.0.6,nodeenv==1.8.0,ops==2.9.0,pyright==1.1.344,PyYAML==6.0.1,sniffio==1.3.0,typing_extensions==4.9.0,websocket-client==1.7.0
static-lib run-test-pre: PYTHONHASHSEED='3542122872'
static-lib run-test: commands[0] | pyright --pythonversion 3.8 /home/ubuntu/repos/observability-libs/lib/charms/observability_libs
/home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/kubernetes_compute_resources_patch.py
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/kubernetes_compute_resources_patch.py:378:16 - error: Expression of type "Unknown | None" cannot be assigned to return type "ResourceRequirements"
    Type "Unknown | None" cannot be assigned to type "ResourceRequirements"
      "None" is incompatible with "ResourceRequirements" (reportGeneralTypeIssues)
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/kubernetes_compute_resources_patch.py:386:16 - error: Expression of type "Unknown | None" cannot be assigned to return type "ResourceRequirements"
    Type "Unknown | None" cannot be assigned to type "ResourceRequirements"
      "None" is incompatible with "ResourceRequirements" (reportGeneralTypeIssues)
/home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/metrics_endpoint_discovery.py
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/metrics_endpoint_discovery.py:211:28 - error: "annotations" is not a known member of "None" (reportOptionalMemberAccess)
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/metrics_endpoint_discovery.py:212:44 - error: "annotations" is not a known member of "None" (reportOptionalMemberAccess)
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/metrics_endpoint_discovery.py:215:68 - error: "containers" is not a known member of "None" (reportOptionalMemberAccess)
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/metrics_endpoint_discovery.py:221:31 - error: "namespace" is not a known member of "None" (reportOptionalMemberAccess)
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/metrics_endpoint_discovery.py:222:26 - error: "name" is not a known member of "None" (reportOptionalMemberAccess)
/home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v1/kubernetes_service_patch.py
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v1/kubernetes_service_patch.py:317:77 - error: "ports" is not a known member of "None" (reportOptionalMemberAccess)
8 errors, 0 warnings, 0 informations 
ERROR: InvocationError for command /home/ubuntu/repos/observability-libs/.tox/static-lib/bin/pyright --pythonversion 3.8 lib/charms/observability_libs (exited with code 1)
__________________________________________________________________ summary ___________________________________________________________________
ERROR:   static-lib: commands failed

We have to fix, and release new versions of:

• lib/charms/observability_libs/v0/kubernetes_compute_resources_patch.py
• lib/charms/observability_libs/v0/metrics_endpoint_discovery.py
• lib/charms/observability_libs/v1/kubernetes_service_patch.py

Decide on either `charm_name` or `charm` and use it consistently

We should start a list for the time we choose to break backwards compat and add there to either "charm" or "charm_name" everywhere.

Originally posted by @sed-i in #11 (comment)

unit tests fail on master

Bug Description

unit tests fail

To Reproduce

tox -e unit

Environment

Relevant log output

=============================================================================================================== FAILURES ===============================================================================================================
______________________________________________________________________________ TestKubernetesComputeResourcesPatch.test_invalid_config_emits_custom_event ______________________________________________________________________________
Traceback (most recent call last):
  File "/home/maksim/git/observability-libs/tests/unit/test_kubernetes_compute_resources.py", line 35, in setUp
    self.harness = Harness(
  File "/home/maksim/git/observability-libs/.tox/unit/lib/python3.10/site-packages/ops/testing.py", line 175, in __init__
    config_ = self._get_config(config)
  File "/home/maksim/git/observability-libs/.tox/unit/lib/python3.10/site-packages/ops/testing.py", line 466, in _get_config
    assert isinstance(charm_config, str)  # type guard
AssertionError
________________________________________________________________________ TestKubernetesComputeResourcesPatch.test_listener_is_attached_for_config_changed_event ________________________________________________________________________
Traceback (most recent call last):
  File "/home/maksim/git/observability-libs/tests/unit/test_kubernetes_compute_resources.py", line 35, in setUp
    self.harness = Harness(
  File "/home/maksim/git/observability-libs/.tox/unit/lib/python3.10/site-packages/ops/testing.py", line 175, in __init__
    config_ = self._get_config(config)
  File "/home/maksim/git/observability-libs/.tox/unit/lib/python3.10/site-packages/ops/testing.py", line 466, in _get_config
    assert isinstance(charm_config, str)  # type guard
AssertionError
__________________________________________________________________________ TestKubernetesComputeResourcesPatch.test_patch_is_applied_during_startup_sequence ___________________________________________________________________________
Traceback (most recent call last):
  File "/home/maksim/git/observability-libs/tests/unit/test_kubernetes_compute_resources.py", line 35, in setUp
    self.harness = Harness(
  File "/home/maksim/git/observability-libs/.tox/unit/lib/python3.10/site-packages/ops/testing.py", line 175, in __init__
    config_ = self._get_config(config)
  File "/home/maksim/git/observability-libs/.tox/unit/lib/python3.10/site-packages/ops/testing.py", line 466, in _get_config
    assert isinstance(charm_config, str)  # type guard
AssertionError
______________________________________________________________________ TestKubernetesComputeResourcesPatch.test_patch_is_applied_regardless_of_leadership_status _______________________________________________________________________
Traceback (most recent call last):
  File "/home/maksim/git/observability-libs/tests/unit/test_kubernetes_compute_resources.py", line 35, in setUp
    self.harness = Harness(
  File "/home/maksim/git/observability-libs/.tox/unit/lib/python3.10/site-packages/ops/testing.py", line 175, in __init__
    config_ = self._get_config(config)
  File "/home/maksim/git/observability-libs/.tox/unit/lib/python3.10/site-packages/ops/testing.py", line 466, in _get_config
    assert isinstance(charm_config, str)  # type guard
AssertionError
=========================================================================================================== warnings summary ===========================================================================================================
.tox/unit/lib/python3.10/site-packages/_pytest/config/__init__.py:1253
  /home/maksim/git/observability-libs/.tox/unit/lib/python3.10/site-packages/_pytest/config/__init__.py:1253: PytestConfigWarning: Unknown config option: asyncio_mode
  
    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================================================================= short test summary info ========================================================================================================
FAILED tests/unit/test_kubernetes_compute_resources.py::TestKubernetesComputeResourcesPatch::test_invalid_config_emits_custom_event - AssertionError
FAILED tests/unit/test_kubernetes_compute_resources.py::TestKubernetesComputeResourcesPatch::test_listener_is_attached_for_config_changed_event - AssertionError
FAILED tests/unit/test_kubernetes_compute_resources.py::TestKubernetesComputeResourcesPatch::test_patch_is_applied_during_startup_sequence - AssertionError
FAILED tests/unit/test_kubernetes_compute_resources.py::TestKubernetesComputeResourcesPatch::test_patch_is_applied_regardless_of_leadership_status - AssertionError

Additional context

No response

[CI] Attempting to run tox without checking if an env exists

In CI we run tox -e <env_name> without checking first that the env exists:

observability-libs/.github/workflows/ci.yaml

Lines 23 to 24 in 3d71ec8

	- name: Run static analysis (lib)
	run: tox -vve static-lib

This results in:

ERROR: unknown environment 'static-lib'

(cc: @dstathis re: canonical/cos-configuration-k8s-operator#24)

Hook failed on "certificates-relation-broken"

Bug Description

When certification broken events occur, an all_certificates_invalidated event is emitted from tls_certificates_interface/v2. This event is received in cert_handler, which subsequently runs this code. However, this execution fails because the certificate no longer exists in the relation.

To Reproduce

1. juju deploy self-signed-certificates & any cos-charm
2. relate ca to any cos-charm over the certificates relation
3. juju reomve relation ca:certificates alertmanager:certificates

Environment

Model     Controller  Cloud/Region        Version  SLA          Timestamp
cos-loki  k8s         microk8s/localhost  3.1.7    unsupported  16:55:37+02:00

App            Version  Status   Scale  Charm                     Channel  Rev  Address         Exposed  Message
alertmanager   0.26.0   waiting      1  alertmanager-k8s          stable   101  10.152.183.169  no       installing agent
ca                      active       1  self-signed-certificates  edge     117  10.152.183.167  no         

Unit              Workload  Agent  Address      Ports  Message
alertmanager/0*   error     idle   10.1.54.106         hook failed: "certificates-relation-broken" for ca:certificates
ca/0*             active    idle   10.1.54.66               

Integration provider                Requirer                     Interface              Type     Message
ca:certificates                     alertmanager:certificates    tls-certificates       regular  

Relevant log output

unit-ca-0: 16:51:34 INFO juju.worker.uniter.operation ran "certificates-relation-departed" hook (via hook dispatching script: dispatch)
unit-alertmanager-0: 16:51:35 INFO juju.worker.uniter.operation ran "certificates-relation-departed" hook (via hook dispatching script: dispatch)
unit-ca-0: 16:51:35 INFO juju.worker.uniter.operation ran "certificates-relation-broken" hook (via hook dispatching script: dispatch)
unit-alertmanager-0: 16:51:35 WARNING unit.alertmanager/0.juju-log certificates:28: 'app' expected but not received.
unit-alertmanager-0: 16:51:36 WARNING unit.alertmanager/0.juju-log certificates:28: 'app_name' expected in snapshot but not found.
unit-alertmanager-0: 16:51:36 INFO unit.alertmanager/0.juju-log certificates:28: Creating CSR for alertmanager-0 with DNS ['alertmanager-0.alertmanager-endpoints.cos-loki.svc.cluster.local'] and IPs []
unit-alertmanager-0: 16:51:36 ERROR unit.alertmanager/0.juju-log certificates:28: Uncaught exception while in charm code:
Traceback (most recent call last):
  File "./src/charm.py", line 572, in <module>
    main(AlertmanagerCharm)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/venv/ops/main.py", line 456, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/venv/ops/main.py", line 144, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/venv/ops/framework.py", line 351, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/venv/ops/framework.py", line 853, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/venv/ops/framework.py", line 943, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/lib/charms/tls_certificates_interface/v2/tls_certificates.py", line 1861, in _on_relation_broken
    self.on.all_certificates_invalidated.emit()
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/venv/ops/framework.py", line 351, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/venv/ops/framework.py", line 853, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/venv/ops/framework.py", line 943, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 420, in _on_all_certificates_invalidated
    self._generate_csr(overwrite=True, clear_cert=True)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 272, in _generate_csr
    self.certificates.request_certificate_creation(certificate_signing_request=csr)
  File "/var/lib/juju/agents/unit-alertmanager-0/charm/lib/charms/tls_certificates_interface/v2/tls_certificates.py", line 1614, in request_certificate_creation
    raise RuntimeError(
RuntimeError: Relation certificates does not exist - The certificate request can't be completed

Additional context

No response

K8s service patch: avoid racing with controller

Bug Description

We are worried that we're accidentally racing/clobbering the controller patching the object with something important.
Log messages like this one:
ERROR juju.worker.caasapplicationprovisioner.runner exited "prometheus-k8s": Operation cannot be fulfilled on pods "prometheus-k8s-0": the object has been modified; please apply your changes to the latest version and try again
It would be ideal to have some hook/event where we know "Juju is done trying to modify these objects, and it's safe to patch it now".
Is there a time where we can "know" what whatever the controller is trying to do in the message near the top of this is done?
-- @rbarry82

To Reproduce

Deploy charm.

Environment

Github actions and local.

Relevant log output

ERROR juju.worker.caasapplicationprovisioner.runner exited "prometheus-k8s": Operation cannot be fulfilled on pods "prometheus-k8s-0": the object has been modified; please apply your changes to the latest version and try again

Additional context

No response

Fix `_generate_privkey`

Enhancement Proposal

In _generate_privkey we are getting the private key from relation not secrets:

private_key = relation.data[self.charm.unit].get("private-key", None)

We should use:

self.private_key

Issue found in: https://github.com/canonical/alertmanager-k8s-operator/actions/runs/7957275284/job/21736430013?pr=215

Typo in regex in `is_valid_uuid()`

Bug Description

I think we are having an issue in the method is_valid_uuid() of juju_topology.py. Seems the regex has a typo, and unit test are failing.

    def is_valid_uuid(self, uuid):
        """Validates the supplied UUID against the Juju Model UUID pattern."""
        regex = re.compile(
            "^[a-f0-9]{8}-?[a-f0-9]{4}-?4[a-f0-9]{3}-?[89ab][a-f0-9]{3}-?[a-f0-9]{12}$"
        )
        return bool(regex.match(uuid))

With this 👆 regex the UUID f2c1b2a6-e006-11eb-ba80-0242ac130004 is not matching, but with this one UUIDs matches.

To Reproduce

Just run unit test in any charm that uses juju_topology.py

Environment

edge

Relevant log output

_____ TestActiveStatus.test_unit_is_blocked_if_reload_configuration_fails ______
Traceback (most recent call last):
  File "/home/runner/work/prometheus-k8s-operator/prometheus-k8s-operator/tests/unit/test_charm_status.py", line 78, in test_unit_is_blocked_if_reload_configuration_fails
    self.harness.begin_with_initial_hooks()
  File "/home/runner/work/prometheus-k8s-operator/prometheus-k8s-operator/.tox/unit/lib/python3.8/site-packages/ops/testing.py", line 225, in begin_with_initial_hooks
    self.begin()
  File "/home/runner/work/prometheus-k8s-operator/prometheus-k8s-operator/.tox/unit/lib/python3.8/site-packages/ops/testing.py", line 188, in begin
    self._charm = TestCharm(self._framework)
  File "/home/runner/work/prometheus-k8s-operator/prometheus-k8s-operator/src/charm.py", line 62, in __init__
    self._scraping = MetricsEndpointProvider(
  File "/home/runner/work/prometheus-k8s-operator/prometheus-k8s-operator/lib/charms/prometheus_k8s/v0/prometheus_scrape.py", line 1410, in __init__
    self.topology = JujuTopology.from_charm(charm)
  File "/home/runner/work/prometheus-k8s-operator/prometheus-k8s-operator/lib/charms/observability_libs/v0/juju_topology.py", line 144, in from_charm
    return cls(
  File "/home/runner/work/prometheus-k8s-operator/prometheus-k8s-operator/lib/charms/observability_libs/v0/juju_topology.py", line 120, in __init__
    raise InvalidUUIDError(model_uuid)
charms.observability_libs.v0.juju_topology.InvalidUUIDError: 'f2c1b2a6-e006-11eb-ba80-0242ac130004' is not a valid UUID.

Additional context

With this change, UUIDs like f2c1b2a6-e006-11eb-ba80-0242ac130004 are validated.

@@ -128,7 +128,7 @@ class JujuTopology:
     def is_valid_uuid(self, uuid):
         """Validates the supplied UUID against the Juju Model UUID pattern."""
         regex = re.compile(
-            "^[a-f0-9]{8}-?[a-f0-9]{4}-?4[a-f0-9]{3}-?[89ab][a-f0-9]{3}-?[a-f0-9]{12}$"
+            "^[a-f0-9]{8}-?[a-f0-9]{4}-?[a-f0-9]{4}-?[89ab][a-f0-9]{3}-?[a-f0-9]{12}$"
         )
         return bool(regex.match(uuid))

`private_key` may end up missing from peer data

Bug Description

Related alertmanager to self-signed-certificate and saw the error.
Haven't figured out yet the code path that can lead to private_key not being set.

To Reproduce

1. Deploy alertmanager (2 units)
2. Upgrade alertmanager from path.
3. Relate to self-signed-certificate charm.

Environment

Model                        Controller  Cloud/Region        Version  SLA          Timestamp
test-prometheus-alerts-32k4  uk8s        microk8s/localhost  3.2.0    unsupported  19:10:57-04:00

App   Version  Status   Scale  Charm                     Channel  Rev  Address         Exposed  Message
am    0.25.0   waiting      2  alertmanager-k8s          edge       0  10.152.183.174  no       waiting for units to settle down
ca             active       1  self-signed-certificates  edge       8  10.152.183.38   no       

Unit     Workload  Agent  Address       Ports  Message
am/0     error     idle   10.1.166.78          hook failed: "certificates-relation-joined" for ca:certificates
am/1*    error     idle   10.1.166.80          hook failed: "certificates-relation-joined" for ca:certificates
ca/0*    active    idle   10.1.166.99          

Relevant log output

unit-am-0: 19:06:31.353 DEBUG unit.am/0.juju-log certificates:7: Emitting Juju event certificates_relation_joined.
unit-am-0: 19:06:31.371 ERROR unit.am/0.juju-log certificates:7: Uncaught exception while in charm code:
Traceback (most recent call last):
  File "./src/charm.py", line 540, in <module>
    main(AlertmanagerCharm, use_juju_for_storage=True)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/main.py", line 441, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/main.py", line 149, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/framework.py", line 344, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/framework.py", line 833, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/framework.py", line 922, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-am-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 182, in _on_certificates_relation_joined
    self._generate_csr()
  File "/var/lib/juju/agents/unit-am-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 213, in _generate_csr
    assert private_key is not None  # for type checker
AssertionError

Additional context

No response

cert --> sans

Enhancement Proposal

Originally posted by @sed-i in this traefik PR

In the past it used to be the DNS, but now it's deprecated and certs only use SANs.
In the wild (e.g. google) still use something the looks like a hostname in the CN, but it might as well be anything else.
If our method is called cert_subject but in fact produces SANs then we should probably rename it.

Abstract `ServicePort` from users at instantiation

Enhancement Proposal

Proposal

Change ports parameter to receive a list (or iterable) of dictionaries instead of a list of ServicePort at instantiation. The library would then have to make ServicePort instances for each port that is passed.

Example:

# ...
from charms.observability_libs.v1.kubernetes_service_patch import KubernetesServicePatch

class SomeCharm(CharmBase):
  def __init__(self, *args):
    # ...
    tcp = {"port": 443, "name":{f"{self.app.name}-tcp"}, "protocol":"TCP")
...
    self.service_patcher = KubernetesServicePatch(self, [tcp,...])
    # ...

Reasoning

Users need to add from lightkube.models.core_v1 import ServicePort. Although not harmful at all, this could create some redundancy in charms that have lightkube abstracted by libraries or packages like charmed-kubeflow-chisme.

Refactor `is_valid_uuid` method

Today the is_valid_uuid method in JujuTopology class, utilises a regex to validate UUID v4:

def is_valid_uuid(self, uuid):
    """Validates the supplied UUID against the Juju Model UUID pattern."""
    regex = re.compile(
        "^[a-f0-9]{8}-?[a-f0-9]{4}-?4[a-f0-9]{3}-?[89ab][a-f0-9]{3}-?[a-f0-9]{12}$"
    )
    return bool(regex.match(uuid))

An old friend told me many years ago:

If you have a problem and you use regular expressions to solve it, you now have two problems.

Jokes aside, regular expressions are powerful but complex to read sometimes.

We can avoid regex in this case by using the python uuid module. The result is much easy to read and maintain.

def is_valid_uuid(self, _uuid) -> bool:
     """Validate the supplied UUID against the Juju Model UUID pattern."""
     try:
         return True if uuid.UUID(_uuid).version == 4 else False
     except (TypeError, ValueError):
         return False

fix static-lib check

Enhancement Proposal

Our tox -e static-lib check is failing, and that prevents us from merging simple PRs.
We have to fix all these warnings:

$ tox -e static-lib 
static-lib installed: anyio==4.3.0,certifi==2024.2.2,exceptiongroup==1.2.0,h11==0.14.0,httpcore==1.0.3,httpx==0.26.0,idna==3.6,lightkube==0.15.2,lightkube-models==1.29.0.7,nodeenv==1.8.0,ops==2.10.0,pyright==1.1.351,PyYAML==6.0.1,sniffio==1.3.0,typing_extensions==4.9.0,websocket-client==1.7.0
static-lib run-test-pre: PYTHONHASHSEED='797004744'
static-lib run-test: commands[0] | pyright --pythonversion 3.8 /home/ubuntu/repos/observability-libs/lib/charms/observability_libs
/home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/kubernetes_compute_resources_patch.py
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/kubernetes_compute_resources_patch.py:367:16 - error: No overloads for "equals_canonically" match the provided arguments (reportCallIssue)
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/kubernetes_compute_resources_patch.py:367:35 - error: Argument of type "ResourceRequirements | None" cannot be assigned to parameter "first" of type "ResourceRequirements" in function "equals_canonically"
    Type "ResourceRequirements | None" cannot be assigned to type "ResourceRequirements"
      "None" is incompatible with "ResourceRequirements" (reportArgumentType)
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/kubernetes_compute_resources_patch.py:400:51 - error: No overloads for "equals_canonically" match the provided arguments (reportCallIssue)
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/kubernetes_compute_resources_patch.py:401:28 - error: Argument of type "ResourceRequirements | None" cannot be assigned to parameter "second" of type "ResourceRequirements" in function "equals_canonically"
    Type "ResourceRequirements | None" cannot be assigned to type "ResourceRequirements"
      "None" is incompatible with "ResourceRequirements" (reportArgumentType)
/home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/metrics_endpoint_discovery.py
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/metrics_endpoint_discovery.py:218:22 - error: No overloads for "__new__" match the provided arguments (reportCallIssue)
  /home/ubuntu/repos/observability-libs/lib/charms/observability_libs/v0/metrics_endpoint_discovery.py:218:60 - error: Argument of type "List[ContainerPort] | None" cannot be assigned to parameter "__iterable" of type "Iterable[_T@filter]" in function "__new__"
    Type "List[ContainerPort] | None" cannot be assigned to type "Iterable[ContainerPort]"
      "None" is incompatible with protocol "Iterable[ContainerPort]"
        "__iter__" is not present (reportArgumentType)
6 errors, 0 warnings, 0 informations 
ERROR: InvocationError for command /home/ubuntu/repos/observability-libs/.tox/static-lib/bin/pyright --pythonversion 3.8 lib/charms/observability_libs (exited with code 1)
__________________________________________________________________ summary ___________________________________________________________________
ERROR:   static-lib: commands failed

Decouple privkey management from cert handling

          In a separate issue, should we decouple privkey management from cert handling?

Originally posted by @sed-i in #66 (comment)

[cert handler] incorrect error message on import failure

When importing tls_certificates fails, the error suggests fetch-lib:

observability-libs/lib/charms/observability_libs/v0/cert_handler.py

Line 52 in 54bf73c

"charms.tls_certificates_interface.v2.tls_certificates is missing; please get it through charmcraft fetch-lib"

This is not accurate, as an import failure can be caused by a different reason:

ERROR juju.worker.uniter.operation hook "install" (via hook dispatching script: dispatch) failed: exit status 1
INFO juju.worker.uniter awaiting error resolution for "install" hook
INFO juju.worker.uniter awaiting error resolution for "install" hook
INFO juju.worker.uniter awaiting error resolution for "install" hook
WARNING unit.grafana-agent/0.install Traceback (most recent call last):
WARNING unit.grafana-agent/0.install   File "/var/lib/juju/agents/unit-grafana-agent-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 41, in <module>
WARNING unit.grafana-agent/0.install     from charms.tls_certificates_interface.v2.tls_certificates import (  # type: ignore
WARNING unit.grafana-agent/0.install   File "/var/lib/juju/agents/unit-grafana-agent-0/charm/lib/charms/tls_certificates_interface/v2/tls_certificates.py", line 285, in <module>
WARNING unit.grafana-agent/0.install     from cryptography import x509
WARNING unit.grafana-agent/0.install   File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/cryptography/x509/__init__.py", line 7, in <module>
WARNING unit.grafana-agent/0.install     from cryptography.x509 import certificate_transparency
WARNING unit.grafana-agent/0.install   File "/var/lib/juju/agents/unit-grafana-agent-0/charm/venv/cryptography/x509/certificate_transparency.py", line 11, in <module>
WARNING unit.grafana-agent/0.install     from cryptography.hazmat.bindings._rust import x509 as rust_x509
WARNING unit.grafana-agent/0.install ImportError: libssl.so.3: cannot open shared object file: No such file or directory
WARNING unit.grafana-agent/0.install 
WARNING unit.grafana-agent/0.install During handling of the above exception, another exception occurred:
WARNING unit.grafana-agent/0.install 
WARNING unit.grafana-agent/0.install Traceback (most recent call last):
WARNING unit.grafana-agent/0.install   File "./src/charm.py", line 20, in <module>
WARNING unit.grafana-agent/0.install     from grafana_agent import CONFIG_PATH, GrafanaAgentCharm
WARNING unit.grafana-agent/0.install   File "/var/lib/juju/agents/unit-grafana-agent-0/charm/src/grafana_agent.py", line 31, in <module>
WARNING unit.grafana-agent/0.install     from charms.observability_libs.v0.cert_handler import CertHandler
WARNING unit.grafana-agent/0.install   File "/var/lib/juju/agents/unit-grafana-agent-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 51, in <module>
WARNING unit.grafana-agent/0.install     raise ImportError(
WARNING unit.grafana-agent/0.install ImportError: charms.tls_certificates_interface.v2.tls_certificates is missing; please get it through charmcraft fetch-lib

on config changed we are not creating the private key before generating the csr

Enhancement Proposal

unit-grafana-0: 17:24:10.771 ERROR unit.grafana/0.juju-log certificates:31: Uncaught exception while in charm code:
Traceback (most recent call last):
  File "./src/charm.py", line 1480, in <module>
    main(GrafanaCharm, use_juju_for_storage=True)
  File "/var/lib/juju/agents/unit-grafana-0/charm/venv/ops/main.py", line 456, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-grafana-0/charm/venv/ops/main.py", line 144, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-grafana-0/charm/venv/ops/framework.py", line 351, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-grafana-0/charm/venv/ops/framework.py", line 853, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-grafana-0/charm/venv/ops/framework.py", line 943, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-grafana-0/charm/lib/charms/tls_certificates_interface/v2/tls_certificates.py", line 1708, in _on_relation_broken
    self.on.all_certificates_invalidated.emit()
  File "/var/lib/juju/agents/unit-grafana-0/charm/venv/ops/framework.py", line 351, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-grafana-0/charm/venv/ops/framework.py", line 853, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-grafana-0/charm/venv/ops/framework.py", line 943, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-grafana-0/charm/lib/charms/tempo_k8s/v1/charm_tracing.py", line 538, in wrapped_function
    return callable(*args, **kwargs)  # type: ignore
  File "/var/lib/juju/agents/unit-grafana-0/charm/lib/charms/observability_libs/v1/cert_handler.py", line 399, in _on_all_certificates_invalidated
    self._generate_csr(overwrite=True, clear_cert=True)
  File "/var/lib/juju/agents/unit-grafana-0/charm/lib/charms/tempo_k8s/v1/charm_tracing.py", line 538, in wrapped_function
    return callable(*args, **kwargs)  # type: ignore
  File "/var/lib/juju/agents/unit-grafana-0/charm/lib/charms/observability_libs/v1/cert_handler.py", line 234, in _generate_csr
    raise RuntimeError(
RuntimeError: private key unset. call _generate_privkey() before you call this method.

[CI] Add D401 to linting

Enhancement Proposal

I know our linting is not checking this, but just in case some day we decide to check it, we may get:

D401: First line should be in imperative mood (perhaps 'Return', not 'Returns') [D401]

Originally posted by @Abuelodelanada in canonical/prometheus-k8s-operator#373 (comment)

---

$ rg '"""\w+s\s' . returns a lot of matches.

Move some common objects to observability-libs

Enhancement Proposal

class AlertRules

AlertRules is used in both prometheus_scrape and remote_write. A variation also exists in loki_push_api.

class CosTool

CosTool is used in prometheus_scrape, remote_write and loki_push_api.

_type_convert_stored

_type_convert_stored is used in cos-configuration-k8s, alertmanager-k8s, loki-k8s, avalanche-k8s, grafana-agent-k8s, grafana-k8s, traefik-k8s, prometheus-k8s, cassandra-k8s.

Unless OF team would be willing to accept this somewhere.

relation direction validation

All of our charms either do it, or should do it, and the code definitely is generic enough to fit as a lib. One thing to keep in mind there however is that it needs to be a scoped import, as we don't want to impose the dependency on anyone else.
-- @simskij