devopsspiral / kubelibrary Goto Github PK

Cluster:

NAME                        READY   STATUS    RESTARTS   AGE
getquota-6b85cd9c49-4mjdb   1/1     Running   0          2d
traefik-6cd978c459-2l7rw    1/1     Running   0          2d3h

KubeLibrary:

Library  KubeLibrary

***Test Cases***

Pods in kube-system are ok
    [Documentation]  Test if all pods in kube-system initiated correctly and are running or succeeded
    [Tags]    cluster    smoke
    Given kubernetes API responds
    When getting all pods in  "kube-system"
    Then all pods in "kube-system" are running or succeeded

Results:

poetry run robot --outputdir reports testcases
==============================================================================
Testcases
==============================================================================
Testcases.Kubernetes
==============================================================================
Testcases.Kubernetes.Pods
==============================================================================
Pods in kube-system are ok :: Test if all pods in kube-system init... | FAIL |
No keyword with name 'Given kubernetes API responds' found.
------------------------------------------------------------------------------
Testcases.Kubernetes.Pods                                             | FAIL |
1 test, 0 passed, 1 failed
==============================================================================
Testcases.Kubernetes                                                  | FAIL |
1 test, 0 passed, 1 failed
==============================================================================
Testcases                                                             | FAIL |
1 test, 0 passed, 1 failed
==============================================================================
Output:  /Users/mariuss/work/robotest/reports/output.xml
Log:     /Users/mariuss/work/robotest/reports/log.html
Report:  /Users/mariuss/work/robotest/reports/report.html

Versions:

KubeLibrary: 0.8.0
Python: 3.9.15
Kubernetes: 1.22

This keyword should list all objects in a namesapce by listing all available api-resources and then checking if this namespace has any resources of that type.

with kubectl this can be done via

kubectl api-resources --verbs=list --namespaced -o name \
  | xargs -n 1 kubectl get --show-kind --ignore-not-found -n default

This keyword can be used to validate the existence of k8s objects for a certain namespace.

I believe it would be nice to have the keyword return a json object like this:

{
  {
    "configmaps": [
      "cm_name1",
      "cm_name2",
      "cm_name3"
    ]
  },
    {
    "pods": [
      "pod_name1",
      "pod_name2",
      "pod_name3"
    ]
  }
}

https://app.circleci.com/pipelines/github/devopsspiral/KubeLibrary/922/workflows/dbfac75b-77dc-4ce3-8be0-f61757ac5722/jobs/2643

Today, the kubernetes python library version 25.3.0 was released and it must have broken the API. Maybe a beta API was dropped.

I see the following error message in our integration tests:

Error in file '/path/to/kube.common.resource' on line 7: Initializing library 'KubeLibrary' with no arguments failed: AttributeError: module 'kubernetes.client' has no attribute 'BatchV1beta1Api' Traceback (most recent call last):   File "/usr/local/lib/python3.10/site-packages/KubeLibrary/KubeLibrary.py", line 96, in __init__     self.reload_config(kube_config=kube_config, context=context, api_url=api_url, bearer_token=bearer_token,   File "/usr/local/lib/python3.10/site-packages/KubeLibrary/KubeLibrary.py", line 263, in reload_config     self._add_api('batchv1_beta1', client.BatchV1beta1Api) AttributeError: module 'kubernetes.client' has no attribute 'BatchV1beta1Api'
--

Python package versions:

• kubernetes: 25.3.0
• robotframework-kubelibrary: 0.7.0

Improve test coverage, by writing unit test for getting configmap based on a mock k8s object that can be generated like the one in

• Create test for generating object .json
• Write test that checks target keyword
• After coverage is up modify the CI threshold in .coveragerc to new value:

(...)
[report]
fail_under = 67

When using the Get Namespaced Pod Exec keywork on a k8s cluster using a custom CA, the following error occurs :

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get issuer certificate (_ssl.c:1129)

Other keywords (Read Namespaced Pod Status, List Namespaced Pod By Pattern ...) are working as expected.

As a quick fix, I'm adding the following line in the _add_api method of the library :

def _add_api(self, reference, class_name):
    self.__dict__[reference] = class_name(self.api_client)
    if not self.cert_validation:
        self.__dict__[reference].api_client.rest_client.pool_manager.connection_pool_kw['cert_reqs'] = ssl.CERT_NONE
        self.__dict__[reference].api_client.configuration.verify_ssl = False

Am I missing something regarding the library configuration ?

Versions :

KubeLibrary: 0.8.0
Python: 3.9.13
Kubernetes: 1.24

KubeLibrary :

Library    KubeLibrary    kube_config=${KUBECONFIG_FILE}    cert_validation=False

KubeLibrary.Get Namespaced Pod Exec    
...    name=my-pod
...    namespace=${namespace}
...    argv_cmd=${command}

I was unable to get a working connection to a kubernetes cluster (v1.21.2) while using the environment variable HTTPS_PROXY, connectivity to the cluster is working for kubectl.

Looking at the code it would appear that it is only checking for http_proxy/HTTP_PROXY, and setting those environment variables also failed to successfully connect to the cluster.

        self.api_client.configuration.proxy = environ.get('http_proxy') or environ.get('HTTP_PROXY')

Removing the above code, and adding the following code instead after the config.load_kube_config(...) resolves the issue and proxy works just fine for KubeLibrary.

try:
    config.load_kube_config(kube_config, context)
    # fix for http proxy
    proxy_url = (
        environ.get("https_proxy", None)
        or environ.get("HTTPS_PROXY", None)
        or environ.get("http_proxy", None)
        or environ.get("HTTP_PROXY", None)
    )
    if proxy_url:
        client.Configuration._default.proxy = proxy_url
    # end of fix

Reach 90% unit test coverage and set CI threshold so that we don't go below it forcing creation of unit tests for each new keyword.

Add keyword(s) that could check failures on pod creation, i.e.:

• Unscheduled pods
• Pods that failed to create
• Container restarts

robotframework-kubelibrary==0.8.5

Noticed, that a test case was failing whilst we had an issue with our cluster dev environment, and that the the test produced the following output:

Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/KubeLibrary/KubeLibrary.py", line 807, in filter_pods_containers_statuses_by_name for container_status in pod.status.container_statuses: TypeError: 'NoneType' object is not iterable
While we know that there is an issue with the rabbitmq pod at this moment in time, I wasn't expecting the test to fail with this error. A snippit of the trace can be found here.

'status': {'conditions': [{'last_probe_time': None, 'last_transition_time': datetime.datetime(2023, 10, 3, 16, 37, 56, tzinfo=tzlocal()), 'message': '0/4 nodes are available: 1 node(s) had ' 'taint {node-role.kubernetes.io/master: ' "}, that the pod didn't tolerate, 1 " 'node(s) had taint ' '{node.kubernetes.io/disk-pressure: }, ' "that the pod didn't tolerate, 2 " "node(s) didn't match pod " 'affinity/anti-affinity rules, 2 ' "node(s) didn't match pod anti-affinity " 'rules.', 'reason': 'Unschedulable', 'status': 'False', 'type': 'PodScheduled'}], 'container_statuses': None, 'ephemeral_container_statuses': None, 'host_i_ps': None, 'host_ip': None, 'init_container_statuses': None, 'message': None, 'nominated_node_name': None, 'phase': 'Pending', 'pod_i_ps': None, 'pod_ip': None, 'qos_class': 'BestEffort', 'reason': None, 'resize': None, 'resource_claim_statuses': None, 'start_time': None}}] | '^rabbitmq$' ]

As can be seen in the data above, the container_statuses has returned as 'None', and this is then breaking the filter_pods_containers_statuses_by_name when it attempts to iterate with that result.

Can't say for sure how often we encounter this issue, and just something that I saw in our lab today.

Improve test coverage, by writing unit test for getting configmap based on a mock k8s object that can be generated like the one in

• Create test for generating object .json
• Write test that checks target keyword
• After coverage is up modify the CI threshold in .coveragerc to new value:

(...)
[report]
fail_under = 67

Apparently the only way to get access to for example the CNI configuration is through CustomObjectsApi.

Base on read_namespaced_ingress a keyword for listing Ingress objects in a namespace can be build.

The keyword can be similar to this:

Not find a function of list_namespaced_persistent_volume_claim_by_pattern.
Only find list_namespaced_persistent_volume_claim

We need to review and complete:

• keyword documentation to include all the parameters
• make sure README and user guides are complete and easy to follow
• Add/review guides for developers, contribution etc.

Following v0.8.4, http_proxy/https_proxy/HTTP_PROXY/HTTPS_PROXY are now well managed (#128, #129).
But it misses the no_proxy/NO_PROXY retrieval from env var and assignation on Configuration._default.no_proxy

Due to that I'm now facing a regression compared to v0.8.3 as the api server I must connect to is concerned by the no_proxy/NO_PROXY and proxy environnement usage is now enforced in v0.8.4.

My workaround for this case is to drop proxy setting before using KubeLibrary:

unset http_proxy;
unset HTTP_PROXY;
unset https_proxy;
unset HTTPS_PROXY;

Currently there is no support for custom resource definition.
We need at least add list and read, in order to test whether the required CRD exists for many test cases.

Improve test coverage, by writing unit test for getting configmap based on a mock k8s object that can be generated like the one in

• Create test for generating object .json
• Write test that checks target keyword
• After coverage is up modify the CI threshold in .coveragerc to new value:

(...)
[report]
fail_under = 67

By installing the API Gateway Ambassador to your Kubernetes Cluster a couple of custom resource definitions are installed.
Most important are Mappings and Hosts as those determine the access to your services inside your cluster.

The kubernetes client offers the following functions to list/read custom resource definitions
list_custom_resource_definition
read_custom_resource_definition
read_custom_resource_definition_status

Wrapping this kubernetes client funtion will give us the possiblity to list daemonsets.
list_namespaced_daemon_set

After upgrading to the latest 0.8.0 of kubelibrary I am now seeing the following error:

[ ERROR ] Error in file '/opt/robotframework/tests/k8s/smoke.robot' on line 3: Initializing library 'KubeLibrary' with arguments [ incluster=True ] failed: TypeError: openssl_md5() takes at most 1 argument (2 given)
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/KubeLibrary/KubeLibrary.py", line 96, in __init__
    self.reload_config(kube_config=kube_config, context=context, api_url=api_url, bearer_token=bearer_token,
  File "/usr/local/lib/python3.8/site-packages/KubeLibrary/KubeLibrary.py", line 267, in reload_config
    self._add_api('dynamic', DynamicClient)
  File "/usr/local/lib/python3.8/site-packages/KubeLibrary/KubeLibrary.py", line 270, in _add_api
    self.__dict__[reference] = class_name(self.api_client)
  File "/usr/local/lib/python3.8/site-packages/kubernetes/dynamic/client.py", line 84, in __init__
    self.__discoverer = discoverer(self, cache_file)
  File "/usr/local/lib/python3.8/site-packages/kubernetes/dynamic/discovery.py", line 224, in __init__
    Discoverer.__init__(self, client, cache_file)
  File "/usr/local/lib/python3.8/site-packages/kubernetes/dynamic/discovery.py", line 48, in __init__
    default_cachefile_name = 'osrcp-{0}.json'.format(hashlib.md5(default_cache_id, usedforsecurity=False).hexdigest())
TypeError: openssl_md5() takes at most 1 argument (2 given)

After installing with 'pip install robotframework-kubelibrary', the 'get_deployments_in_namespace' function is missing from the KubeLibrary.py file.

Using 'pip install robotframework-kubelibrary --upgrade' does not correct the problem.

There is a workaround to copy the KubeLibrary.py file from this repository and manually paste into the installation.

Wrapping this kubernetes client funtion will give us the possiblity to list cronjobs.
list_namespaced_cron_job

Add the keywords to create and delete ingress in a namespace.

Write keyword for listing and filtering existing secrets similar to:

It would be nice to test multiple versions of k8s in CI for main branch so that we are sure it works fine for multiple version before release

HI All,
I have posted this message as a comment but I think this will get a better exposure an issue is created to report this issue.
I am unable to get all cron jobs in a namespace by calling the keyword Get Cron Jobs In Namespace. It seems that the Python function get_cron_jobs_in_namespace can not be looked up from the keywords file cronjob_kw.robot. Is there a way to make the function work?

robot -e prerelease .
==============================================================================
Cronjob
==============================================================================
Cronjob.Cronjob
==============================================================================
Job test case example                                                 | FAIL |
No keyword with name 'Get Cron Jobs In Namespace' found. Did you mean:
    KubeLibrary.Get Jobs In Namespace
------------------------------------------------------------------------------
Cronjob.Cronjob                                                       | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
Cronjob                                                               | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================

Write keyword(s) for checking cluster level health, can be based on https://kubernetes.io/docs/reference/using-api/health-checks/.

Improve test coverage, by writing unit test for getting configmap based on a mock k8s object that can be generated like the one in

• Create test for generating object .json
• Write test that checks target keyword
• After coverage is up modify the CI threshold in .coveragerc to new value:

(...)
[report]
fail_under = 67

Add the keywords to find repplicasets available in a namespace

It should be great if library will inherit from DynamicCore/HybridCore (https://github.com/robotframework/PythonLibCore)

If you also decorate functions, that are keywords and you will not utilise @library(auto_keywords=True) you will get several features:

• Not all functions will be exposed (only that ones, that are definitely a keywords)
• Library will be much easier to use with other Dynamic/Hybrid libraries, event it will be possible to add this library as a plugin for other 'cloud based' libraries, or just to incorporate into different libraries
• You can also create 'plugin' system, that will be able to add new features to your library even without need to commit/change something in current implementation

Improve test coverage, by writing unit test for getting configmap based on a mock k8s object that can be generated like the one in

• Create test for generating object .json
• Write test that checks target keyword
• After coverage is up modify the CI threshold in .coveragerc to new value:

(...)
[report]
fail_under = 67

Add a keyword to create and delete the role , role binding in a namespace.

This could work similar to the kubectl paramater "--all-namespaces"
The kubernetes client offers often two functions, one for performing an operation on a specific namespace, and one for all namespaces.
e.g.
list_namespaced_role
and
list_role_for_all_namespaces

It would be handy to impelment a switch case for some keywords to use the function for all namespaces when for exmaple the namespace parameter contains the phrase "all_namespaces".

Keywords could be implemented based on the kubernetes client functions for
list_cluster_role
and
list_cluster_role_binding

I meant the same keyword as implemented for pods, i.e. Get Pod Names In Namespace - or one that simply returns a list of strings that corresponds to the list of deployment names that match the defined name_pattern. This would allow an easy way to iterate over deployment names in order to, for example, run a Delete keyword.

Not find a function of list_namespaced_stateful_set_by_pattern.

It would be great if this library could support the copy command to copy files to a pod. I foresee this keyword needing to know the following information (arguments):

• Namespace
• Filename(s) to copy
• Source Path (local)
• Destination Path (on the pod)
• Pod Name (the name of the pod to copy the files to)
• Timeout (the maximum length of time to wait before erroring out and moving on)

I'm already in the K8S cluster with proper context set via SSHLibrary Keywords in interactive shell. I want to avoid login kube config. how can i skip this. Please suggest.

Initializing library 'KubeLibrary' with arguments [ INIT_FOR_LIBDOC_ONLY=1 ] f
ailed: ConfigException: Invalid kube-config file. No configuration found.
Traceback (most recent call last):
File "/home/xx/.local/lib/python3.10/site-packages/KubeLibrary/KubeLibrary.py", line 97, in init
self.reload_config(kube_config=kube_config, context=context, api_url=api_url, bearer_token=bearer_token,
File "/home/xx/.local/lib/python3.10/site-packages/KubeLibrary/KubeLibrary.py", line 270, in reload_config
config.load_kube_config(kube_config, context)
File "/home/user/.local/lib/python3.10/site-packages/kubernetes/config/kube_config.py", line 815, in load_kube_config
loader = _get_kube_config_loader(
File "/home/xx/.local/lib/python3.10/site-packages/kubernetes/config/kube_config.py", line 772, in _get_kube_config_loader
raise ConfigException(
kubernetes.config.config_exception.ConfigException: Invalid kube-config file. No configuration found.

The objective is to create a keyword in KubeLibrary to fect replicaset in a namespace

keywords could be implemented based on the kubernetes client functions for
list_namespaced_role
and
list_namespaced_role_binding

Hi,

I have been trying to use the KubeLibrary tool for testing my kubernetes clusters. However, as my kubeconfig file has multiplle clusters declared, but a single current context defined the tool is unable to read the current-context in the kubeconfig file and use it to establish connection to a cluster thus throwing the following SSL error:

[ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get issuer certificate (_ssl.c:1123)'))': /k8s/clusters/local/api/v1/namespaces/cattle-system/pods?labelSelector=&watch=False

How can I establish connection if my kubeconfig has multiple clusters defined withing in ? Is there a way I could use a Bearer token to connect to K8s instead of using the kubeconfig file ?

Thanks !

Generating index.html should be done as a part of CI to avoid situations where docs are not updated.

Inspired by: https://snyk.io/advisor/python/robotframework-kubelibrary