openshift / managed-scripts

BackplaneM3-managed scripts

Backplane Managed scripts

This is a repository storing OpenShift Dedicated managed scripts.

Repository layout

scripts folder contains various scripts used by different teams.

hack contains various helper script for CI/CD tooling and building.

Each Red Hat managed role has a dedicated directory under root and each of the contains any number of scripts.

Each script directory has to contain a metadata.yaml file, the format of the metadata file is documented below.

Besides the metadata.yaml file, each directory should contain a single script file, written in one of the supported languages.

metadata.yaml

All metadata.yaml shall pass validation against hack/metadata.schema.json see here for more details.

Validation methods:

1. In order to check all metadata.yaml from all scripts, you can run the command below from your managed-scripts root directory:

   make validation

2. To run the validation just for specifics scripts, you can use the command below parsing the scripts as arguments:

   make validation SCRIPTS="<script1> <scriptN>"

   example:

   make validation SCRIPTS="under-replicated-partition rolling-restart-broker"

Release Cycle

Managed Scripts has the following release cycle:

Staging:

The staging environment is pinned to consume the main branch of the managed-scripts repository.

Production:

Once every 3 weeks.

In case you have changes that have immediate impact and would need an immediate promotion, please reach out to: Managed-scripts team (alias : @managed-scripts) in #sd-ims-backplane slack channel

managed-scripts's Issues

Build process fails locally using Podman

The build process fails locally because the tests fail. In the CI, the tests work using Docker, but locally they produce a permission error.

Traceback (most recent call last):
  File "/home/jsonschema/.local/bin/jsonschema", line 8, in <module>
    sys.exit(main())
  File "/home/jsonschema/.local/lib/python3.9/site-packages/jsonschema/cli.py", line 76, in main
    sys.exit(run(arguments=parse_args(args=args)))
  File "/home/jsonschema/.local/lib/python3.9/site-packages/jsonschema/cli.py", line 69, in parse_args
    arguments = vars(parser.parse_args(args=args or ["--help"]))
  File "/usr/local/lib/python3.9/argparse.py", line 1818, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/local/lib/python3.9/argparse.py", line 1851, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/local/lib/python3.9/argparse.py", line 2060, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/usr/local/lib/python3.9/argparse.py", line 2000, in consume_optional
    take_action(action, args, option_string)
  File "/usr/local/lib/python3.9/argparse.py", line 1912, in take_action
    argument_values = self._get_values(action, argument_strings)
  File "/usr/local/lib/python3.9/argparse.py", line 2443, in _get_values
    value = self._get_value(action, arg_string)
  File "/usr/local/lib/python3.9/argparse.py", line 2476, in _get_value
    result = type_func(arg_string)
  File "/home/jsonschema/.local/lib/python3.9/site-packages/jsonschema/cli.py", line 21, in _json_file
    with open(path) as file:
PermissionError: [Errno 13] Permission denied: '/json/./template/metadata.json'
make: *** [Makefile:29: validation] Error 1```

./hack/schema_validation.sh
convert ./template/metadata.yaml to json
convert succeed
convert ./scripts/CEE/get-kafka-instance-state/metadata.yaml to json
convert succeed
convert ./scripts/CEE/list-alerts/metadata.yaml to json
convert succeed
convert ./scripts/CEE/registry-s3-bucket-size/metadata.yaml to json
convert succeed
convert ./scripts/CEE/describe-kafka-transaction/metadata.yaml to json
convert succeed
convert ./scripts/CEE/pcap-collector/metadata.yaml to json
convert succeed
convert ./scripts/CEE/check-tech-preview-features/metadata.yaml to json
convert succeed
convert ./scripts/CEE/etcd-health-check/metadata.yaml to json
convert succeed
convert ./scripts/CEE/get-rhosak-operators/metadata.yaml to json
convert succeed
convert ./scripts/CSSRE/under-replicated-partitions/metadata.yaml to json
convert succeed
convert ./scripts/CSSRE/get-kafka-topics/metadata.yaml to json
convert succeed
convert ./scripts/CSSRE/rolling-restart-brokers/metadata.yaml to json
convert succeed
convert ./scripts/CSSRE/under-min-isr-partitions/metadata.yaml to json
convert succeed
convert ./scripts/CSSRE/get-node-restarts/metadata.yaml to json
convert succeed
convert ./scripts/SREP/manage-silence/metadata.yaml to json
convert succeed
convert ./scripts/SREP/mg-log-extractor/metadata.yaml to json
convert succeed
convert ./scripts/SREP/get-targets-down/metadata.yaml to json
convert succeed
convert ./scripts/SREP/node-logs/metadata.yaml to json
convert succeed
convert ./scripts/SREP/tsdb-status/metadata.yaml to json
convert succeed
convert ./scripts/SREP/replace-master/metadata.yaml to json
convert succeed
convert ./scripts/SREP/example/metadata.yaml to json
convert succeed
convert ./scripts/SREP/delete-olm-operator/metadata.yaml to json
convert succeed
convert ./scripts/SREP/cluster-operator-status/metadata.yaml to json
convert succeed
convert ./scripts/SREP/elasticsearch-status/metadata.yaml to json
convert succeed
convert ./scripts/SREP/mg-log-ls/metadata.yaml to json
convert succeed
validating the jsonschema for ./template/metadata.json
validation failed: ./template/metadata.json```

Rename default branch to `master`

Following our old conventions, we should consider reverting the default branch to master

Noisy output when using oc debug inside managed script

Hi team, can it be fixed the Warning message? It comes up whenever oc debug is run from inside a managedscript, and really make it noisy in the output of logs.

Warning: would violate PodSecurity "restricted:latest": host namespaces (hostNetwork=true, hostPID=true, hostIPC=true), privileged (container "container-00" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "container-00" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "container-00" must set securityContext.capabilities.drop=["ALL"]), restricted volume types (volume "host" uses restricted volume type "hostPath"), runAsNonRoot != true (pod or container "container-00" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "container-00" must not set runAsUser=0), seccompProfile (pod or container "container-00" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

Thank you!

Injection attack possibility in kafka.py.

Currently this library doesn't guard against the injection attacks. For instance, a malicious user could pass in a specially crafted kafka instance name or topic name (containing a white space) to cause undesired execution effects. Best practice tells us we should always code to avoid this possibility.

Building the cmd arrays explicitly, rather than splitting on whitespace, would be one possible resolution.

managed-scripts/scripts/CSSRE/lib/py/kafka.py

Line 91 in 3211e57

cmd = "-i statefulset/"+kafka+"-kafka -c kafka -- env - bin/kafka-topics.sh --bootstrap-server localhost:9096 --describe "

@robshelly

Add yq to command line tools

In our use-case (Managed Kafka) we are using oc adm inspect to grab kubernetes state. In some cases, we want to exclude some fields from the yaml output files generated by inspect. Unfortunately the oc adm inspect command line does not support a --output=json mode, so we cannot use jq.

Could you consider adding yq to the command line tools available to an ocm backplace script?

Happy to raise a PR.

2023 Change Freeze | branch:main

main branch is frozen for merges until Friday Jan-06-2023 @ 14:00 UTC