project-flotta / flotta-operator Goto Github PK

I am new to the Flotta project and the whole open source community, if there is something I am missing please correct me.

The Problem

when trying to install the project on a machine with Go 1.18 it gives me an error of bin/controller-gen: No such file or directory

running make install gives the following

go: creating new go.mod: module tmp
Downloading sigs.k8s.io/controller-tools/cmd/[email protected]
go: added github.com/fatih/color v1.7.0
go: added github.com/gobuffalo/flect v0.2.0
go: added github.com/gogo/protobuf v1.3.1
go: added github.com/google/gofuzz v1.1.0
go: added github.com/inconshreveable/mousetrap v1.0.0
go: added github.com/json-iterator/go v1.1.8
go: added github.com/mattn/go-colorable v0.1.2
go: added github.com/mattn/go-isatty v0.0.8
go: added github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: added github.com/modern-go/reflect2 v1.0.1
go: added github.com/spf13/cobra v1.0.0
go: added github.com/spf13/pflag v1.0.5
go: added golang.org/x/mod v0.2.0
go: added golang.org/x/net v0.0.0-20200226121028-0de0cce0169b
go: added golang.org/x/sys v0.0.0-20191022100944-742c48ecaeb7
go: added golang.org/x/text v0.3.2
go: added golang.org/x/tools v0.0.0-20200616195046-dc31b401abb5
go: added golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: added gopkg.in/inf.v0 v0.9.1
go: added gopkg.in/yaml.v2 v2.2.8
go: added gopkg.in/yaml.v3 v3.0.0-20190905181640-827449938966
go: added k8s.io/api v0.18.2
go: added k8s.io/apiextensions-apiserver v0.18.2
go: added k8s.io/apimachinery v0.18.2
go: added k8s.io/klog v1.0.0
go: added k8s.io/utils v0.0.0-20200324210504-a9aa75ae1b89
go: added sigs.k8s.io/controller-tools v0.4.1
go: added sigs.k8s.io/structured-merge-diff/v3 v3.0.0
go: added sigs.k8s.io/yaml v1.2.0
bash: line 1: /home/ateya/Desktop/flotta-operator/bin/controller-gen: No such file or directory
make: * [Makefile:63: manifests] Error 127

after some search, I found this issue in kubebuilder, they are not supporting Go 1.18 yet

Notes:

• I tested it with the Go 1.17 and it works fine.
• I think the root cause for this issue is the deprecation of go get command in Go 1.18

Proposed solutions

1. update the docs and wait until kubebuilder finishes their update to get better results

I think we can wait until they finish their work in supporting Go 1.18 (and I can see they are actively working on this, due to the open issues and PRs they have related to this) and then update our project based on what they will do. and in the meantime, we can update our docs here and here in the Prerequisites section that the project is supporting Go 1.18 yet, this could be a good first issue and I can work on this

2. fixing it ourselves

I tried to replace the command with go install here but it didn't work, gave me this error when running make install

go: creating new go.mod: module tmp
Downloading sigs.k8s.io/kustomize/kustomize/[email protected]
go: sigs.k8s.io/kustomize/kustomize/[email protected] (in sigs.k8s.io/kustomize/kustomize/[email protected]):
The go.mod file for the module providing named packages contains one or
more exclude directives. It must not contain directives that would cause
it to be interpreted differently than if it were the main module.
make: * [Makefile:192: kustomize] Error 1

The following branches are being fast-forwarded from the current development branch (main) as placeholders for future releases. No merging is allowed into these release branches until they are unfrozen for production release.

• release-4.17

For more information, see the branching documentation.

In order to move to latest libraries, go version needs to be updated as well to 1.17.

• controller-runtime changes

The image value in release 0.0.1 of the flotta operator for OCP points to the wrong image value:
image: controller:latest
The correct value should be:
image: quay.io/project-flotta/flotta-operator as it is shown in the K8s manifest.

deploy_graphana.sh script deploys grafana on OCP and imports flotta's dashboard.
We'd like to extend the script to work for k8s as well.

The current implementation of flotta's logging is done by watching a configmap.
When the log_level element of that map is changed, the pod is being restarted.
While this allows to apply new log verbosity level in all operators, there are less desired side-effects, such as
initializing the entire-cache (depends on the scale, can take sometime + introduce unnecessary load on etcd/api-server).

Using https://pkg.go.dev/go.uber.org/zap#AtomicLevel, the log verbosity could be changed on the fly.
We'd like to explore that approach and to minimize the restarted when can be avoided.

See operator-framework/operator-sdk#4684

I experience the error:

Error: container create failed: time="2021-10-13T02:11:50Z" level=error msg="container_linux.go:366: starting container process caused: chdir to cwd (\"/home/nonroot\") set in config.json failed: permission denied"

Recent client-go introduced a retry watcher that can replace some of the internal implementation of the configmap_watcher.
One of its benefit is not loosing any events, so current implementation might miss an event of updating the configmap.
With retrywatcher, this shouldn't happen.

Port 9100 on devices is reserved for node_exporter service:

[root@fedora ~]# netstat -tunlp | grep node_exporter
tcp        0      0 127.0.0.1:9100          0.0.0.0:*               LISTEN      917/node_exporter

Therefore validation is required for edge-workload not to create one using this port.

Once #164 is merged, swagger.yaml can be validated by running:

make validate-swagger

We'd like to include it as part of the build rule. However, in order to do so, first reported issues need to be fixed:

make validate-swagger 
docker run -it --rm -v /work/go/src/github.com/project-flotta/flotta-operator/.spectral.yaml:/tmp/.spectral.yaml -v /work/go/src/github.com/project-flotta/flotta-operator/swagger.yaml:/tmp/swagger.yaml stoplight/spectral lint --ruleset "/tmp/.spectral.yaml" /tmp/swagger.yaml 

/tmp/swagger.yaml
   1:1   warning  oas2-api-host           OpenAPI "host" must be present and non-empty string.
   2:6   warning  info-contact            Info object must have "contact" object.                        info
  30:9   warning  operation-description   Operation "description" must be present and non-empty string.  paths./data/{device_id}/in.get
  33:11  warning  operation-tag-defined   Operation tags must be defined in global tags.                 paths./data/{device_id}/in.get.tags[0]
  55:10  warning  operation-description   Operation "description" must be present and non-empty string.  paths./data/{device_id}/out.post
  58:11  warning  operation-tag-defined   Operation tags must be defined in global tags.                 paths./data/{device_id}/out.post.tags[0]
  87:9   warning  operation-description   Operation "description" must be present and non-empty string.  paths./control/{device_id}/in.get
  90:11  warning  operation-tag-defined   Operation tags must be defined in global tags.                 paths./control/{device_id}/in.get.tags[0]
 112:10  warning  operation-description   Operation "description" must be present and non-empty string.  paths./control/{device_id}/out.post
 115:11  warning  operation-tag-defined   Operation tags must be defined in global tags.                 paths./control/{device_id}/out.post.tags[0]
 140:32  warning  oas2-unused-definition  Potentially unused definition has been detected.               definitions.device-configuration-message
 152:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.device-configuration-message.properties.workloads.description
 160:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.device-configuration-message.properties.secrets.description
 175:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.device-configuration.properties.os.description
 180:24    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.device-configuration.properties.log-collection.additionalProperties.description
 260:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.metrics-configuration.properties.retention.description
 263:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.metrics-configuration.properties.system.description
 286:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.system-metrics-configuration.properties.allow_list.description
 318:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.workload.properties.data.description
 321:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.workload.properties.imageRegistries.description
 324:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.workload.properties.metrics.description
 328:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.workload.properties.configmaps.description
 364:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.metrics.properties.allow_list.description
 373:24    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.metrics.properties.containers.additionalProperties.description
 416:25  warning  oas2-unused-definition  Potentially unused definition has been detected.               definitions.registration-response
 423:21  warning  oas2-unused-definition  Potentially unused definition has been detected.               definitions.registration-info
 427:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.registration-info.properties.hardware.description
 604:13  warning  oas2-unused-definition  Potentially unused definition has been detected.               definitions.heartbeat
 619:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.heartbeat.properties.upgrade.description
 622:22    error  no-$ref-siblings        $ref must not be placed next to any other properties           definitions.heartbeat.properties.hardware.description

✖ 30 problems (16 errors, 14 warnings, 0 infos, 0 hints)

make: *** [Makefile:201: validate-swagger] Error 1

For example the following EdgeWorkload:
(example source https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/)

---
apiVersion: management.project-flotta.io/v1alpha1
kind:EdgeWorkload
metadata:
  name: greeting-with-env
spec:
  deviceSelector:
    matchLabels:
      app: ee-ansible
  type: pod
  pod:
    spec:
      containers:
        - name: greeting-with-env-container
          image: gcr.io/google-samples/node-hello:1.0
          env:
          - name: DEMO_GREETING
            value: "Hello from the environment"
          - name: DEMO_FAREWELL
            value: "Such a sweet sorrow"

Produces the error:

Jul 30 17:12:28 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:28 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:52: sending HTTP request: GET https://project-flotta.io:8043/api/flotta-management/v1/control/edgedevice1/in
Jul 30 17:12:28 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:28 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:53: request: &{GET https://project-flotta.io:8043/api/flotta-management/v1/control/edgedevice1/in HTTP/1.1 1 1 map[User-Agent:[yggdrasil/0.2.99]] <nil> <nil> 0 [] false project-flotta.io:8043 map[] map[] <nil> map[]   <nil> <nil> <nil> 0xc0000b2020}
Jul 30 17:12:28 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:28 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:99: received HTTP 200 OK:
Jul 30 17:12:28 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:28 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:52: sending HTTP request: GET https://project-flotta.io:8043/api/flotta-management/v1/data/edgedevice1/in
Jul 30 17:12:28 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:28 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:53: request: &{GET https://project-flotta.io:8043/api/flotta-management/v1/data/edgedevice1/in HTTP/1.1 1 1 map[User-Agent:[yggdrasil/0.2.99]] <nil> <nil> 0 [] false project-flotta.io:8043 map[] map[] <nil> map[]   <nil> <nil> <nil> 0xc0000b2020}
Jul 30 17:12:28 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:28 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/transport/http.go:71: cannot get HTTP request: cannot download from URL: Get "https://project-flotta.io:8043/api/flotta-management/v1/data/edgedevice1/in": stream error: stream ID 1707; INTERNAL_ERROR; received from peer
+++Jul 30 17:12:33 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:33 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:52: sending HTTP request: GET https://project-flotta.io:8043/api/flotta-management/v1/control/edgedevice1/in
Jul 30 17:12:33 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:33 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:53: request: &{GET https://project-flotta.io:8043/api/flotta-management/v1/control/edgedevice1/in HTTP/1.1 1 1 map[User-Agent:[yggdrasil/0.2.99]] <nil> <nil> 0 [] false project-flotta.io:8043 map[] map[] <nil> map[]   <nil> <nil> <nil> 0xc0000b2020}
Jul 30 17:12:33 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:33 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:99: received HTTP 200 OK:
Jul 30 17:12:33 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:33 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:52: sending HTTP request: GET https://project-flotta.io:8043/api/flotta-management/v1/data/edgedevice1/in
Jul 30 17:12:33 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:33 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/http/client.go:53: request: &{GET https://project-flotta.io:8043/api/flotta-management/v1/data/edgedevice1/in HTTP/1.1 1 1 map[User-Agent:[yggdrasil/0.2.99]] <nil> <nil> 0 [] false project-flotta.io:8043 map[] map[] <nil> map[]   <nil> <nil> <nil> 0xc0000b2020}
Jul 30 17:12:33 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:33 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/internal/transport/http.go:71: cannot get HTTP request: cannot download from URL: Get "https://project-flotta.io:8043/api/flotta-management/v1/data/edgedevice1/in": stream error: stream ID 1711; INTERNAL_ERROR; received from peer
Jul 30 17:12:36 ece4fdf0da6d yggdrasild[124]: [yggdrasild] 2022/07/30 17:12:36 /builddir/build/BUILD/yggdrasil-0.2.99-0.86.git.3eb009b/cmd/yggd/worker.go:114: /usr/libexec/yggdrasil/device-worker: no workloads to return. DeviceID: edgedevice1;  

Removing the env section "resolve" the problem.

However, the environment variables can be successfully be defined via a ConfigMap.

swagger.yaml defines the API between edge-devices and the control-plane.
The validity of that file isn't verified.
There are tools such as spectral that can be added as a job to validate that file.