Comments (11)
debug mod does not reveal much
2021-05-31T09:06:37.980Z DEBUG setup manager/main.go:110 Confluent Telemetry Reporter is not enabled
I0531 09:06:39.031502 1 request.go:621] Throttling request took 1.044329091s, request: GET:https://172.30.0.1:443/apis/whereabouts.cni.cncf.io/v1alpha1?timeout=32s
2021-05-31T09:06:40.988Z INFO controller-runtime.metrics manager/main.go:138 metrics server is starting to listen {"addr": ":7778"}
2021-05-31T09:06:40.989Z INFO runtime/proc.go:203 OpenShift route API exist
2021-05-31T09:06:40.989Z DEBUG setup runtime/asm_amd64.s:1357 PPROF started {"port": "6060"}
2021-05-31T09:06:40.991Z INFO setup runtime/proc.go:203 Starting Confluent Operator {"version": "v0.174.6", "build_time": "2021-05-11T00:44:09Z", "kubernetes_version": "v1.20.0+75370d3"}
2021-05-31T09:06:41.095Z INFO controller-runtime.manager manager/internal.go:676 starting metrics server {"path": "/metrics"}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Kafka", "controller": "kafka", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KafkaTopic", "controller": "kafkatopic", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Connect", "controller": "connect", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Zookeeper", "controller": "zookeeper", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KafkaRestClass", "controller": "kafkarestclass", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "SchemaRegistry", "controller": "schemaregistry", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KsqlDB", "controller": "ksqldb", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "MigrationJob", "controller": "migrationjob", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "ConfluentRolebinding", "controller": "confluentrolebinding", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.096Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "ControlCenter", "controller": "controlcenter", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.197Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Kafka", "controller": "kafka", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.197Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Kafka", "controller": "kafka", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.198Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KsqlDB", "controller": "ksqldb", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.198Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KsqlDB", "controller": "ksqldb", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.199Z INFO controller manager/internal.go:676 Starting Controller {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KafkaTopic", "controller": "kafkatopic"}
2021-05-31T09:06:41.199Z INFO controller manager/internal.go:676 Starting Controller {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "ConfluentRolebinding", "controller": "confluentrolebinding"}
2021-05-31T09:06:41.245Z INFO controller manager/internal.go:676 Starting Controller {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "MigrationJob", "controller": "migrationjob"}
2021-05-31T09:06:41.245Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Zookeeper", "controller": "zookeeper", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.245Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KafkaRestClass", "controller": "kafkarestclass", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.345Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "ControlCenter", "controller": "controlcenter", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.345Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "ControlCenter", "controller": "controlcenter", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.346Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "SchemaRegistry", "controller": "schemaregistry", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.346Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Connect", "controller": "connect", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.346Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "SchemaRegistry", "controller": "schemaregistry", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.347Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Connect", "controller": "connect", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.347Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Kafka", "controller": "kafka", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.348Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Zookeeper", "controller": "zookeeper", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.449Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Zookeeper", "controller": "zookeeper", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.699Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KsqlDB", "controller": "ksqldb", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.699Z INFO controller manager/internal.go:676 Starting workers {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KafkaTopic", "controller": "kafkatopic", "worker count": 1}
2021-05-31T09:06:41.699Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KsqlDB", "controller": "ksqldb", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.699Z INFO controller manager/internal.go:676 Starting workers {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "ConfluentRolebinding", "controller": "confluentrolebinding", "worker count": 1}
2021-05-31T09:06:41.700Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KsqlDB", "controller": "ksqldb", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.700Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "internal-schemaregistry-1", "namespace": "confluent"}
2021-05-31T09:06:41.700Z INFO kafkatopic controller/controller.go:244 reconcile triggered {"name": "elastic-0", "namespace": "confluent"}
2021-05-31T09:06:41.700Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "internal-schemaregistry-1", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.700Z DEBUG kafkatopic controllers/kafkarestclass_controller.go:215 trying to discover kafka cluster... {"name": "elastic-0", "namespace": "confluent", "in-namespace": "confluent"}
2021-05-31T09:06:41.700Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "internal-schemaregistry-1", "namespace": "confluent"}
2021-05-31T09:06:41.701Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "internal-connect-0", "namespace": "confluent"}
2021-05-31T09:06:41.745Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "internal-connect-0", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.745Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "internal-connect-0", "namespace": "confluent"}
2021-05-31T09:06:41.745Z INFO controller manager/internal.go:676 Starting workers {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "MigrationJob", "controller": "migrationjob", "worker count": 1}
2021-05-31T09:06:41.745Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "internal-connect-2", "namespace": "confluent"}
2021-05-31T09:06:41.745Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "internal-connect-2", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.745Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "internal-connect-2", "namespace": "confluent"}
2021-05-31T09:06:41.745Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "internal-controlcenter-0", "namespace": "confluent"}
2021-05-31T09:06:41.745Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "internal-controlcenter-0", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.745Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "internal-controlcenter-0", "namespace": "confluent"}
2021-05-31T09:06:41.746Z INFO controller manager/internal.go:676 Starting Controller {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "KafkaRestClass", "controller": "kafkarestclass"}
2021-05-31T09:06:41.746Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "testadmin-rb", "namespace": "confluent"}
2021-05-31T09:06:41.746Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "testadmin-rb", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.746Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "testadmin-rb", "namespace": "confluent"}
2021-05-31T09:06:41.746Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "testadmin-rb-connect", "namespace": "confluent"}
2021-05-31T09:06:41.746Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "testadmin-rb-connect", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.746Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "testadmin-rb-connect", "namespace": "confluent"}
2021-05-31T09:06:41.746Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "internal-schemaregistry-0", "namespace": "confluent"}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "internal-schemaregistry-0", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "internal-schemaregistry-0", "namespace": "confluent"}
2021-05-31T09:06:41.747Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "testadmin-rb-sr", "namespace": "confluent"}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "testadmin-rb-sr", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "testadmin-rb-sr", "namespace": "confluent"}
2021-05-31T09:06:41.747Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "internal-ksqldb-0", "namespace": "confluent"}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "internal-ksqldb-0", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.747Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "ControlCenter", "controller": "controlcenter", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "internal-ksqldb-0", "namespace": "confluent"}
2021-05-31T09:06:41.747Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "testadmin-rb-ksql", "namespace": "confluent"}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "testadmin-rb-ksql", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "testadmin-rb-ksql", "namespace": "confluent"}
2021-05-31T09:06:41.747Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "internal-connect-1", "namespace": "confluent"}
2021-05-31T09:06:41.747Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Connect", "controller": "connect", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "internal-connect-1", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "internal-connect-1", "namespace": "confluent"}
2021-05-31T09:06:41.747Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "ControlCenter", "controller": "controlcenter", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.747Z INFO confluent-rolebinding controller/controller.go:244 confluentrolebinding reconcile triggered {"name": "internal-ksqldb-1", "namespace": "confluent"}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controllers/confluentrolebinding_controller.go:413 trying to discover kafkarestclass... {"name": "internal-ksqldb-1", "namespace": "confluent", "in-namespace": "confluent", "in-name": "default"}
2021-05-31T09:06:41.747Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "ControlCenter", "controller": "controlcenter", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.747Z DEBUG confluent-rolebinding controller/controller.go:244 skipping reconcile {"name": "internal-ksqldb-1", "namespace": "confluent"}
2021-05-31T09:06:41.747Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Connect", "controller": "connect", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.748Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Connect", "controller": "connect", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.748Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "SchemaRegistry", "controller": "schemaregistry", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.748Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Kafka", "controller": "kafka", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.748Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Kafka", "controller": "kafka", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.749Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "SchemaRegistry", "controller": "schemaregistry", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.749Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "SchemaRegistry", "controller": "schemaregistry", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.749Z INFO controller manager/internal.go:676 Starting EventSource {"reconcilerGroup": "platform.confluent.io", "reconcilerKind": "Zookeeper", "controller": "zookeeper", "source": "kind source: /, Kind="}
2021-05-31T09:06:41.764Z ERROR kafkatopic controller/controller.go:244 apply failed {"name": "elastic-0", "namespace": "confluent", "error": "kafka rest API request failed: Unauthorized (0)"}
github.com/go-logr/zapr.(*zapLogger).Error
/build/vendor/github.com/go-logr/zapr/zapr.go:132
github.com/confluentinc/confluent-operator/pkg/log.operatorLogger.Error
/build/pkg/log/log.go:35
github.com/confluentinc/confluent-operator/controllers.(*KafkaTopicReconciler).Reconcile
/build/controllers/kafkatopic_controller.go:114
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:244
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:218
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:197
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.Until
/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90
2021-05-31T09:06:41.764Z INFO kafkatopic controller/controller.go:244 reconcile triggered {"name": "elastic-1", "namespace": "confluent"}
2021-05-31T09:06:41.764Z DEBUG controller-runtime.manager.events runtime/asm_amd64.s:1357 Warning {"object": {"kind":"KafkaTopic","namespace":"confluent","name":"elastic-0","uid":"f9418110-ad5e-4c42-a5a7-b402efd6f887","apiVersion":"platform.confluent.io/v1beta1","resourceVersion":"2295292"}, "reason": "Warning", "message": "kafka rest API request failed: Unauthorized (0)"}
2021-05-31T09:06:41.764Z DEBUG kafkatopic controllers/kafkarestclass_controller.go:215 trying to discover kafka cluster... {"name": "elastic-1", "namespace": "confluent", "in-namespace": "confluent"}
2021-05-31T09:06:41.784Z ERROR kafkatopic controller/controller.go:244 apply failed {"name": "elastic-1", "namespace": "confluent", "error": "kafka rest API request failed: Unauthorized (0)"}
github.com/go-logr/zapr.(*zapLogger).Error
/build/vendor/github.com/go-logr/zapr/zapr.go:132
github.com/confluentinc/confluent-operator/pkg/log.operatorLogger.Error
/build/pkg/log/log.go:35
github.com/confluentinc/confluent-operator/controllers.(*KafkaTopicReconciler).Reconcile
/build/controllers/kafkatopic_controller.go:114
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:244
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:218
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
/build/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:197
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.Until
/build/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90
from confluent-kubernetes-examples.
Tried curl inside the operator pod against REST API:
First one with bad credentials
sh-4.4$ curl -u "kafka:kafka-secre" https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/ --insecure
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 401 Unauthorized</title>
</head>
<body><h2>HTTP ERROR 401 Unauthorized</h2>
<table>
<tr><th>URI:</th><td>/kafka/v3/clusters/</td></tr>
<tr><th>STATUS:</th><td>401</td></tr>
<tr><th>MESSAGE:</th><td>Unauthorized</td></tr>
<tr><th>SERVLET:</th><td>default</td></tr>
</table>
</body>
</html>
And the second one with good credentials
sh-4.4$ curl -u "kafka:kafka-secret" https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/ --insecure
{"kind":"KafkaClusterList","metadata":{"self":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters","next":null},"data":[{"kind":"KafkaCluster","metadata":{"self":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w","resource_name":"crn:///kafka=3QV_o-TsS26FDdenfi-o_w"},"cluster_id":"3QV_o-TsS26FDdenfi-o_w","controller":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/brokers/1"},"acls":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/acls"},"brokers":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/brokers"},"broker_configs":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/broker-configs"},"consumer_groups":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/consumer-groups"},"topics":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/topics"},"partition_reassignments":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/topics/-/partitions/-/reassignment"}}]}
from confluent-kubernetes-examples.
KafkaRestClass spec:
spec:
kafkaRest:
authentication:
bearer:
secretRef: rest-bearer
type: bearer
rest-bearer secret
kind: Secret
apiVersion: v1
metadata:
selfLink: /api/v1/namespaces/confluent/secrets/rest-bearer
resourceVersion: '2343011'
name: rest-bearer
uid: 7010deca-2fa6-4963-bb5b-6d5a1b4680b8
creationTimestamp: '2021-05-31T09:03:12Z'
namespace: confluent
ownerReferences:
- apiVersion: platform.confluent.io/v1beta1
kind: KafkaRestClass
name: default
uid: c0078d28-6f94-401a-85e2-0865ea94535f
finalizers:
- secret.finalizers.platform.confluent.io/kafkarestclass-default
data:
bearer.txt: dXNlcm5hbWU9a2Fma2EKcGFzc3dvcmQ9a2Fma2Etc2VjcmV0
type: Opaque
from confluent-kubernetes-examples.
Also I can curl with the Bearer Token that I get from Control Center
curl https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/ --insecure -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6bnVsbH0.eyJqdGkiOiJ5ZXVpUEJfOG5IVGwxWTVzXzVKdXVnIiwiaXNzIjoiQ29uZmx1ZW50Iiwic3ViIjoidGVzdGFkbWluIiwiZXhwIjoxNjIyNTM1Mjc0LCJpYXQiOjE2MjI1MzE2NzQsIm5iZiI6MTYyMjUzMTYxNCwiYXpwIjoidGVzdGFkbWluIiwiYXV0aF90aW1lIjoxNjIyNTMxNjc0fQ.oF7oCW23t2LeyKdY_z2R4xC5ZclqveCenqfjdZZ7XViL0VZAzoY9pEoyU0r9_HpbxugRizCRO8mm8AIt3YWI8yvZn3iry0g_kLdOApm8xLiSqTaOR-EyASItEt8PlKR-4j7XhfC4GLNLCS4J7bxlgWMKafzsLjxIcbpS1letuXRzyQddpAZMqVX4F00t4aBXYYLmWjecerx0ZYffc7Hs26CRL3QixO_1-YtReAT-MKhLD7SSYywBJu-gZh4v6cam5cr0jHfVdXSYqj3yX0fMa_xz9AI2aXz2HZFmKPfOnANmbfLzb8ag8u6-Wwcdbw0MM_0ael1DO_HcRRlyoDjKeQ"
{"kind":"KafkaClusterList","metadata":{"self":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters","next":null},"data":[{"kind":"KafkaCluster","metadata":{"self":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w","resource_name":"crn:///kafka=3QV_o-TsS26FDdenfi-o_w"},"cluster_id":"3QV_o-TsS26FDdenfi-o_w","controller":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/brokers/1"},"acls":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/acls"},"brokers":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/brokers"},"broker_configs":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/broker-configs"},"consumer_groups":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/consumer-groups"},"topics":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/topics"},"partition_reassignments":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/topics/-/partitions/-/reassignment"}}]}```
from confluent-kubernetes-examples.
kafka user can create topics. no problem there, it has rights to do so.
curl -XPOST https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/topics --insecure -u "kafka:kafka-secret" -H "Content-Type: application/json" -d '{"topic_name":"topic-0","partitions_count":3,"replication_factor":3,"configs":[{"name":"cleanup.policy","value":"delete"}]}'
{"kind":"KafkaTopic","metadata":{"self":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/topics/topic-0","resource_name":"crn:///kafka=3QV_o-TsS26FDdenfi-o_w/topic=topic-0"},"cluster_id":"3QV_o-TsS26FDdenfi-o_w","topic_name":"topic-0","is_internal":false,"replication_factor":3,"partitions":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/topics/topic-0/partitions"},"configs":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/topics/topic-0/configs"},"partition_reassignments":{"related":"https://kafka.confluent.svc.cluster.local:8090/kafka/v3/clusters/3QV_o-TsS26FDdenfi-o_w/topics/topic-0/partitions/-/reassignment"}}
from confluent-kubernetes-examples.
All these concludes that somehow Operator either picks wrong values from the Kubernetes secret or something wrong with the topic creation part of the code.
Maybe secret's data is wrong, but the documentation is not crystal clear about it neither.
from confluent-kubernetes-examples.
@omersiar could you also share your topic.yaml? since in your kafkaRestClass, you are using secret rest-bearer
, could you help make sure topic also uses the right credential to create? thanks
from confluent-kubernetes-examples.
Isn't it a little weird?
When I reference the kafkaRestClassRef as below
apiVersion: platform.confluent.io/v1beta1
kind: KafkaTopic
metadata:
name: elastic-1
namespace: confluent
finalizers:
- kafkatopic.finalizers.platform.confluent.io
spec:
configs:
cleanup.policy: delete
kafkaRestClassRef:
name: default
namespace: confluent
partitionCount: 1
replicas: 1
Operator prints an error:
"msg":"apply failed","name":"elastic-1","namespace":"confluent","error":"missing .spec.kafkaRest.tls.secretRef in spec"
This reference has everything for operator to apply the configuration, -endpoint, -tls, -credentials
Why it is looking at kafkaRest spec?
from confluent-kubernetes-examples.
@omersiar could you also share your topic.yaml? since in your kafkaRestClass, you are using secret
rest-bearer
, could you help make sure topic also uses the right credential to create? thanks
It did not help:
$ kubectl explain kafkaTopic.spec.kafkaRest.tls.secretRef
KIND: KafkaTopic
VERSION: platform.confluent.io/v1beta1
FIELD: secretRef <string>
DESCRIPTION:
SecretRef defines the secret name referenced for certificates More
information about certificates key/value format can be found in Confluent
Operator document
And it is also for the TLS certificates and I think it is nothing to do with credentials.
from confluent-kubernetes-examples.
Ok figured out.
kafkaRestClass needs tls.secretRef as well. Thanks for your help. @yangpei1214
from confluent-kubernetes-examples.
@omersiar tls is not required, it's optional, if you set tls.enabled true, then you will need to have tls secretRef.
from confluent-kubernetes-examples.
Related Issues (20)
- kafka not able to connect to zookeeper HOT 2
- connect: connection refused from kafka server (got this issue without any code changes)
- LDAP custom users not able to authenticate on control center
- DOCS update - fix available for kafka not starting
- LDAP authentication into control center not working when bindDn and bindPassword are configured using secrets HOT 1
- what about multiple kafka cluster addresses?there is only one kafka address can be set HOT 1
- FailedToRetrieveImagePullSecret for CFK confluent-for-kubernetes-0.824.33
- Labels not discovered from Prometheus -> Grafana
- Problems running the example with selfsigned cert-manager
- Confluent for Kubernetes (CFK) on Red Hat OpenShift Kubernetes Cluster
- Headless service for Zookeeper or Kraftcontrollers are not reachable
- grafana dashboards not working for confluent platform HOT 1
- kraftmigrationjob crd not getting deployed
- kubernetes cluster is not creating object connect-0
- Connect startup error with confluent-platform-zookeeper-7.6.0.yaml HOT 3
- Pod kraftcontroller-0 in StatefulSet kraftcontroller failed error: Pod "kraftcontroller-0" is invalid: spec.initContainers[0].name: Duplicate value: "config-init-container" HOT 1
- Confluent Control Center stops working after a couple of hours
- Schemaregistry ERROR Timed out waiting for join group to complete
- Security examples missing KRaft setup HOT 2
- kafkarestclass kafka cluster [kafka] in not in running state
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from confluent-kubernetes-examples.