consdata / kouncil Goto Github PK

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

2023-03-13 14:57:54 2023-03-13 11:57:54.771  WARN 1 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'authController': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'authenticationManager' available

I used the latest Docker image:

consdata/kouncil                   latest    b0c8f4bb29ef   4 days ago      529MB

Does kouncil supports TLS (SSL) and SASL for encryption and authentication?

[main] c.c.kouncil.config.KouncilConfiguration :
KouncilConfiguration(initialBootstrapServers=[172.23.0.20:9092, 172.23.0.21:9094, 172.23.0.22:9096],
clusters=
[ClusterConfig(name=kafka-n, jmxPort=null, jmxUser=null, jmxPassword=null,
brokers=
BrokerConfig(host=172.23.0.20, port=9092, jmxPort=null, jmxUser=null, jmxPassword=null),
BrokerConfig(host=172.23.0.21, port=9093, jmxPort=null, jmxUser=null, jmxPassword=null),
BrokerConfig(host=172.23.0.22, port=9094, jmxPort=null, jmxUser=null, jmxPassword=null)])],

clusterConfig={kafka_ndsal=ClusterConfig(name=kafka-n, jmxPort=null, jmxUser=null, jmxPassword=null,

brokers=[BrokerConfig(host=172.23.0.20, port=9092, jmxPort=null, jmxUser=null, jmxPassword=null),
BrokerConfig(host=172.23.0.21, port=9093, jmxPort=null, jmxUser=null, jmxPassword=null), BrokerConfig(host=172.23.0.22, port=9094, jmxPort=null, jmxUser=null, jmxPassword=null)])})

**All fine but then this? Why does it search in localhost while i changed the ip-address doesn't make any sense. I can't change the IP **
`
2021-12-08 15:09:32.081 WARN 1 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient : [AdminClient clientId=adminclient-1] Connection to node 2 (localhost/127.0.0.1:9094) could not be established. Broker may not be available.

2021-12-08 15:09:32.084 WARN 1 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient : [AdminClient clientId=adminclient-1] Connection to node 3 (localhost/127.0.0.1:9096) could not be established. Broker may not be available.

2021-12-08 15:09:32.086 WARN 1 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient : [AdminClient clientId=adminclient-1] Connection to node 1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
`

When you build the Front end using POM the Front end adds proxy from settings.xml and installation Fails for Yarn. as shown below :
INFO] --- frontend-maven-plugin:1.6:yarn (yarn run lint) @ kouncil-frontend ---
[INFO] Found proxies: [default{protocol='https', host='http-proxy.ntrs.com', port=80}]
[INFO] Running 'yarn lint --https-proxy=http://http-proxy.ntrs.com:80 --proxy=http://http-proxy.ntrs.com:80' in /Users/AD472/anil/kouncil-kouncil-1.2/kouncil-frontend
[INFO] yarn run v1.22.18
[ERROR] warning ../../../package.json: No license field
[INFO] $ nx lint --https-proxy=http://http-proxy.ntrs.com:80 --proxy=http://http-proxy.ntrs.com:80
[INFO]
[INFO] > nx run kouncil:lint --httpsProxy=http://http-proxy.ntrs.com:80 --proxy=http://http-proxy.ntrs.com:80
[INFO]
[ERROR] 'httpsProxy' is not found in schema
[INFO]
[INFO]
[INFO]
[INFO] > NX Running target "kouncil:lint" failed
[INFO]
[INFO] Failed tasks:
[INFO]
[INFO] - kouncil:lint
[INFO]
[INFO] Hint: run the command with --verbose for more details.
[INFO]
[ERROR] error Command failed with exit code 1.
[INFO] info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.908 s
[INFO] Finished at: 2022-04-20T21:00:50-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.6:yarn (yarn run lint) on project kouncil-frontend: Failed to run task: 'yarn lint --https-proxy=http://http-proxy.ntrs.com:80 --proxy=http://http-proxy.ntrs.com:80' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
What is the resolution for this ?
To work around we need to add yarnInheritsProxyConfigFromMaven as shown below so that Proxy configurations are not inherited from settings.xml



com.github.eirslett
frontend-maven-plugin
1.6

false
target

https://docs.confluent.io/platform/current/kafka/authentication_sasl/index.html#jaas-configurations

mvn spring-boot:run -Dspring-boot.run.arguments="bootstrapServers=localhost:9092"

For a production cluster....

Description
When using Kouncil UI 1.5, the filter field is not responding as it used to do. When something is typed, the filter does not get triggered properly.

Expected Result
The filter should be triggered (so the results must be shown) while typing in the filter field like it was with Kouncil 1.3.

Workaround
When typing a search string in the search-box and then pressing space followed by backspace, the search result is updated.

Kouncil version
Docker: consdata/kouncil:1.5

Browsers
Chrome Version 111.0.5563.65 (Official Build) (64-bit); Firefox 110.0.1 (64-Bit)

Question
Is there anything you can do about it? Or has this behavior been removed on purpose?

I want to change the default internal port from 8080 to something different, for instance 9000

Only json message is displayed. For non json message raw view should be presented

This features should be enabled when companion is running in "dev" mode.

Not getting Dashboard

Shown below are the Logs

/\ / ' __ _ () __ __ _ \ \ \
( ( )__ | '_ | '| | ' / ` | \ \ \
\/ )| |)| | | | | || (| | ) ) ) )
' || .__|| ||| |_, | / / / /
=========||==============|/=////
:: Spring Boot :: (v2.5.12)

2022-04-14 18:32:39.594 INFO 2364 --- [ main] com.consdata.kouncil.KouncilApplication : Starting KouncilApplication using Java 11.0.1 on DESKTOP-IVPJJN1 with PID 2364 (C:\anil\kouncil-kouncil-1.2\kouncil-backend\target\classes started by C-SNP3897 in C:\anil\kouncil-kouncil-1.2\kouncil-backend)
2022-04-14 18:32:39.596 DEBUG 2364 --- [ main] com.consdata.kouncil.KouncilApplication : Running with Spring Boot v2.5.12, Spring v5.3.18
2022-04-14 18:32:39.598 INFO 2364 --- [ main] com.consdata.kouncil.KouncilApplication : No active profile set, falling back to 1 default profile: "default"
2022-04-14 18:32:41.395 INFO 2364 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8081 (http)
2022-04-14 18:32:41.404 INFO 2364 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2022-04-14 18:32:41.405 INFO 2364 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.60]
2022-04-14 18:32:41.532 INFO 2364 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2022-04-14 18:32:41.533 INFO 2364 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1885 ms
2022-04-14 18:32:41.675 INFO 2364 --- [ main] c.c.kouncil.config.KouncilConfiguration : Using simple Kouncil configuration: bootstrapServers=[], schemaRegistryUrl=
2022-04-14 18:32:41.682 INFO 2364 --- [ main] c.c.kouncil.config.KouncilConfiguration : KouncilConfiguration(initialBootstrapServers=[], schemaRegistryUrl=, clusters=null, clusterConfig={})
2022-04-14 18:32:42.190 INFO 2364 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 2 endpoint(s) beneath base path '/actuator'
2022-04-14 18:32:42.293 WARN 2364 --- [ main] .s.s.UserDetailsServiceAutoConfiguration :

Using generated security password: 1b9bd1cc-6431-42e5-ab58-783cb1f49f89

This generated password is for development use only. Your security configuration must be updated before running your application in production.

2022-04-14 18:32:42.326 INFO 2364 --- [ main] c.c.kouncil.config.WebSecurityConfig : Configuring spring security
2022-04-14 18:32:42.344 INFO 2364 --- [ main] o.s.s.web.DefaultSecurityFilterChain : Will not secure any request
2022-04-14 18:32:42.356 DEBUG 2364 --- [ main] c.c.kouncil.config.WebSocketConfig : allowedOrigins=[*]
2022-04-14 18:32:42.356 WARN 2364 --- [ main] c.c.kouncil.config.WebSocketConfig : Allowed origin pattern set to *. Consider narrowing down.
2022-04-14 18:32:42.654 INFO 2364 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8081 (http) with context path ''
2022-04-14 18:32:42.958 INFO 2364 --- [ main] o.s.m.s.b.SimpleBrokerMessageHandler : Starting...
2022-04-14 18:32:42.959 INFO 2364 --- [ main] o.s.m.s.b.SimpleBrokerMessageHandler : BrokerAvailabilityEvent[available=true, SimpleBrokerMessageHandler [org.springframework.messaging.simp.broker.DefaultSubscriptionRegistry@7074da1d]]
2022-04-14 18:32:42.960 INFO 2364 --- [ main] o.s.m.s.b.SimpleBrokerMessageHandler : Started.
2022-04-14 18:32:42.974 INFO 2364 --- [ main] com.consdata.kouncil.KouncilApplication : Started KouncilApplication in 3.808 seconds (JVM running for 4.33)
2022-04-14 18:32:57.537 INFO 2364 --- [nio-8081-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-04-14 18:32:57.538 INFO 2364 --- [nio-8081-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2022-04-14 18:32:57.542 INFO 2364 --- [nio-8081-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 2 ms
2022-04-14 18:33:41.944 INFO 2364 --- [MessageBroker-1] o.s.w.s.c.WebSocketMessageBrokerStats : WebSocketSession[0 current WS(0)-HttpStream(0)-HttpPoll(0), 0 total, 0 closed abnormally (0 connect failure, 0 send limit, 0 transport error)], stompSubProtocol[null], stompBrokerRelay[null], inboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], outboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], sockJsScheduler[pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 0]
2022-04-14 19:03:41.947 INFO 2364 --- [MessageBroker-1] o.s.w.s.c.WebSocketMessageBrokerStats : WebSocketSession[0 current WS(0)-HttpStream(0)-HttpPoll(0), 0 total, 0 closed abnormally (0 connect failure, 0 send limit, 0 transport error)], stompSubProtocol[null], stompBrokerRelay[null], inboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], outboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], sockJsScheduler[pool size = 2, active threads = 1, queued tasks = 0, completed tasks = 1]
2022-04-14 19:33:41.957 INFO 2364 --- [MessageBroker-2] o.s.w.s.c.WebSocketMessageBrokerStats : WebSocketSession[0 current WS(0)-HttpStream(0)-HttpPoll(0), 0 total, 0 closed abnormally (0 connect failure, 0 send limit, 0 transport error)], stompSubProtocol[null], stompBrokerRelay[null], inboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], outboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], sockJsScheduler[pool size = 3, active threads = 1, queued tasks = 0, completed tasks = 2]

Users who use confluent's kafka have a lot of internal confluent topics in brokers and they all start with "_confluent-" so it'd be nice if there was a simple toggle to hide all of them in the topic list page as one has to scroll past them everytime the topic page is loaded (they start with a special character so they appear before all conventionally named topics) which is a bit frustrating. Thanks!

Does the tool support avro records because currently it returns null when i try to view the value of avro records

If not, is that something you guys plan to introduce in the future?

Thanks!

The frontend UI which displays topic, has 2 columns. Topic Name and Partition. Topic name is default sorted in alphabetic order.
When clicking the header Partition, there is no change in the list order.

Can this be corrected.

I am trying to use kouncil with AWS MSK as described in docs. I am trying to host kouncil as a ECS task. I have created a config file as given below:

kouncil:
  clusters:
    - name: demo-cluster-1
      brokers:
        - host:  b-1.de.XXXXX6.c4.kafka.ap-south-1.amazonaws.com
          port: 9098
          saslMechanism: AWS_MSK_IAM
          saslProtocol: SASL_SSL
          saslJassConfig: software.amazon.msk.auth.iam.IAMLoginModule required awsProfileName="ecsTaskExecutionRole";
          saslCallbackHandler: software.amazon.msk.auth.iam.IAMClientCallbackHandler
        - host: b-2.de.XXXXX6.c4.kafka.ap-south-1.amazonaws.com
          port: 9098
          saslMechanism: AWS_MSK_IAM
          saslProtocol: SASL_SSL
          saslJassConfig: software.amazon.msk.auth.iam.IAMLoginModule required awsProfileName="ecsTaskExecutionRole";
          saslCallbackHandler: software.amazon.msk.auth.iam.IAMClientCallbackHandler
        - host:  b-3.de.XXXXX6.c4.kafka.ap-south-1.amazonaws.com
          port: 9098
          saslMechanism: AWS_MSK_IAM
          saslProtocol: SASL_SSL
          saslJassConfig: software.amazon.msk.auth.iam.IAMLoginModule required awsProfileName="ecsTaskExecutionRole";
          saslCallbackHandler: software.amazon.msk.auth.iam.IAMClientCallbackHandler
       

but i get the error:
2024-02-25 13:22:22.468 INFO 1 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient : [AdminClient clientId=adminclient-1] Cancelled in-flight API_VERSIONS request with correlation id 673 due to node -1 being disconnected (elapsed time since creation: 61ms, elapsed time since send: 61ms, request timeout: 3600000ms)

After starting the app from docker , cannot login use admin and password admin.

docker run -p 7070:8080 -v /home/ssm-user/kouncil/:/config/ consdata/kouncil:latest

Hello team, thank you so much for this great application, I was wondering if it could be possible to authenticate Kouncil using OKTA. I see in the doc that LDAP and SSO with Github is supported, but I don't see any SAML or OICD options. Thank you so much!

How can i enable topic menu for UI?