Kafka Topics UI timing out on topic content load about kafka-topics-ui

[Antwnis]

Hi @heshansp - What version of Kafka-Rest are you using ?

For Kafka-Rest <= 3.1.2 you should use Kafka-Topics-UI 0.8.3
For Kafka-Rest >= 3.2.0 you should use the latest release

[heshansp]

Well I'm not using any Schema Registry. Is it mandatory please?

[Antwnis]

Sorry @heshansp i meant to ask about the version of (Kafka-Rest)

Kafka-topics-ui is using that as a backend.
Kafka-topics-ui works even when not using Avro / Schema Registry

[heshansp]

Well i compiled the github master from confluent along with the prescribed dependencies. So I believe I'm running the latest.

[Antwnis]

So on the first comment you mention the API

/topics/_schemas/partitions

As you don't use the Schema Registry - the topic _schemas should NOT be there

I'd suggest you to set DEBUG_LOGS_ENABLED: true in env.js ( see example here https://github.com/Landoop/kafka-topics-ui#setup-kafka-rest-clusters ) and then

open up the Chrome Dev Tools ( right click and Inspect ) on chrome and then refresh the page - and view curl commands on the console

[heshansp]

Hi @Antwnis,
I was running with DEBUG_LOGS_ENABLED: true anyway. The only reason why I know that the UI is making those API calls is because I checked the curl commands appearing on the command console. Looks like the UI is expecting the Schema Registry to be there by default.

Can you please point me in the code where that evaluation is made whether Schema Registry is on?

[Antwnis]

The schema registry is bringing in additional information (and allows the UI) to deserialize Avro data as well. Without it, it cannot deserialize the Avro - and displays them as binary

Have a look at http://cloudera.landoop.com:13031/#/cluster/Fast%20Data/topic/n/pos-reps-test-01/

I've just shutdown the schema-registry on the above setup & it seems to be working

[heshansp]

Ok that makes sense. Yet what is happening in my UI does not make sense.
This is what I'm seeing on the Chrome console.

[heshansp]

I downloaded the latest code from https://github.com/confluentinc/kafka-rest and compiled it myself.

[andmarios]

Just to be a bit more clear, we always search for Schema Registry, but this is ok to fail.
I don't know if this introduces some kind of lag (due to some kind of timeout perhaps) because Topics UI indeed seems to me to take longer to return messages without the Schema Registry. I asked the developers to take a look at it.

Another problem with the current REST Proxy, is a combination of default settings and a bug.
On topics with many messages, a kafka consumer takes some time to return the first messages.
REST proxy has a default timeout for consumer.request.timeout.ms of 1 second, so it fails to bring messages from big topics and shows them as empty instead.
Unfortunately due to a bug, this setting, if you set it explicitly, will propagate to proxy's consumer request.timeout.ms which is a totally different setting. What this means, is that if you decide to set consumer.request.timeout.ms you have to set it to at least 30 seconds.

So maybe you could try add this to your REST proxy configuration and see how it goes?

consumer.request.timeout.ms=30000

[heshansp]

Hi @Antwnis,
In the Kafka Topic I am trying to query there are only 2 messages and extending the timeout on the REST proxy configuration didn't really help either. When I curl on the topics, I get the response within about 40ms, so I highly doubt that this is anything to do with the performance.

I just downloaded the REST Proxy Dokcer images and started running that assuming there is an issue with the way I compiled the Java code. The behaviour is exactly the same, so I'm inclined to believe that there is an issue with the UI.

[andmarios]

Could you try our docker image? I suspect some kind of misconfiguration on nginx.

docker run --rm -p 8000:8000 \
    -e "KAFKA_REST_PROXY_URL=http://localhost:8082" \
    -e PROXY=true \
    landoop/kafka-topics-ui:0.9.2

Also I think your decision to use docker for Kafka REST was a good one. Usually you want its version to match your kafka version, so compiling from master could lead to issues.

[heshansp]

Looks like there is an issue there as well.

Enabling proxy.
/run.sh: line 20: can't create /caddy/Caddyfile: Interrupted system call
Kafka REST Proxy URL to /api/kafka-rest-proxy.
Activating privacy features... done.
http://0.0.0.0:8000

The shell command seems to have a bug and the REST Proxy URL seems to be hardcoded to /api/kafka-rest-proxy.

[andmarios]

This is very funny, your docker can't write inside the container? If you are on macOS or Windows, maybe docker's VM disk is full?

I am afraid I won't be able to help without knowledge of your configuration. I.e OS, Kafka version, how you setup nginx, how you run the docker containers, etc.

Also it is still not clear to me, whether Kafka Topics UI does not even list your topics, or if it lists your topics but can't show any messages.

[ssiahetiong]

Im also encountering this issue on the latest docker image, but not in 0.8.3

[WraithCadmus]

I'm having this issue too, version 3.3 of the REST API and git commit e96bf2d of the UI from this morning, connecting to Kafka 1.1.0 (with Scala 2.12) using Zookeeper 3.4.12. All are running on CentOS 7.

With that out the way here's the Console output from Firefox, if that helps shed any light.

_schemas CREATING CONSUMER:  
Object { group: "kafka_topics_ui_json_2018-05-02-02-16-36-088", instance: "kafka-topics-ui-json" }
 2018-05-02-02-16-36-088
angular.js:14525:17
HttpFactory:  curl -X POST -H "Content-Type: application/vnd.kafka.v2+json --data {"name": "kafka-topics-ui-json", "format": "json", "auto.offset.reset": "earliest", "auto.commit.enable": "false"} http://172.16.68.30:8084/consumers/kafka_topics_ui_json_2018-05-02-02-16-36-088
angular.js:14525:17
_schemas POLL STEPS START
angular.js:14525:17
HttpFactory:  curl -X GET -H "Content-Type:  --data  http://172.16.68.30:8084/topics/_schemas/partitions
angular.js:14525:17
HTTP ERROR:  Failed at method [GET] [http://172.16.68.30:8084/topics/_schemas/partitions] with error: 
{"data":{"error_code":40401,"message":"Topic not found."},"status":404,"config":{"method":"GET","transformRequest":[null],"transformResponse":[null],"jsonpCallbackParam":"callback","url":"http://172.16.68.30:8084/topics/_schemas/partitions","data":"","dataType":"json","headers":{"Content-Type":"","Accept":"application/vnd.kafka.v2+json, application/vnd.kafka+json, application/json"}},"statusText":"Not Found"} 
DATA SENT: 
angular.js:14525:17
Possibly unhandled rejection: Failed at method [GET] [http://172.16.68.30:8084/topics/_schemas/partitions] with error: 
{"data":{"error_code":40401,"message":"Topic not found."},"status":404,"config":{"method":"GET","transformRequest":[null],"transformResponse":[null],"jsonpCallbackParam":"callback","url":"http://172.16.68.30:8084/topics/_schemas/partitions","data":"","dataType":"json","headers":{"Content-Type":"","Accept":"application/vnd.kafka.v2+json, application/vnd.kafka+json, application/json"}},"statusText":"Not Found"}
angular.js:14525:17

[Antwnis]

@WraithCadmus the latest version of kafka-topics-ui uses the V2 of the APIs

See kafka.v2+json request

For this to work - you will have to configure your Kafka REST backing service according (i.e. adding the bootstrap servers and so on)

[WraithCadmus]

I do have bootstrap servers listed in my config (attached for reference), but as I'm still on v3.3 of the REST API maybe that's where I'm going wrong. We had issues with v4 last time but this was last year some time. Thanks for the hints.

id=kafka-rest-test-server
zookeeper.connect=172.16.68.27:2181,172.16.68.26:2181,172.16.68.25:2181/kafka
bootstrap.servers=PLAINTEXT://172.16.68.29:9092,PLAINTEXT://172.16.68.28:9092

access.control.allow.methods=GET,POST,PUT,DELETE,OPTIONS
access.control.allow.origin=*

listeners=http://0.0.0.0:8084
consumer.request.timeout.ms=30000

[WraithCadmus]

Okay, now I'm a little calmer I've re-read the thread and I'm making some headway. I still have nothing happen, but I think it might be my Kafka config rather than anything wrong with the UI. Thank-you for taking the time to respond @Antwnis

[WraithCadmus]

So I think I see my problem now (comparing it to one that works, but I don't have the docs for). On the working one it tries to get the Avro version, fails, then tries JSON and works. On this new one which doesn't work it just hangs forever on the Avro step.

HttpFactory: curl -X GET -H "Content-Type: application/vnd.kafka.v2+json --data http://172.16.68.30:8084/consumers/kafka_topics_ui_avro_2018-05-03-01-37-23-518/instances/kafka-topics-ui-avro/records?timeout=3000&max_bytes=50000

EDIT: So a further comparison to the other system shows that the avro return is near-instant (small dev topics), so it's less the format of the request and more the fact it's just hanging.