Comments (3)
Hey @kanimaru, the problem is that the topics
field isn't standard in Kafka Connect, so we can't rely on it in order to detect the topics your connector uses.
Thus, trying to be smart, we will use any configuration option that includes topic
in its name in order to detect topics. Alas in your case it backfires and Connect UI detects erroneous topic names.
I don't know whether there are any plans to fix this in the short term from the dev team.
from kafka-connect-ui.
@andmarios, that is true for source connectors, but per Kafka documentation, sink connectors will always have a topics
option. Thus, I believe an explicit assumption could be made for sinks.
Here are the topic-related configuration option(s) for some common source connectors:
Connector | Option |
---|---|
JdbcSourceConnector |
topic.prefix |
ReplicatorSourceConnector |
topic.regex , topic.whitelist , topic.blacklist |
FileStreamSource |
topic |
SpoolDirCsvSourceConnector |
topic |
FsSourceConnector |
topic |
FtpSourceConnector |
ftp.monitor.tail , ftp.monitor.update |
GitHubSourceConnector |
topic |
CloudPubSubSourceConnector |
kafka.topic |
IotHubSourceConnector |
Kafka.Topic |
CouchbaseSourceConnector |
topic.name |
ReplicateSourceConnector |
topic.prefix |
KineticaSourceConnector |
kinetica.topic_prefix |
HANASourceConnector |
topics |
Clearly some of these are more complex than others (i.e. connectors with configuration options similar to ReplicatorSourceConnector
), but thoughts on updating the source connector "topic extraction" algorithm to first try and find a single configuration option that matches the following regular expression?
/(?:kafka[._])?topics?(?:[._]prefix)?$/
It seems like that would fix most of the issues.
from kafka-connect-ui.
Thanks! This is very helpful indeed.
I tagged some members of the UIs dev team to have a look and implement it when possible.
from kafka-connect-ui.
Related Issues (20)
- We have SSL enabled kafka cluster. How can we define below configuration for kafka-connet-ui HOT 2
- 502 - Connectivity error for some configurations HOT 1
- CONNECTIVITY ERROR HOT 1
- [Question] JDBC Postgres required fields
- Display the downtime of task
- Page is not changed HOT 1
- Execution mode 'CONNECT' is not currently supported
- Obfuscate database.password in connector properties HOT 1
- The worker_id is null for running connector/tasks HOT 1
- kafka-connect-ui exit(2) when running on linux HOT 5
- Problems if cluster has no connectors
- Mutliple user creation
- Add description for the Hive connector
- can not run HOT 4
- [Improvement] Reload Caddy Server on Config Change
- Dashboard Error - Source connector appears as sink connector HOT 2
- D3 v3 is going to be deleted
- Not supported connector is always counted as sink connector on the dashboard
- Warning message and connector tasks is more then UI alignment looks differently. HOT 4
- Is this project abandoned? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kafka-connect-ui.