Comments (9)
On another instance where I've debugged around this problem, I get:
java.lang.IllegalStateException: KafkaBroker is not running
from kafka-junit.
ClassRule
s must be static
. That's probably the reason why it's not getting started up.
from kafka-junit.
Cool. I revised my code as below and it worked:
package com.termmerge.nlpcore.obtainer;
import junit.framework.TestCase;
import org.junit.Test;
import com.github.charithe.kafka.KafkaJunitRule;
import com.github.charithe.kafka.EphemeralKafkaBroker;
import net.jodah.concurrentunit.Waiter;
import java.util.Map;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import java.util.Properties;
public class KafkaStreamObtainerTest extends TestCase
{
@Test
public void testOneMessage() throws Throwable
{
Waiter waiter = new Waiter();
EphemeralKafkaBroker kafkaBroker =
EphemeralKafkaBroker.create(8000, 8001);
kafkaBroker.start().get();
KafkaJunitRule kafkaRule = new KafkaJunitRule(kafkaBroker);
KafkaProducer testProducer = kafkaRule.helper().createStringProducer();
testProducer.send(
new ProducerRecord<>("testTopic", "testKey", "testValue")
);
testProducer.close();
Map consumerSettings = new Properties();
consumerSettings.put(
"connection_string",
"localhost:" + Integer.toString(kafkaRule.helper().kafkaPort())
);
consumerSettings.put("group_id", "test");
KafkaStreamObtainer kafkaStream =
new KafkaStreamObtainer(consumerSettings);
kafkaStream.addListener((record) -> {
waiter.assertEquals(record.get("key"), "testKey");
waiter.assertEquals(record.get("value"), "testValue");
waiter.resume();
});
kafkaStream.listenToStream("testTopic");
waiter.await(50000);
}
}
Now, however, I have a problem with my Kafka consumer hanging up at the poll() stage where it is not returning at all (after calling poll). Can we use the regular Kafka API client to interface with this?
from kafka-junit.
The rule starts a standard Kafka broker that can be accessed using any Kafka client.
Your revised test is not quite correct for a couple of reasons. Firstly, you are calling get
on a future at the start of the test. It would just block there and not execute the rest of the code. Secondly, you're mixing JUnit3 style with JUnit4 style. I recommend switching completely to the JUnit4 style as that's the version that introduced rules and what this project has been tested with.
from kafka-junit.
@charithe I know I was doing a blocking call with get() for debugging purposes. I specifically wanted it to block so that I know full well that the broker has started before I even mount any producers or consumers on it. I'm using IntelliJ and I know that using the debugger, I am getting passed the blocking call without a problem. My problem lies in the fact that it takes so long for a Mock Producer to send a data and my consumer to poll for such data (even though my poll interval is only 100ms).
from kafka-junit.
The error I get is:
[kafka-producer-network-thread | kafka-junit] WARN org.apache.kafka.clients.NetworkClient - Error while fetching metadata with correlation id 2 : {testTopic=UNKNOWN}
[kafka-request-handler-3] ERROR kafka.server.KafkaApis - [KafkaApi-1] Error when handling request {topics=[testTopic]}
from kafka-junit.
Just confirmed that using a normal Apache Kafka Docker container, that I am able to pass my unit tests, but using the EphemeralKafkaBroker, my tests seem to fail. The image below shows that my unit tests pass using Apache Kafka straight on.
For reference, I am using the Apache Kafka Docker container at https://hub.docker.com/r/wurstmeister/kafka/
from kafka-junit.
I am not sure how you could move past the Future.get()
because that means the broker had stopped.
Anyway, I noticed in your screenshot that you're using Kafka 0.10.1.0. The JUnit rule is still on 0.10.0.1. The problem could be due to the library version mismatch.
This is an example of how to use the class rule: https://github.com/charithe/kafka-junit/blob/master/src/test/java/com/github/charithe/kafka/KafkaJunitClassRuleTest.java. I hope that helps.
from kafka-junit.
Got it running. The fix was to set my Kafka consumer's "auto.offset.reset" to "earliest" policy.
from kafka-junit.
Related Issues (20)
- Getting error when running unit test HOT 5
- Add support for Kafka version 0.10.2.0 HOT 2
- Failing JUnits on Windows HOT 9
- java.lang.NoSuchFieldError: configFileStr HOT 3
- Does not work with dependency kafka-avro-serializer HOT 6
- JUnit 5 example test fails with NoSuchFieldError: configFileStr HOT 2
- Transactional producer not working HOT 2
- do not support transaction HOT 1
- Update to Kafka 2.1.1 HOT 2
- When using timeout to simulate "no-message" scenario, other regular test cases stop working HOT 4
- Kafka 2.3 is out HOT 2
- KafkaHelper threading issue HOT 3
- Kafka 2.4 HOT 3
- Multipartition topic HOT 2
- consumeStrings() discards buffered messages HOT 3
- Support Kafka 2.6
- kafka-junit support for kafka_2.13 (Kafka 2.6.0 recommends using Scala 2.13.x) HOT 3
- Breaking changes from Kafka 2.8.0 HOT 3
- EphemeralKafkaCluster with more than 2 brokers fails with NOT_ENOUGH_REPLICAS error HOT 1
- OWASP Dependency checker reported vulnerabilities in snappy java.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kafka-junit.