Comments (6)
from flink-spector.
Yes myFlatMap() changes the structure of the eventStreamPayload, but no matter what I compared it to, it still passed.
So I switched to JUnit4 in case that was the problem and now see different results. The following asserts still always pass not matter what:
assertStream(dataStream, hasItems(jsonObject));
assertStream(dataStream,expectedRecords);
But the sink now fails and I don't know why. The line:
dataStream.addSink(outputFormat);
Fails with:
java.lang.AssertionError:
Expected: (a collection containing <{"id":"aschuhl","type":"account","eId":"123456789","eType":"test"}>)
but: a collection containing <{"id":"aschuhl","type":"account","eId":"123456789","eType":"test"}>
But both collections seem the same to me.
from flink-spector.
Also here are the imports
import io.flinkspector.core.collection.ExpectedRecords;
import io.flinkspector.core.input.Input;
import io.flinkspector.core.input.InputBuilder;
import io.flinkspector.datastream.DataStreamTestBase;
import io.flinkspector.datastream.DataStreamTestEnvironment;
import io.flinkspector.datastream.functions.TestSink;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.json.JSONObject;
import org.junit.jupiter.api.Test;
import java.util.Arrays;
import static org.hamcrest.Matchers.*;
public class test extends DataStreamTestBase {
from flink-spector.
The DataStreamTestBase has a lot of built in semantic sugar to reduce boiler plate code.
You've called some methods which the DataStreamTestBase normally hides. Also the TestBase only works with JUnit 4 for now.
DataStream<JSONObject> dataStream =
createTestStream(Arrays.asList(eventStreamPayload)).flatMap(new myFlatMap());
JSONObject jsonObject = new JSONObject(eventStreamPayload2);
assertStream(dataStream, hasItems(jsonObject));
This should be all you need to setup the test.
assertStream
already instantiates a TestSink
and attaches it to the dataStream.
The TestSink validates the output this is why you see the AssertionError "too early".
The failed comparison between the two objects seems to stem from the equals Method on org.json.JSONObject, which only seems to do a reference comparison.
https://stackoverflow.com/questions/2253750/compare-two-json-objects-in-java
from flink-spector.
import org.junit.jupiter.api.Test;
means junit 5 was used.
I had the same and 'fixed' it by calling initialize()
and executeTest()
manually. Are there 'plans' to switch or support junit5?
from flink-spector.
Yes there are plans but I have never used unit 5 myself and have to look into it.
from flink-spector.
Related Issues (20)
- Low performance when using EventTimeInputBuilder HOT 1
- Tests don't work while making mvn test on CircleCI HOT 2
- Convert Message Headers to Long
- Global Windows don't emit data on time HOT 2
- Publish versions to OSS Repository Hosting HOT 2
- "Task was cancelled" exception when testing CoProcessFunction HOT 6
- Tests flicker when running in parallel (when using Managed State) HOT 4
- Publish 0.8.2 to Maven Central HOT 5
- NPE in test HOT 6
- Consider splitting out JUnit 4 functionality into separate module and documenting HOT 2
- There is no DataStreamTestBase.emit variant that accepts an arbitrary timestamp HOT 1
- Run pipeline without flushing all the windows HOT 2
- TestBaseUtils.startCluster is deprecated as of Flink 1.7.0 HOT 5
- Release 0.8.5? HOT 3
- Flinkspector 0.9.1 references package/classes not published by Flink HOT 4
- Is there any plan to support flink 1.8? HOT 6
- setParallelism changes test outcome HOT 6
- unresolved dependencies scala 2.12
- unmaintained? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flink-spector.