Comments (6)
I have been successful in recreating your problem and found a fix for it. #93 I'm going to release it and wait for your confirmation.
from flink-spector.
Any parallelism greater than one will remove determinism from the test run. Because the events are now processed in parallel by multiple instances of the operators. For example the order of the events is not predefined any more. If I interpret your test correctly no expectation is checking the order of events. It could be the case, that the logic you're testing depends on order, or that a certain instance of a function sees certain events.
Without more information it's hard to say what's the case. I hope this helps.
from flink-spector.
@lofifnc Thank you for your reply. I have nothing that requires a deterministic ordering, but I have a keyBy
operator in there as well as a BoundedOutOfOrderTimestampExtractor
. In fact, all of the code fits in this screengrab:
When setParallelism(2)
is run, the matcher gets zero output events, but like you can see I have a println in a map
, which gets run with parallelism enabled; and it outputs the correct value... I'm at a bit of a loss as to why the asserter would get zero values...
from flink-spector.
Version 0.9.3 has been released. Can you confirm this resolves your issue?
from flink-spector.
Thank you very much, I'm going to try it tomorrow morning first thing!
from flink-spector.
@lofifnc I didn't make it "first thing" but I've tried it now, and it works perfectly! Thank you @lofifnc
from flink-spector.
Related Issues (20)
- Low performance when using EventTimeInputBuilder HOT 1
- Tests don't work while making mvn test on CircleCI HOT 2
- Convert Message Headers to Long
- Global Windows don't emit data on time HOT 2
- Publish versions to OSS Repository Hosting HOT 2
- "Task was cancelled" exception when testing CoProcessFunction HOT 6
- Tests flicker when running in parallel (when using Managed State) HOT 4
- Publish 0.8.2 to Maven Central HOT 5
- NPE in test HOT 6
- Consider splitting out JUnit 4 functionality into separate module and documenting HOT 2
- assertStream always passes HOT 6
- There is no DataStreamTestBase.emit variant that accepts an arbitrary timestamp HOT 1
- Run pipeline without flushing all the windows HOT 2
- TestBaseUtils.startCluster is deprecated as of Flink 1.7.0 HOT 5
- Release 0.8.5? HOT 3
- Flinkspector 0.9.1 references package/classes not published by Flink HOT 4
- Is there any plan to support flink 1.8? HOT 6
- unresolved dependencies scala 2.12
- unmaintained? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flink-spector.