"Task was cancelled" exception when testing CoProcessFunction about flink-spector

[lofifnc]

I think I have an idea what's the problem. I will debug this during the weekend. Thanks for pointing this out.

[whipit]

I got it working - but by changing my design a bit. I missed mentioning a key design in my Test classes that I think is causing flink-spector to not work. Essentially, I am using nested non-static classes to implement a Given-When-Then (GWT) testing pattern. So I have something like this:

public class GivenMyInputStream extends DataStreamTestBase {
       KeyedStream<MyInput, Integer> myInputStream;
       @Before
       public void setup() {
             Input<MyType> myTypeInput = InputBuilder.startsWith(new MyType(1)).emit(new MyType(2));
             myInputStream = this.createTestStream(myTypeInput).keyBy(new MyKeySelector());           
       }

       public class WhenJoinedToMyOtherInputStream extends DataStreamTestBase {
             KeyedStream<MyInput, Integer> myOtherInputStream;

             @Before
             public void setup() {
                       Input<MyOtherType> myOtherTypeInput = InputBuilder.startsWith(new MyOtherType(1)).emit(new MyOtherType(2));
                      myOtherInputStream = this.createTestStream(myOtherTypeInput).keyBy(new MyOtherKeySelector());
             }

              @Test
              public void ensureJoinWorks() {                             
                            DataStream outputStream = myInputStream.connect(myOtherInputStream).process (new DummyJoinFunction());

                              assertStream (outputStream, new MyOutputMatcher());             
              }
      }
}

This pattern does not play well with flink spector -- likely because my 2 streams are in 2 different classes (one nested) both extending from DataStreamTestBase.

I would still like to understand why exactly, but I am inclined to not call this an "Issue" but rather "User error" if its hard to fix.

Edit: I should have mentioned that I got it working by including both streams in the same class (i.e. eliminate the nested class)

[lofifnc]

The DataStreamTestBase is just a template for setting up JUnit test suites. It takes care of setting up and executing the test. Each instance will set up it's own DataStreamTestEnvironment in the background. I tried to make specifing test as concise as possible so createTestStream() will register a source with the Environment in the Background. So you're working with two distinct test environments. However the Input object is portable and reusable. The stream is registered with one environment.

If you want to combine DataStreamTestBases you can take a look at the (DataStreamTestBase)[https://github.com/ottogroup/flink-spector/blob/master/flinkspector-datastream/src/main/java/io/flinkspector/datastream/DataStreamTestBase.java], and how the DataStreamTestEnvironment is utilized. You can find how a test is setup and executed there. The testEnv variable in the DataStreamTestBase is also accessible so you can make sure both are using the same environment.

[whipit]

Thank you! That makes sense. I'll close this issue.

[whipit]

Related (but different) question: Does the framework support running multiple tests (different implementations and instances of DataStreamTestBase) in parallel, or even in the same JUnit JVM?

I am trying to do this, but am getting flickering results. For instance, I have 2 nearly identical implementations of DataStreamTestBase. For simplicity sake, both are testing exactly the same input data.

When I run the tests, one of them succeeds whereas the other one fails. The latter fails because the output stream seems to have incorrect data -- specifically it seems to have data from the first instance.

However, if I run the test in debug mode, and pause at a break point long enough, then both tests succeed.

Edit: In my operator, I am using a ValueState member. I can confirm that its value gets restored from the first test run. So I suspect somewhere state is not being cleaned up between runs.

[whipit]

to keep it clean, I opened a separate issue for the shared state problem.

#72