Sharding running same test multiple times in a single run about karma-parallel

[joeljeske]

Thank you for the report. As mentioned in the readme, karma-parallel may add additional tests to your output.

If this plugin discovers that you have focused some tests (fit, it.only, etc...) in other browser instances, it will add an extra focused test in the current browser instance to limit the running of the tests in the given browser. Similarly, when dividing up the tests, if there are not enough tests for a given browser, it will add an extra test to prevent karma from failing due to no running tests

Essentially, by the time we know that there is a focused describe block, it is too late to decide not to shard; we have already started up all the browsers instances and started parsing the tests. The only fail-safe way (that I know of) to tell mocha/jasmine to skip other tests, is by adding a focused test. Otherwise, shards that happen to not be delegated any focused tests will attempt to run all specs since it is unaware of any focused specs at all.

The spec that is running, is a very lightweight spec that does nothing at all. See:
https://github.com/joeljeske/karma-parallel/blob/v0.2.9/lib/karma-parallelizer.js#L142

[leviwilson]

Makes sense, thanks!

[kaashan]

@joeljeske : we have a project set-up where we use karma-parallel with 5 executors to run ~8500 tests. Below is a snapshot of a successful run:

We have also set jasmine's failSpecWithNoExpectations to true in karma.conf.js file

client: {
      jasmine: {
        random: false,
        failSpecWithNoExpectations: true
      }
},

Recently, for one odd build on our CI/CD pipeline karma-parallel added the lightweight spec you mentioned above and since this spec does not has any expectation, it caused that build to fail:

I can understand additional tests introduced by karma-parallel in case of focused tests or not enough tests to be run for every executor instance. But why did this happen for our use case?

From the failure screenshot, it could be seen that 6956 tests were split up amongst shard-2, shard-3, shard-4 and shard-5. The left 1577 remaining tests should have been received by shard-1 and there was no need for karma-parallel to add any default test. So, what could be the reason for addition of default test in this particular case?