JUnit files are appended to with RSpec formatters about knapsack_pro-ruby

[cccCody]

The link in the previous comment no longer works. Looks like that information now lives here: https://knapsackpro.com/faq/question/how-to-use-junit-formatter#how-to-use-junit-formatter-with-knapsack_pro-queue-mode

[ArturT]

Hey @bobbytables

I could create custom JUnit formatter. I did similar thing for summary:
https://github.com/KnapsackPro/knapsack_pro-ruby/blob/master/lib/knapsack_pro/formatters/rspec_queue_summary_formatter.rb
in this PR:
https://github.com/KnapsackPro/knapsack_pro-ruby/pull/31/files

After that the formatter could be specified in the knapsack command similar as here:
https://github.com/KnapsackPro/knapsack_pro-ruby#how-to-use-junit-formatter

[ArturT]

I've tried the approach with hook and I did an example with hook after a subset of tests executed from the queue to rename the output rspec.xml file but then I figure out that actually the junit formatter cumulate results so each new rspec.xml has inside of it all results from previous subset queue run and there are multiple opening xml tags.

Related PR: #41

So it seems the only way would be to write custom junit formatter then. I will look into that or I will fork the gem to adjust it to knapsack_pro case.

[bobbytables]

Well with the ability to have an "after" hook, you could just rename the file. That way the next run just creates a new file.

[ArturT]

Yes, that's what I did but the problem is with the way how formatter works when I execute multiple times the RSpec::Core::Runner for subset of work queue.

knapsack_pro-ruby/lib/knapsack_pro/runners/queue/rspec_runner.rb

Line 32 in fc911fa

exit_code = RSpec::Core::Runner.new(options).run($stderr, $stdout)

The junit formatter keeps in memory xml from previous run so the next rspec.xml file has double xml tags even when the first rspec.xml file was renamed.

I did custom formatter for RSpec Queue Summary so I will have to do a similar thing for junit formatter.
#31

[bobbytables]

Is there a way to force the formatter to write out?

[ArturT]

This is how junit formatter dump xml to file:
https://github.com/sj26/rspec_junit_formatter/blob/master/lib/rspec_junit_formatter/rspec3.rb#L19
https://github.com/sj26/rspec_junit_formatter/blob/master/lib/rspec_junit_formatter.rb#L18

I noticed when I use just junit formtter without my "after" hook then sometimes the xml file has weird content like the content was incorrectly appended to the file.

When I did rename rspec.xml file after each subset of work queue then the last generated rspec.xml file has correct content with duplicated xml tags. The last one xml tag contains the full report. So it means we would have to remove from rspec.xml the content till the last opening xml tag.

Here is short example of how last generated rspec.xml file looks like.

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="rspec" tests="1" failures="0" errors="0" time="0.027553" timestamp="2017-05-23T23:02:03+02:00">
  <!-- Randomized with seed 24503 -->
  <properties/>
  <testcase classname="spec.features.calculator_spec" name="Calculator when try to add without provided numbers result is 0" file="./spec/features/calculator_spec.rb" time="0.094628"/>
</testsuite>
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="rspec" tests="2" failures="0" errors="0" time="0.027553" timestamp="2017-05-23T23:02:03+02:00">
  <!-- Randomized with seed 24503 -->
  <properties/>
  <testcase classname="spec.features.homepage_spec" name="Homepage Features has welcome text" file="./spec/features/homepage_spec.rb" time="0.197960"/>
  <testcase classname="spec.features.calculator_spec" name="Calculator when try to add without provided numbers result is 0" file="./spec/features/calculator_spec.rb" time="0.094628"/>
</testsuite>

I will play more with this idea tomorrow.

[bobbytables]

Sounds good, thanks for looking at this so intently 😄

[ArturT]

@bobbytables I figured out this. Could you update knapsack_pro version and add the hook as described here:
https://github.com/KnapsackPro/knapsack_pro-ruby#how-to-use-junit-formatter-with-knapsack_pro-queue-mode

Basically, we need to rename the rspec.xml in after subset queue so the file won't be appended with duplicated xml content. The last execution of tests subset from work queue will create the rspec.xml with xml that contains the tests from all subset runs (all tests from the particular CI node).

Please let me know if this works for you.

[ArturT]

Here is the current docs page related to formatters: https://docs.knapsackpro.com/ruby/rspec/#formatters-rspec_junit_formatter-json