orta / danger-junit Goto Github PK
View Code? Open in Web Editor NEWLets you report your test suite problems back to the PR elegantly
License: MIT License
Lets you report your test suite problems back to the PR elegantly
License: MIT License
We have a small issue wherein since we retry our tests for failure, how do we incorporate that with danger-junit, because junit.report would fail even for single test ๐
My Danger file include this lines
junit_tests_dir = "**/build/**/reports/jacoco/report.xml"
Dir[junit_tests_dir].each do |file_name|
junit.parse file_name
junit.report
end
I have exception
[!] Invalid `Dangerfile` file: undefined method `nodes' for #<Ox::DocType:0x000055b4b0207848>
# from Dangerfile:14
# -------------------------------------------
# Dir[junit_tests_dir].each do |file_name|
> junit.parse file_name
# junit.report
# -------------------------------------------
Do you have any ideas?
Also, XML file is really large. ~30Mb
Starting today (with version 1.0.1) I get the following error when running Danger with a junit report on an iOS project.
bundler: failed to load command: danger (/Users/vagrant/.rbenv/versions/2.6.5/bin/danger)
Danger::DSLError:
[!] Invalid `Dangerfile` file: undefined method `header' for #<Danger::Dangerfile:0x00007fba2c314c08>
# from Dangerfile:12
# -------------------------------------------
# markdown(message)
> junit.report
# else
# -------------------------------------------
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/danger_core/dangerfile.rb:70:in `method_missing'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/plugin_support/plugin.rb:23:in `method_missing'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-junit-1.0.1/lib/junit/plugin.rb:174:in `report'
Dangerfile:12:in `processTestFiles'
Dangerfile:32:in `block in eval_file'
Dangerfile:31:in `each'
Dangerfile:31:in `eval_file'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/danger_core/dangerfile.rb:307:in `eval'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/danger_core/dangerfile.rb:307:in `eval_file'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/danger_core/dangerfile.rb:200:in `block in parse'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/danger_core/dangerfile.rb:197:in `instance_eval'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/danger_core/dangerfile.rb:197:in `parse'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/danger_core/dangerfile.rb:283:in `run'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/danger_core/executor.rb:29:in `run'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/lib/danger/commands/runner.rb:73:in `run'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/claide-1.0.3/lib/claide/command.rb:334:in `run'
/Users/vagrant/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/danger-8.0.2/bin/danger:5:in `<top (required)>'
/Users/vagrant/.rbenv/versions/2.6.5/bin/danger:23:in `load'
/Users/vagrant/.rbenv/versions/2.6.5/bin/danger:23:in `<top (required)>'
[!] The exception involves the following plugins:
- danger-junit
I am running Danger as part of a Bitrise step as follows:
def processTestFiles(path, name)
if !Dir.glob(path).empty? then
junit.parse path
message = "## #{name}\n" <<
"Status | Number of tests |\n" <<
"--- | --- | \n" <<
" โ
| #{junit.passes.count} tests passed |\n"
if junit.failures.count > 0 then
message = message << " ๐ซ | #{junit.failures.count} tests failed |\n"
end
markdown(message)
junit.report # <--- Fails here
else
markdown("## #{name}\nโ๏ธ: No tests run")
end
end
......
Dir["build/split-results/*"].each do |f|
processTestFiles(f, File.basename(f, ".*"))
end
It was working OK in version 1.0.0
I'm trying to run danger-junit as one of the steps in my CI setup. This is the contents of my danger/UnitTestDangerfile
:
junit.parse "app/build/outputs/junit/merged-test-results.xml"
junit.show_skipped_tests = true
junit.report
But I get the complaint that I am using puts instead of message which is confusing for couple of reasons:
(1) I could not find any puts
command in this plugin
(2) That error message assumes I am running on GitHub
This is the raw output from my CI server:
[15:17:14]: $ bundle exec danger --danger_id=unit-tests --dangerfile=danger/UnitTestDangerfile --new-comment
[15:17:15]: โธ You used `puts` in your Dangerfile. To print out text to GitHub use `message` instead
Regarding (2) - although I can see from the code it should support GitLab too and there are else-clauses if we are running in an environment other than GitHub.
What can be wrong?
Our CI setup leads to multiple junit format files. The simplest reason is that our backend generates results from rspec whereas the frontend has jest tests.
The current version of this plugin supports only one file. Calling parse multiple times means that only the last one will get reported.
I'm going to make a PR to support another method: junit.parse_files
Hello @orta due to a dependency to the Ox gem, it appears the junit report functionality is broken on M1 machines, have you run into this issue? Do you have any advice or could you offer any help to work around this problem with danger-junit?
Raised it with the Ox project too in order to see if a fix could be developed there (bit sad as we do not quite know what it entails at the moment so not able to commit a PR to fix this issue at the source sadly): ohler55/ox#297
/CI/Jenkins/workspace/tests-NA-all-commits-ios-Lobby-pr/ruby_path/ruby/3.1.0/gems/ox-2.14.11/lib/ox.rb:79:in `require': dlopen(/CI/Jenkins/workspace/tests-NA-all-commits-ios-Lobby-pr/ruby_path/ruby/3.1.0/gems/ox-2.14.11/ext/ox/ox.bundle, 0x0009): tried: '/CI/Jenkins/workspace/tests-NA-all-commits-ios-Lobby-pr/ruby_path/ruby/3.1.0/gems/ox-2.14.11/ext/ox/ox.bundle' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e))) -
We researched the issue and tried the commonly suggested workarounds like:
Ensuring all gems would be installed with the native extensions recompiled for the host machine:
bundle config set --local force_ruby_platform true
and/or
bundle config set force_ruby_platform true
We do have ruby as one of the supported platforms in our Gemfile.lock btw:
PLATFORMS
arm64-darwin-21
ruby
x86_64-darwin-19
x86_64-darwin-20
x86_64-darwin-21
Adding the following platform restriction in the Gemfile:
# HACK(bouk): Overwrite Bundler's platform matcher to ignore universal CPU
# The protobuf and gRPC 'universal' macOS gems break on M1
module Bundler::MatchPlatform
def match_platform(p)
return false if ::Gem::Platform === platform && platform.cpu == "universal"
Bundler::MatchPlatform.platforms_match?(platform, p)
end
end
But nothing seems to do the trick :/...
References: protocolbuffers/protobuf#8199 (and similar ones)
FAILED source/ci_source/providers/_tests/_dockerCloud.test.ts
We are using the ci_reporter gem ( https://github.com/ci-reporter/ci_reporter_rspec ) with parrelle:spec (https://github.com/grosser/parallel_tests) which generate individual JUnit result file under the spec/reports directory.
The list of files is dynamic, based on the test name (i.e. when someone add a new spec file a new xml file pops up).
Currently there is no way to easily as for all file matching a pattern to be processed. File need to be listed out explicitly.
Hi, thank you for providing the useful library!
We would like to handle multiple result files and found the usage in the current master. However, it seems v1.0 hasn't been released yet.
https://rubygems.org/gems/danger-junit
Is there any concern to block the release? Thanks.
I don't have the permission to do it myself
Seems like this plugin doesn't support all kinds of JUnit files yet
TestFile.xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="2" failures="1">
<testsuite name="ThemojiUITests" tests="2" failures="1">
<testcase classname="ThemojiUITests" name="testExample()">
</testcase>
<testcase classname="ThemojiUITests" name="testIsBroken()">
<failure message="XCTAssertTrue failed - /Users/fkrause/Developer/themoji/ThemojiUITests/ThemojiUITests.swift:24">
</failure>
</testcase>
</testsuite>
</testsuites>
=> [#<Ox::Element:0x007ffed5b24928 @attributes={:classname=>"ThemojiUITests", :name=>"testExample()"}, @nodes=["\n "], @value="testcase">,
#<Ox::Element:0x007ffed5b24798
@attributes={:classname=>"ThemojiUITests", :name=>"testIsBroken()"},
@nodes=
[#<Ox::Element:0x007ffed5b246a8
@attributes={:message=>"XCTAssertTrue failed - /Users/fkrause/Developer/themoji/ThemojiUITests/ThemojiUITests.swift:24"},
@nodes=["\n "],
@value="failure">],
@value="testcase">]
[5] pry(#<Danger::DangerJunit>)> failed_tests.select { |test| test.nodes.count > 0 }
=> [#<Ox::Element:0x007ffed5b24928 @attributes={:classname=>"ThemojiUITests", :name=>"testExample()"}, @nodes=["\n "], @value="testcase">,
#<Ox::Element:0x007ffed5b24798
@attributes={:classname=>"ThemojiUITests", :name=>"testIsBroken()"},
@nodes=
[#<Ox::Element:0x007ffed5b246a8
@attributes={:message=>"XCTAssertTrue failed - /Users/fkrause/Developer/themoji/ThemojiUITests/ThemojiUITests.swift:24"},
@nodes=["\n "],
@value="failure">],
@value="testcase">]
Stack Trace
[!] Invalid `Dangerfile` file: undefined method `value' for "\n ":String. Updating the Danger gem might fix the issue.
# from Dangerfile:4
# -------------------------------------------
# puts "Parsing JUnit file at path '#{path}'"
> junit.parse(path)
# junit.report
# -------------------------------------------
/Users/fkrause/Developer/hacking/danger-junit/lib/junit/plugin.rb:100:in `block in parse'
/Users/fkrause/Developer/hacking/danger-junit/lib/junit/plugin.rb:100:in `select'
/Users/fkrause/Developer/hacking/danger-junit/lib/junit/plugin.rb:100:in `parse'
Dangerfile:4:in `block (2 levels) in parse'
Dangerfile:2:in `each'
Dangerfile:2:in `block in parse'
/Users/fkrause/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bundler/gems/danger-a67bc71710d9/lib/danger/danger_core/dangerfile.rb:193:in `eval'
/Users/fkrause/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bundler/gems/danger-a67bc71710d9/lib/danger/danger_core/dangerfile.rb:193:in `block in parse'
/Users/fkrause/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bundler/gems/danger-a67bc71710d9/lib/danger/danger_core/dangerfile.rb:189:in `instance_eval'
/Users/fkrause/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bundler/gems/danger-a67bc71710d9/lib/danger/danger_core/dangerfile.rb:189:in `parse'
/Users/fkrause/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bundler/gems/danger-a67bc71710d9/lib/danger/commands/local.rb:109:in `run'
/Users/fkrause/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/claide-1.0.0/lib/claide/command.rb:334:in `run'
/Users/fkrause/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bundler/gems/danger-a67bc71710d9/bin/danger:5:in `<top (required)>'
/Users/fkrause/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bin/danger:22:in `load'
/Users/fkrause/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bin/danger:22:in `<top (required)>'
Before working on a fix I'd just like to know if I'm missing something, or if I misunderstood how the JUnit format works.
It would be nice if we could customize failure message.
For example I'd like to show number of succeeded/failed tests.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.