Comments (17)
I don't understand the latest state of this issue.
Is it now built into Newman?
Or does it to be converted with a custom script by each guy who wants a xUnit report?
from newman.
Hi @mattaylor, this sounds like a great idea. We'll work on this and release an update. Will keep you posted! 👍
from newman.
+1 on this!
from newman.
FYI heres a brain dead livescript module to convert the json output to xunit...
report = require './testReport.json'
xml = '<?xml version="1.0" encoding="UTF-8"?>'
for suite in report.results
xml += "<testsuite name='#{suite.name}'>"
for test, count of suite.testPassFailCounts then xml += "<testcase label='#test' failures=#{count.fail} />"
xml += "</testsuite>"
console.log xml
from newman.
+1 on junit/xunit reporter to be built into newman
from newman.
This would be awesome to have junit/xunit built in.
from newman.
This would be fantastic since most CI tools can easily parse xunit style reporting.
from newman.
+1 one this. Any update? would be very nice to integrate with jenkins
from newman.
+1
from newman.
Newman 1.2.0 now supports junit-style output for tests. (Thanks @pal-thomassen)
Sample response:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="TestRequest" id="609a5233-6542-c2ed-b308-446101ac693d" timestamp="2014-12-02T07:06:21.186Z" time="51.50 ms" totalTime="103 ms" tests="4" iterations="2">
<testcase name="SampleTest1" successes="2" failures="0" />
<testcase name="SampleTest2" successes="2" failures="0" />
<testcase name="SampleTest3" successes="1" failures="1" />
<testcase name="SampleTest4" successes="0" failures="2" />
</testsuite>
</testsuites>
from newman.
The syntax of the report file does not seem to correspond to the junit supported syntax found at http://llg.cubic.org/docs/junit/
successes is not recognised.
failures property should live in the testsuite not testcase. Just tested with Jenkins junit plugin and success es/failures are not reported correctly because of malformed syntax
from newman.
Jenkins seems to expect the optional failure inside the testcase.
<testcase ...>
<failure message="I so fail" type="epic" />
</testcase>
from newman.
A <testsuite>
element corresponds to a request.
A single request can have multiple tests (represented by the tests[] array).
Each test corresponds to a <testcase>
.
In case of multiple iterations, each test is run multiple times. So having a single <failure>
element inside the <testcase>
isn't suitable.
The next newman version (v1.2.5) will have successes
and failures
attributes in the <testsuite>
element, that'll aggregate the counts in the constituent <testcase>
elements
from newman.
just tested 1.2.6. successes and failures are in the testsute and testcase node. The testsuite node is aggregating the test case node. However Jenkins still isn't parsing that format it seems. Jenkins reads that junit output, it shows them all as successful, even when there are failures in the junit xml.
from newman.
To add some additional detail to this, the Jenkins xsd for jUnit is located here: https://svn.jenkins-ci.org/trunk/hudson/dtkit/dtkit-format/dtkit-junit-model/src/main/resources/com/thalesgroup/dtkit/junit/model/xsd/junit-4.xsd
So, an example file would like like below, if there were an error. Seems like the current version that newman creates is missing the failures sumed up in the testsuites node, and its also missing the failure node within the testcase (if it had an error). For example:
<testsuite failures="1" tests="3">
<testcase classname="foo1" name="ASuccessfulTest"/>
<testcase classname="foo2" name="AnotherSuccessfulTest"/>
<testcase classname="foo3" name="AFailingTest" failures="1">
<failure type="NotEnoughFoo"> details about failure </failure>
</testcase>
</testsuite>
from newman.
+10
from newman.
Tested the patch from commit #184 and it worked. Thanks!
from newman.
Related Issues (20)
- Getting Error 403 Using Newman
- Automatic Token Refresh Not Working in Newman CLI When OAuth 2.0 Access Token Expires HOT 1
- There is a problem with the options for the language setting. HOT 1
- [Feature Request] Need Package Library feature in newman HOT 5
- Running collection with packages results in `pm.require` is not a function HOT 8
- Using JUnit reporter generates the wrong name per test
- Using json reporter generates the wrong items in the section 'run->execution'
- critical vulnerabilities from newman postman
- Option to not truncate stacktraces when using `--verbose ` HOT 2
- jose dependency is vulnerable moderate HOT 1
- Hub
- ERROR [tc.postman/newman:latest] (main) Could not start container: java.lang.IllegalStateException: Container did not start correctly.
- Option "--global-var" unknown when running newman as a Docker container HOT 1
- Newman tests run out of memory HOT 2
- Test failure reason details truncated in Newman HTML report
- Cannot read properties of undefined (reading 'setNextRequest') HOT 7
- Newman can't parse the http verb dynamicaly
- newman
- how to run NEWMAN automation with the help of Oauth 2,0 token?? HOT 1
- newman 6.2.0 exception - node:internal/event_target:1090 process.nextTick(() => { throw err; }); HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from newman.