aghassemi / tap-xunit

TAP to xUnit XML converter

tap-xunit

Converts TAP to xUnit XML format.

TAP output from testing frameworks such as tape or node-tap can be piped directly to tap-xunit

node test.js | tap-xunit

Installation

npm install tap-xunit -g

Usage

CLI

node test.js | tap-xunit

less results.tap | tap-xunit --package="MyCompany.MyTool" > results.xml

By default TAP comments are used as test-suite names and considered to mark test boundaries. CLI flag --dontUseCommentsAsTestNames can be used to turn that feature off, in which case comments are ignored and all assertions go inside a single <testsuite name="Default"> with name Default

Library

var converter = require('tap-xunit');

// Optional configuration
var opts = {}

var tapToxUnitConverter = converter(opts);

tapInputStream.pipe(tapToxUnitConverter).pipe(xUnitOutStream);

Options

Options can be passed as CLI arguments by being prefixed with --

dontUseCommentsAsTestNames

default: false

By default TAP comments are used as test-suite names and considered to mark test boundaries. This option can be used to turn that feature off, in which case comments are ignored and all assertions go inside a single <testsuite name="Default"> with name Default

replaceWithUnicodeDot

default: false

Whether the '.' in test-suite names should be replaced with a Unicode homoglyph. This feature exists because many xUnit reporters assume '.' in test-suite name implies package hierarchy, which may not be the case.

package

default: ''

If specified, all test-suites will be prefixed with the given package name. NOTE: replaceWithUnicodeDot option does not apply to package and . can be used to specify package hierarchy.

License

MIT

tap-xunit's Issues

Support Execution from a Docker Container and Control Format Output Options

It would provide more flexibility if this was made available through a Docker container, and also support StdOut output, along with a --format parameter.

Example:
docker run --rm -t -v $(System.DefaultWorkingDirectory)/DirectoryWithTAPFile:/src aghassemi/tap-xunit /src --format JUnit (or whatever other supported formats) > $(System.DefaultWorkingDirectory)/results.xml

My ultimate goal with this, is to covert the output from the GitHub Super Linter, into a supported format that can be then published as Test Results in Azure DevOps pipelines.

Outputs jUnit output not xUnit XML Report

Specs are here:

• https://xunit.net/docs/format-xml-v1#test
• https://xunit.net/docs/format-xml-v2#test

Images for readme

Make tap strict mode optional

The library was working fine for us with the previous version, without strict tap mode.

Is it possible to make it optional? I can send a PR, but I would like to know if there was a particular reason to do so.

Ava tests piped through tap and then tap-xunit and then to temrinal?

Hi, tap-xunit is working great for the purpose of Jenkins integration but I'd love to be able to have the output passed through so that I can have it both in xml and the terminal. Something like:

$ ava --tap | tap-xunit --file=foo.xml --passthrough | tap-spec

Would this be the kind of feature that could be added?

Upgrade tap-parser to latest version

Hi. Could you upgrade tap-parser version to the latest?

Missing string sanitation before passing failure text to xmlbuilder

line 33 in lib/serialize.js:
failureElement.txt(formatFailure(a.diag));

is prone to passing unicode characters that xmlbuilder can't handle which can cause tap-xunit to throw an error such as:

node_modules\tap-xunit\node_modules\xmlbuilder\lib\XMLStringifier.js:149
15:21:40 throw new Error("Invalid character (" + chr + ") in string: " + str + " at index " + chr.index);
15:21:40 ^
15:21:40
15:21:40 Error: Invalid character (�) in string:

As a result, no test results are generated.

The following pull request fixes the issue:
Add missing string sanitation #26

Test names are not generated if plan is in the beginning

If plan (1..1234) is written after all the tests, test names are taken from comments, however if it's being written before all the tests (which is very common), test names are generated as #123 assertionmsg instead.

Tests execution benchmarks (time statistic)

Could you add test execution time to the testsuite or maybe the testCase ?

Working with node-tap 14?

I'm running tap --no-coverage-report 'test/**/*.test.js' | npx tap-xunit against a suite of tests that has a failure and I'm not seeing any output. The installed tap version is 14.4.3.

ESLint output cannot be captured

tap-xunit expects particular TAP format, which is slightly different for ESLint

ESLint output

eslint . --format tap
20:55 $ ./node_modules/.bin/eslint . --format tap
TAP version 13
1..12
ok 1 - /Users/nkbt/github/npm-http-server/modules/__tests__/getProperty-test.js
ok 2 - /Users/nkbt/github/npm-http-server/modules/__tests__/parsePackageURL-test.js
ok 3 - /Users/nkbt/github/npm-http-server/modules/createPackageURL.js
ok 4 - /Users/nkbt/github/npm-http-server/modules/getMaxAge.js
ok 5 - /Users/nkbt/github/npm-http-server/modules/getPackage.js
ok 6 - /Users/nkbt/github/npm-http-server/modules/getPackageInfo.js
ok 7 - /Users/nkbt/github/npm-http-server/modules/getProperty.js
ok 8 - /Users/nkbt/github/npm-http-server/modules/parsePackageURL.js
not ok 9 - /Users/nkbt/github/npm-http-server/modules/resolveFile.js
  ---
  message: Unnecessary semicolon.
  severity: error
  data:
    line: 24
    column: 2
    ruleId: no-extra-semi
  ...
ok 10 - /Users/nkbt/github/npm-http-server/modules/ResponseUtils.js
ok 11 - /Users/nkbt/github/npm-http-server/modules/serveNPMPackageFile.js
ok 12 - /Users/nkbt/github/npm-http-server/server.js

Produced XML

eslint . --format tap | tap-xunit

<?xml version="1.0"?>
<testsuites>
  <testsuite name="Default" tests="1" failures="1" errors="0">
    <testcase name="#1 undefined">
      <failure>
          ---
          ...
      </failure>
    </testcase>
  </testsuite>
</testsuites>

And report in CircleCI (or any other CI), looks quite unhelpful

ava todos showing up as failures

Given a test that looks like this:

var test = require('ava')
test('normal test', t => t.pass())
test.todo('todo test')

ava's tap output looks like this:

TAP version 13
# normal test
ok 1 - normal test
# todo test
not ok 2 - todo test # TODO

1..1
# tests 1
# pass 1
# fail 0

that gets converted into xunit output that looks like this:

<?xml version="1.0"?>
<testsuites>
  <testsuite name="normal test" tests="1" failures="0" errors="0">
    <testcase name="#1 normal test"/>
  </testsuite>
  <testsuite name="todo test" tests="1" failures="1" errors="0">
    <testcase name="#2 todo test">
      <failure/>
    </testcase>
  </testsuite>
</testsuites>

I don't think the todos should be failures IMO, but I'm not sure if that's on AVA or this module

Not forwarding upstream failures

Consider the following failing test file

// test.js
throw new Error( 'beep' );

When run and piped to tap-xunit

$ node test.js | tap-xunit

the exit code for the pipeline is 0, as a non-zero error code when the process exits is not handled by tap-xunit.

To reproduce locally,

$ node -e 'throw new Error("beep");' | tap-xunit || echo 'Failed'

[eval]:1
throw new Error( "beep" );
^

Error: beep
    at [eval]:1:7
    at ContextifyScript.Script.runInThisContext (vm.js:50:33)
    at Object.runInThisContext (vm.js:139:38)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:635:30)
    at evalScript (bootstrap_node.js:462:27)
    at startup (bootstrap_node.js:163:9)
    at bootstrap_node.js:608:3
<?xml version="1.0"?>
<testsuites/>

I would expect, similar to other tap reporters, such as tap-spec, that an upstream failure would propagate through tap-xunit. For an example of handling upstream exit statuses, see tap-spec.

$ node -e 'throw new Error( "beep" );' | tap-spec || echo 'failed'

[eval]:1
throw new Error( "beep" );
^

Error: beep
    at [eval]:1:7
    at ContextifyScript.Script.runInThisContext (vm.js:50:33)
    at Object.runInThisContext (vm.js:139:38)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:635:30)
    at evalScript (bootstrap_node.js:462:27)
    at startup (bootstrap_node.js:163:9)
    at bootstrap_node.js:608:3


failed

For context, I am using tap-xunit on CircleCI. If upstream tests contain syntax errors, as opposed to test failures, the build succeeds, when, in fact, the build should fail. While I am aware of setting the pipefail option, I believe a better option would be for tap-xunit to account for upstream failures ensuring consistent behavior with other TAP reporters.

Input not being properly escpaed

if the tap output has ansi characters they aren't escaped before being shoved into the XML, which is invalid XML.

See: avajs/ava#723 (comment)

I'll probably submit a PR that I think should fix this

Exception thrown if assertion has no name

An exception is thrown if an assertion has no name / description, eg ok 123.

I guess this may not actually be valid tap output but it should probably be handled none the less.