a7ex / xcresultparser Goto Github PK

If run tests simultaneously on 2 devices for example we run tests on iPhone + iPad simultaneously, and as I understood the xcresult structure will be different from the run on 1 device, so txt results will be corrupted (it contains wrong failed tests count + doesn't contain any data about failed tests and locales)

and txt result after run the command: xcresultparser -o txt -s failed -f result.xcresult > results.txt

If the cobertura.sourceforge.net server is down, the tool will fatalError
Why not back up a local dtd file to avoid this problem ? @fadookie

When I run xcresultparser -o xml Test-logging.xcresult
output is like below.

<?xml version="1.0" encoding="UTF-8"?>
<testExecutions version="1">
    <file path="LoggingTests">
        <testCase name="testWhenSuccess()" duration="5"/>
        <testCase name="testWhenFailure()" duration="17"/>
    </file>
</testExecutions>

but when I passed this to sonar-scanner, it wasn't processed.
because sonar-scanner expects,
<file path="Full/Path/LoggingTests.swift">
https://docs.sonarqube.org/latest/analyzing-source-code/test-coverage/generic-test-data/#generic-test-execution

Is it possible to put the fullpath info here?

Is it possible to ignore specific directories/files? I'm currently using the parser for a Swift package and the parser is including in the results the files in ~/Library/Developer/Xcode/DerivedData which are coming from other package dependencies for which I'm not interested in their code coverage.

Thank you for this awesome project, it's been really useful.

We have a testplan with ~17 configurations (the difference between configurations are system region + system language) we use it to perform visual testing on different locales, also we've added 2 native retries in testplan configuration. I made some changes to fail 1 test on 2 locales.
And after generation txt result with this command:
xcresultparser -o txt -s failed -f result.xcresult > results.txt

I see in Summary 4 failed tests, but I assume that number should be 2, because other 2 it's just a retries, so the correct number should be 2 failed.

Would it be possible for you to add binaries to the releases here on GitHub? It would be helpful for a workflow we're using at my job.

First of all thanks for the great project!
It would be nice if it's possible to add json format to output type since we are using xcresult summaries in slack report messages, so It would be much easier if the whole summary data would be in json format, cuz it's easier to work with this format later :)

I've tried running the parser from brew and the binary from the latest release (1.4.2) and they both error out with the same error message/code combo:

XcresultparserLib/resource_bundle_accessor.swift:40: Fatal error: unable to find bundle named Xcresultparser_XcresultparserLib

The command I'm running follows the following pattern:

xcresultparser -p my-dir -o cobertura results/result.xcresult > output/coverage.xml

The weird thing to me is that I was running the parser successfully yesterday, using the exact same command, but today it just doesn't work. This makes me think that I'm the one making the error, but I can't figure out any difference between yesterday and today. One thing that I do notice is that the Package.swift declares a XcresultparserLib library and target, but there is no such directory with that name in Sources, only xcresultparser. I'm left wondering if that might be the issue.

Our project repo has capital letters in it, and we're using $(pwd) to get the current path (and using the --project-root option to correct the file paths so SonarQube can ingest the xml files). Test execution output seems to work fine with this (the paths are properly turned into relative paths), but coverage output seems to require the exact case-sensitive path as the option for --project-root.

Hello we have an issue where using the markdown format wont render span tag to Github actions step summary. Is there an existing way to fix this? If not I suggest having a new output format for markdown compatible with Github. Github has some alternatives for highlighting text listed here https://stackoverflow.com/questions/11509830/how-to-add-color-to-githubs-readme-md-file and in documentation https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax

Example command and result below.

https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/

      - if: failure()
        name: Generate Github Markdown
        run: |
          xcresultparser \
              --failed-tests-only \
              --output-format md \
              --quiet \
              'reports/TestResultsBundle.xcresult' >> $GITHUB_STEP_SUMMARY

We have a testplan with ~17 configurations (the difference between configurations are system region + system language) we use it to perform visual testing on different locales. Some of the tests are disabled:
example:

func testOne() throws {
throw XCTSkip("Disabled")
}

In txt report I see that skipped tests shows as failed on different locales, but I assume they should be excluded from the report and shouldn't be recognized as failed tests

the command I used to generate txt report is:
xcresultparser -o txt -s failed -f result.xcresult > results.txt

I'm evaluating a few options for producing html from xcresult bundles and xcresultparser is one of them.
I just want to ask if you have considered publishing the library on Homebrew? It would make a CI setup a lot easier.
Thanks