Define required test spec format for test reporting
#12 · open · 5 comments
Comments
Randy424
*Test result reporting* - Do developers need specific formatting/information... See Vincent for his use of JUnit (XML)
Randy424
Update: not started :)
dtthuynh
sample of an existing result from Jenkins: ``` <?xml version="1.0" encoding="UTF-8"?> <testsuites name="Mocha Tests" time="2277.7990" tests="2" failures="1"> <testsuite name="Root Suite" timestamp="2023-04-26T21:35:24" tests="0" file="cypress/tests/postUpgrade/postUpgrade.spec.js" time="0.0000" failures="0"> </testsuite> <testsuite name="post upgrade validation" timestamp="2023-04-26T21:35:24" tests="2" time="2277.6440" failures="1"> <testcase name="RHACM4K-857: CLC: Post-Upgrade - Verify all managed clusters from before upgrade" time="71.4650" classname="post upgrade validation RHACM4K-857: CLC: Post-Upgrade - Verify all managed clusters from before upgrade"> </testcase> <testcase name="RHACM4K-862: CLC: Post Upgrade - create and import new clusters after upgrade" time="0.0000" classname="post upgrade validation RHACM4K-862: CLC: Post Upgrade - create and import new clusters after upgrade"> <failure message="Timed out retrying after 10000ms: expected '[ <button.pf-c-table__button>, 1 more... ]' to contain 'Ready'" type="AssertionError"><![CDATA[AssertionError: Timed out retrying after 10000ms: expected '[ <button.pf-c-table__button>, 1 more... ]' to contain 'Ready' at Object.checkCluster (https://multicloud-console.apps.acmqe-258fc2-zupgrade.gcp.dev09.red-chesterfield.com/__cypress/tests?p=cypress/tests/postUpgrade/postUpgrade.spec.js:5544:85) at Context.eval (https://multicloud-console.apps.acmqe-258fc2-zupgrade.gcp.dev09.red-chesterfield.com/__cypress/tests?p=cypress/tests/postUpgrade/postUpgrade.spec.js:3767:58)]]></failure> </testcase> </testsuite> </testsuites> ```
dtthuynh
this allows our reportPortal and Polarion to be updated with the testcase result that's ran: <img width="1286" alt="image" src="https://user-images.githubusercontent.com/53154476/234987211-9b3043a6-236c-4c01-b0a4-993d7eeca417.png">
Randy424
Should not be difficult to install and test. This enables integration with Polarion through XML artifact of test output.