Comments (5)
If you use the automation ID, you should see how matching happens in this article: TestRail CLI - Code first documentation. It matches a custom field named automation_id
, not the actual test name. When test cases are automatically created using the default --case-matcher auto
, the automation_id
is added to later be able to map the results to the correct test cases and not duplicate them.
from trcli.
Hello @EOssesFrazoni
Thank you for bringing this to our attention. The bug fix has just been released in v1.4.1
, which should be available in a few minutes via PyPI.
from trcli.
Hi @d-rede. It is fixed now, it is creating new test cases but I see two issues:
- It always creates duplicated test cases, even when they have the same name (and have been created by the same TRCLI)
- When running with an existing Run Id (using --run-id), it tries to create the test cases again, then it deletes all and finishes with this error:
Field :results contains one or more invalid results (case C863106 unknown or not part of the test run)
I just want to update a TestRun with a new set of results for existing test cases.
Note: the Run Id I passed as an argument was created by the a previous TRCLI run.
from trcli.
Hello again,
It always creates duplicated test cases, even when they have the same name (and have been created by the same TRCLI)
The --case-matcher name
actually doesn't match the cases by name. It expects you to have the ID of the TestRail case in the name of the test instead. You can read more about it in the TestRail CLI - Specification first documentation. We recommend that you don't use the automatic test case creation if you are using the name/property matcher.
When running with an existing Run Id (using --run-id), it tries to create the test cases again, then it deletes all and finishes with this error:
Field :results contains one or more invalid results (case C863106 unknown or not part of the test run)
I just want to update a TestRun with a new set of results for existing test cases.
Note: the Run Id I passed as an argument was created by the a previous TRCLI run.
When using the --run-id
to update a run, it is expected that you will be submitting results for the same tests. If you use the name/property matcher and don't have your TestRail case IDs in your automation code, new tests will be created and won't match the existing tests in the given test run. Once again, we discourage using automatic test creation (-y
) if you use the name/property matcher.
from trcli.
I didn't specify it but In my attempts I tried with and without the case-matcher parameter.
I've tried multiple combinations in the CLI but I'm still not able to do what I expect (adding test results for existing test cases in an existing test run).
If I use the run-id without case-matcher and Auto-create entities: False, but I still can get it working.
I'm expecting to use the first mechanism; "Using Automation ID"
command:
trcli -h "https://host/" -u "[email protected]" -p "key" --project "Project Name" --project-id 1 -n parse_junit --title "Pytest POC Suite" --run-description "Pytest POC Suite" -f "result.xml" --case-fields custom_automation_type:1 --run-id 11197
Test Run 11197 contains the a test case with the same name from my result.xml
This is the result:
`TestRail CLI v1.4.1
Copyright 2021 Gurock Software GmbH - www.gurock.com
Parse JUnit Execution Parameters
Report file: result.xml
Config file: None
TestRail instance: https://host.net/ (user: [email protected])
Project: Existing Project
Run title: Pytest POC Suite
Update run: 11197
Add to milestone: No
Auto-create entities: False
Parsing JUnit report.
Processing suite - None
Processed 1 test cases in 1 sections.
Checking project. Done.
Found 1 test cases not matching any TestRail case.
Updating run: https://host/index.php?/runs/view/11197
Adding results: 0/0, Done.
No attachments found to upload.`
from trcli.
Related Issues (20)
- support Robot Framework 7.0 new output.xml format HOT 1
- --case-fields not working when I attempt to push data to some of the fields (i.e. Type, custom-field in Testrail ) HOT 7
- Test plan name for test runs HOT 3
- unable to import results for test cases having a very long title HOT 2
- Test plan name for test runs HOT 3
- Set automation_id case field when using --case-matcher: "property" HOT 1
- TRCLI failed on the ci cd HOT 5
- not able to upload results when report contains some chars in comment HOT 5
- Parsing testrail_case_field in Junit XML fails when using value but text is not empty
- Unable to parse field title in TestRailCase tag. Title is empty. HOT 15
- Please add parsing "Elapsed" time to parse_robot command. HOT 6
- Empty sections created while using matching based on automation_id
- Empty sections created while using matching based on automation_id HOT 1
- Network error when importing test case from a specific test suite HOT 4
- Connection error while uploading test results HOT 14
- Add options to import stdout/stderr from junit test as comment HOT 2
- not able to upload results when report contains some chars HOT 4
- Unable to update test results generated in junit format from pytest in CircleCI HOT 1
- Unable to update test results generated in junit format from Cypress in CircleCI
- With code first approach, trcli creates duplicates of test cases then chokes on deadlock trying to cleanup and fails
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from trcli.