CUCUMBER: UNDEFINED state on step shown as PASSED about bootstraped-multi-test-results-report-plugin

[BogdanLivadariu]

@Vovanchik is this file the actual report that cucumber-js outputs ? or you have tweaked it.
I'm asking this, because the structure of the file differs from what cucumber-java outputs ...

[BogdanLivadariu]

this is the structure of cucumber-java json reports

[
    {
        "id": "account-holder-withdraws-cash",
        "tags": [
            {
                "name": "@super",
                "line": 1
            }
        ],
        "description": "As a Account Holder\nI want to withdraw cash from an ATM\nSo that I can get money when the bank is closed",
        "name": "Account Holder withdraws cash",
        "keyword": "Feature",
        "line": 2,
        "elements": [
            {
                "description": "",
                "name": "Activate Credit Card",
                "keyword": "Background",
                "line": 7,
                "steps": [
                    {
                        "result": {
                            "duration": 107447000,
                            "status": "passed"
                        },
                        "name": "I have a new credit card",
                        "keyword": "Given ",
                        "line": 8,
                        "match": {
                            "location": "ATMScenario.I_have_a_new_credit_card()"
                        }
                    },
                    {
                        "result": {
                            "duration": 16000,
                            "status": "passed"
                        },
                        "name": "I confirm my pin number",
                        "keyword": "When ",
                        "line": 9,
                        "match": {
                            "location": "ATMScenario.I_confirm_my_pin_number()"
                        }
                    },
                    {
                        "result": {
                            "duration": 14000,
                            "status": "passed"
                        },
                        "name": "the card should be activated",
                        "keyword": "Then ",
                        "line": 10,
                        "match": {
                            "location": "ATMScenario.the_card_should_be_activated()"
                        }
                    }
                ],
                "type": "background"
            }
        ],
        "uri": "net/masterthought/example/ATM.feature"
    }
]

and this is what you have provided

{
  "name": "REQ#2 - Delete assessment",
  "id": "Assessment;req#2---delete-assessment",
  "line": 18,
  "keyword": "Scenario",
  "description": "",
  "type": "scenario",
  "tags": [
    {
      "name": "@PROD",
      "line": 17
    },
    {
      "name": "@PATIENT",
      "line": 17
    },
    {
      "name": "@ASSESMENTS",
      "line": 17
    }
  ],
  "steps": [
    {
      "name": "I am on the login page",
      "line": 19,
      "keyword": "Given ",
      "result": {
        "status": "passed",
        "duration": 1681118300
      },
      "match": {
        "location": "/data/build/branches/Registry-Trunk-UI/cucumber_js/tests-ui-js/src/test/features/step_definitions/assessments/assessments_steps.js:40"
      }
    },
    {
      "name": "I should login as \"administrator\" and \"123qweasd\" password",
      "line": 20,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "name": "I should open patient",
      "line": 21,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "name": "I should add \"Asthma Initiative\" assessment",
      "line": 22,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "name": "I should make completed assessment",
      "line": 23,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "name": "I should add \"Asthma Initiative\" assessment again",
      "line": 24,
      "keyword": "Then ",
      "result": {
        "status": "undefined"
      }
    },
    {
      "keyword": "After ",
      "hidden": true,
      "result": {
        "status": "passed",
        "duration": 84556816
      },
      "match": {
        "location": "/data/build/branches/Registry-Trunk-UI/cucumber_js/tests-ui-js/src/test/features/step_definitions/support/world.js:11"
      }
    }
  ]
}

[Vovanchik]

Hi @BogdanLivadariu,

Ops, It look's like I cut too much. :(

I am using grunt-cucmberjs plugin which producing json file.

Attaching full source in txt file.

test-report-json.txt

[BogdanLivadariu]

@Vovanchik by the looks of it this report does not has undefined steps or tags

can you attach a file that has undefined steps & tagged reports ?

[Vovanchik]

Ops... Hard working day.

Adding file with undefined steps.

What do you mean under tagged reports? I have only tagged scenarios.
In my file it's shown as:
"tags": [ { "name": "@PROD", "line": 3 }

test-report-json.txt

[Vovanchik]

Hi,

Any updates on my issue?

[BogdanLivadariu]

@Vovanchik I'm on vacation :) will try to fill in with updates next week, when I get back :D

[Vovanchik]

Have a nice vacation. Waiting for your results.

[Vovanchik]

For steps it look's very good.

If some of the steps are in undefined state can we marked Scenario as PASSED instead of FAILED as it is right now.

What I see there not status field for scenarios, so plugin decided which status to set.

What do you think?

[BogdanLivadariu]

@Vovanchik I guess I can add an option in the plugin config, that you can tick
so that undefined steps are ignored, and
in a scenario if you have undefined steps, scenario will be marked as passed

what do you think ?

[BogdanLivadariu]

please create a new issue to cover this case, as it is an enhancement :D

[Vovanchik]

Cool thank you ;)

[Vovanchik]

New issue #49 created.