mavdi / grunt-cucumberjs Goto Github PK

Hey.

I faced with the problem.

I want to rerun failed scenarios, it's already implemented in cucumber-js to find failed ones - https://github.com/cucumber/cucumber-js#formatters (see rerun formatter).

When I add the array of failed scenarios like ['feature1.feature:23', 'master.feature:11', ...] to grunt configs it doesn't handle this.

Can you implement the feature?

The bootstrap theme specifically loads resources using https. This causes issues on internal systems that don't have SSL certificates.

It would be useful if they used the "//" shorthand like the Foundation template, to ensure that resources are loaded correctly.

I'd be happy to submit a PR for this.

I have my Grunt file defined as below:
module.exports = function(grunt) {

grunt.initConfig({


        webdriver: {
            test: {
                configFile: 'tests/acceptance/wdio/conf/cuke.conf.js',
                options: {
                    cucumberjs: {
                        options: {
                            format: 'html',
                            output: 'my_report.html',
                            theme: 'bootstrap'
                        }
                    }
                }
            }
        }


});

grunt.loadNpmTasks('grunt-webdriver');
grunt.loadNpmTasks('grunt-cucumberjs');

};

The path to the Feature file and other necessary configuration is given in the cuke.conf.js. On running the task grunt Webdriver, the scripts executed fine but no Report is generated.

Thanks
Simit

Earlier, the console output was saved to the HTML report.
I guess this is broken by @gkushang change: 20afb9e

Would be nice to have it back.

master...caseygoodhew:master

@yellamillis has had a PR open for a 11 days. We've made a couple of additional improvements, but we needed to keep work progressing, so we've forked and re-packaged to https://www.npmjs.com/package/grunt-cucumberjs-fork

I'll be happy to merge changes back in if this repo is still active. If not, I'm happy to run the fork as well.

Hello,

The fail fast option from cucumber is missing in the grunt-cucumberjs plugin.

Right now you have to use dirty tricks if you want to stop executing the test suite after a failed test like using the After hooks to exit if one scenario failed.

This is especially useful when you have lots of tests.

I've submitted the pull request #48 to solve this.

It would be really useful to provide JUnit format without using additional packages such as https://www.npmjs.com/package/cucumber-junit

JUnit format is required by jenkins.

Yesterdays HEAD was fine,
Upgraded to HEAD today:

My config:

    cucumberjs: {
        options: {
            debug: true,
            tags: [ "~@Pending "],
            format: "html",
            templateDir: "cucumber/report-template",
            output: "reports/cucumber.html",
        },features: [],

        signoff: {
            options: {
                tags: [ "~@Pending", "@signoff" ]
            }
        }
    } 

The template is a sligthly modified version of the bootstrap one. the default bootstrap doesnt work either

It seems that somewhere along the line of executing the cucumberjs task, global variables are lost. This is problematic when trying to use grunt-cucumberjs with something like grunt-istanbul which uses a global __coverage__ variable to store coverage information.

Following settings don't create my_report.json file with report.

grunt.initConfig({
  cucumberjs: {
    options: {
      format: 'json',
      output: 'my_report.json'
    }
  }
});

Just an idea, would it be an idea to submit the html reporting part to the cucumber-js project, so we help a larger audience with html reports and keep this project pure about running cucumber through grunt?

According to documentation to re-run failed TCs automatically 2 commands must be executed:

1. grunt cucumberjs -> report generated with failed TCs generated
2. grunt cucumberjs [email protected] -> truncated report generated only with re-ran TCs.

Is it possible to generate summary report which consist of all test cases which were ran?

Here is my grunt task

        cucumberjs: {
            src: 'src/test/features/originatedBy.feature',
            options: {
                formats: ['html','rerun:target/cucumber-reports/rerun.txt'],
                output: 'target/cucumber-reports/test-report.html',
                theme: 'bootstrap',
                debug: true,
                //reportSuiteAsScenarios: true

                //executeParallel: true
            }
        }

I have several different environments local/staging/production where cucumber-js must be run.

Is it simple mechanism to define environments such a parameters baseUrl, seleniumAddress etc and run these tests from command line or I need to develop something own?

I'd like to see both html and console for the same tests.

If I'm correct, that's currently not possible?

It looks like if you do npm install --no-bin-links the path "./node_modules/.bin/cucumber-js" dosen't exist but "./node_modules/cucumber/bin/cucumber.js" seems to be there when installed either way. Unfortunately this won't work for windows. I can make a PR for this but I'm not sure if this is the fix you want.

Due to cucumber library 1.0.0 release, when I try to generate report an error appears in console:
"Unable to parse cucumberjs output into json."

But it was ok before cucumber release.

Hi there,
I would like to keep the default output - aka 'pretty' - when generating the report.
I tried the formats: ['pretty', 'html'] config but when running only html outputs.

Am I missing something here?

Thank you!

Lacks dependencies on: jsonfile, find, open and xmlbuilder.

Hi, we're currently using your library in our project and now we're facing an issue with '--tags' parameter.
We're in a situation when we need to exclude features marked with tag A OR tag B.

According to your spec, --tags parameter takes String or Array as a value, and in case we need OR condition in expression, we got to use String version and write down tags separated with comma.

So our tag expression looks like --tags='~@A,~@B', but for some reason during the tests run it excludes only features tagged with B. We also tried to hardcode this in cucumberjs configuration, like
tags: '~@A, ~@B' and it works just the same.

However, I got to tell that 'Array approach' works fine.
Is it some kind of a bug or we're doing something wrong?

Hello!

I'm using version 0.8.2 and I'm having an error when running cucumber tests with the html format output option.
The error is

Fatal error: Unable to read "node_modules/grunt-cucumberjs/templates/foundation/piechart.js" file (Error code: ENOENT).

The file piechart.js seems to be present in the bootstrap template, but not in the foundation one.

The piechat should just be skipped when using foundation, instead of crashing the whole thing

Edit: It works fine if you use bootstrap

OS: Ubuntu 16.04
From Windows 10 - everything OK

package.json

"cucumber": ">=1.3.1",
    "grunt": "~1.0.1",
    "grunt-cli": "~1.2.0",
    "grunt-cucumberjs": ">=0.10.12",
    "protractor": ">=4.0.14",
    "exceljs": ">=0.2.39"

nodeVersion = v7.3.0
npmVersion = 3.10.10

run from cli
grunt cucumberjs --features=src/test/features/some.feature

output

Running "cucumberjs:src" (cucumberjs) task
 /home/user/tests-ui-js/node_modules/fs-finder/lib/Base.js:33
         throw new Error("Path " + directory + " is not directory");
         ^
 Error: Path /home/user/tests-ui-js/src/test/features/notes.feature is not directory
     at Finder.Base (/home/user/tests-ui-js/node_modules/fs-finder/lib/Base.js:33:15)
     at new Finder (/home/user/tests-ui-js/node_modules/fs-finder/lib/Finder.js:23:43)
     at Function.Finder.from (/home/user/tests-ui-js/node_modules/fs-finder/lib/Finder.js:34:15)
     at getFeatures (/home/user/tests-ui-js/node_modules/cucumber-parallel/lib/tasker/tasks.js:40:31)
     at Object.getTasks (/home/user/tests-ui-js/node_modules/cucumber-parallel/lib/tasker/tasks.js:78:15)
     at EventEmitter.<anonymous> (/home/user/tests-ui-js/node_modules/cucumber-parallel/lib/parallelizer/parallelizer.js:48:23)
     at emitNone (events.js:86:13)
     at EventEmitter.emit (events.js:185:7)
     at EventEmitter.<anonymous> (/home/user/tests-ui-js/node_modules/cucumber-parallel/lib/parallelizer/parallelizer.js:35:22)
     at emitNone (events.js:86:13)
     at EventEmitter.emit (events.js:185:7)
     at Object.invoke (/home/user/tests-ui-js/node_modules/cucumber-parallel/lib/parallelizer/parallelizer.js:103:22)
     at Object.<anonymous> (/home/user/tests-ui-js/node_modules/cucumber-parallel/bin/cucumber-parallel:3:16)
     at Module._compile (module.js:541:32)
     at Object.Module._extensions..js (module.js:550:10)
     at Module.load (module.js:458:32)

It would appear that 'Background: ' steps do not have a result object and these are still processed when generating the report.

That said, for the purpose of report generation it would be nice to at least have reference to what the Background ran was. I would have through these should still have a result object, since these pre conditions could still fail for various reasons

e.g:

Background Step (no result object):

...
"elements": [
      {
        "name": "User is already on Sign In Page",
        "keyword": "Background",
        "description": "",
        "type": "background",
        "line": 7,
        "steps": [
          {
            "name": "I visit the Sign In Page",
            "line": 8,
            "keyword": "Given "
          }
        ]
      }
...

Scenario Step:

.....
"steps": [
          {
            "name": "I visit the Sign In Page",
            "line": 8,
            "keyword": "Given ",
            "result": {
              "duration": 546019,
              "status": "passed"
            },
            "match": {}
          }
....

Error caused when processing this part of task/cucumber.js

            if(step.result.status === 'passed') return element.passed++;
            if(step.result.status === 'failed') return element.failed++;
            if(step.result.status === 'undefined') return element.notdefined++;

Will it be possible to run grunt cucumber task in debug mode, so debugger can be attached from WebStorm or another IDE.

The Bootstrap and Foundation index.tmpl used to generate the HTML reports are missing the http scheme on the stylesheet and script links.

They are incorrectly referenced as "//..." rather than "http://...", which means the generated HTML report doesn't render properly.

Is this still a feature?

• Run failed scenarios by passing --rerun=path/to/@rerun.txt grunt option

I can generate the file of failed scenarios but cannot rerun them by passing the argument on the command line.

The lines committed here [https://github.com/mavdi/grunt-cucumberjs/commit/34a8157c62c49d10e574670e45c51f3380602137] have been removed or refactored somehow?

I've implemented my own grunt task to rerun from the file but I'm not able to specify individual scenario lines, rather entire feature files. Is this a symptom of the way the parameters are parsed (the line numbers are denoted after a colon) or was the the ability to rerun the individual scenarios not part of the rerun option previously anyway?

Thanks

Yes! cucumber-js 1.0.0 is released (gherkin 4.0.0).

When upgrading, my tests still work, but it looks like the json output has changed and is not able to create a html report.

Unable to parse cucumberjs output into json.

I see following error installing grunt-cucumberjs v1.0.0

npm ERR! node v4.6.1
npm ERR! npm  v3.10.10
npm ERR! code ETARGET
npm ERR! notarget No compatible version found: [email protected]

When I have a step in my feature file using a data table:

         And I fill in the slots:
        | slotName | type      | attribute        |
        | Value    | member    | Audience Score % |

This is what my output (bootstrap html) looks like:

I expected the arguments to be rendered.

This is the json output:

        "arguments": [
          {
            "rows": [
              {
                "cells": [
                  "slotName",
                  "type",
                  "attribute"
                ]
              },
              {
                "cells": [
                  "Value",
                  "member",
                  "Audience Score %"
                ]
              }
            ]
          }
        ],
        "keyword": "And ",
        "name": "I fill in the slots:",
        "result": {
          "status": "passed",
          "duration": 5455250
        },
        "line": 10,
        "match": {
          "location": "/home/lennart/dev/github/idvizcontrols-functional/features/step_definitions/ShelfControlStepDefinitions.js:21"
        }

When I insert a reference error on one of my source files
And I run cucumber
Then I expected the job to fail
But instead, it runs no tests and Grunt says "Done, without errors."

it does print the error above it, with stacktrace: e.g.
ReferenceError: sadfs is not defined

Running cucumberjs directly would result in an exit code 1.

Result json and html files are not created, when tests are run in parallel mode using following command:
grunt cucumber-tests --parallel scenarios

When the same test suite is run without executeParallel: 'true' option everything work's as expected.

Here is an example of configuration file which is used to run TCs:

        cucumberjs: {
            src: ['src/test/features/'],
            options: {
                format: 'html',
                output: 'target/cucumber-reports/test-report.html',
                theme: 'bootstrap',
                debug: true,
                executeParallel: 'true'
            }
        }

I used this package to run cucumberjs tests. But it seems on Windows I am getting: Fatal error ENOENT when I try to run them with grunt. Do you know what might be the problem?

@mavdi: can you plublish the latest version?

Hey guys,

Found an issue where if you click on the feature in the html report it expands, but if you click on the scenario under it nothing happens. Found that this only happens for some features.

After some digging i found that this only happens when you have quite a large amount of features, but also quite a large amount of scenarios under a feature. The specific issue is in relation to the id used to target the div. It uses a system like this:

feature 2 and scenario 22 would be represented by #collapseScenario222
The issue rears its head if you also have 22 features, and then 2 scenarios under feature 22, its still represented by #collapseScenario222. This obviously causes problems...

I managed to fix this very easily by separating the feature number from the scenario number like so: #collapseScenario2_22 and #collapseScenario22_2.

The fix needs to be in the template files. I am using bootstrap so i changed all instances of <%= featureIndex %><%= scenarioIndex %> in 'templates/bootstrap/features.tmpl' to <%= featureIndex %>_<%= scenarioIndex %>. I guess you could just do the same for the other feature files?

Seems to work well for me anyway. Thought someone involved in this project would like to know and that it might help others.

Thanks guys, great project.
Shane

The package.json suggests this should work with "node": ">= 0.8.0". I'm using 0.12 at the moment and get the following error

Warning: /Users/luuc04/morph/morph-modules/comments-view/node_modules/grunt-cucumberjs/node_modules/cucumber-html-reporter/node_modules/fs-extra/lib/index.js:3
const assign = require('./util/assign')
^^^^^
Use of const in strict mode. Use --force to continue.

I'm not using the default html format (using pretty format), so should it still be trying to load this module?

Hi, we consider to use cucumber with protractor and your modul. In order to use with jenkins, is the html output in junit format?

thanks for your great work!

Received following error when trying to tests in parallel

Running "cucumberjs:src" (cucumberjs) task

cucumber parallel binary not found at path node_modules\parallel-cucumber\bin\parallel-cucumber-js
NOTE: You cannot install grunt-cucumberjs without bin links on windows

Command used: grunt cucmberjs --parallel scenarios

• [email protected]
• [email protected]

What I missed?

What template system are you using when you create the 'simple' html output for a test output? I created a script that takes the html file generated from running a test and places it in the body of an email. When I read the email on a phone it looks great - exactly to the html file generated by the test. When I look at it on a desktop or laptop it has the same text but none of the styling. Do you have any thoughts as to why this may be? I

I first time installed this plugin and I have a question: is it possible to show somehow TAGS information in report?

I need to have grouping of features/scenarios by tags.

It would be very helpful to extract the template and html report generation to a separate npm module with a Javascript API. I would like to use it in my https://github.com/mucsi96/nightwatch-cucumber module and I don't what to copy-past you html template related code.

Do you think it is possible?
Thanks
Igor

When using a compiler like babel, this needs to be run before the source files can be executed. Cucumber provides a --compiler option to support executing a compiler before running the tests. Without support for the flag, the babel require hook can be used, but this adds a load order requirement to ensure that hook is executed first.

Would it be possible to enable this option as part of the plugin?

When in 'output' parameter some folder structure is defined:

        cucumberjs: {
            src: ['src/test/features/'],
            options: {
                format: 'html',
                output: 'target/cucumber-reports/test-report.html',
                theme: 'bootstrap',
                executeParallel: 'true'
            }
        }

Than following error appeared:

$ grunt cucumberjs
Running "cucumberjs:src" (cucumberjs) task
>> fs.js:584
>>   return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
>>                  ^

When some folder structure is defined in 'output' parameter than plugin must create defined folder structure.

Any idea why this may be happening?

First of all great work on this, very impressive reporting outputs!

However we found 1 issue where the cucumber json output was not being parsed for the html report to be generated. When a test failed with a cucumber error output, it generated the output json in processHandler.js (around line 98) with the error first, followed by the json test results, but when it tried to extract the json test results for generating the html, it incorrectly pulled the errors in as well, causing it to be invalid json and it would give the error: "unable to parse json".

Here is an example of the output:

`
Error: Error: This is some error thrown by cucumber.
Error occurred on line ........

[
{
"id": "This-the-id",
"name": "This the name",
"description": "A description of the object"
},
{
"id": "This-the-second-id",
"name": "This the second name",
"description": "A description of the second object"
}
]
`

I managed to get it working by changing the regex on line 98 in processHandler.js from this:

var featureStartIndex = jsonOutput.search(/\[\s*{\s*\"\[\"/g);

to this:

var featureStartIndex = jsonOutput.search(/\[\s*{\s*\"/g);

It looks for the start of the json object and extracts it from the output.

The problem seemed to be that it wasnt finding the start of the json based on the regex it was using. It seems to work now and will still generate the html report when there are cucumber errors, showing these errors on the report.

If you can find a better solution then great, but this worked for me.

The fix for #56 doesn't work for me.
I've upgraded to the latest (0.8.8) (and verified the 'formats' option is in tasks/cucumber.js)

This is my config.

    cucumberjs: {
        options: {
            tags: [ "~@Pending "],
            // format: "html",
            formats: ["pretty", "html"],
            templateDir: "template",
            output: "reports/cucumber.html"
        },
        features: [],

        signoff: {
            options: {
                tags: [ "@signoff "]
            }
        }
    },

The html report is created, but no cucumber output is shown in the console.

I am getting the following error intermittently when running my cucumber tests, and producing a HTML report via grunt-cucumberjs:

"Unable to parse cucumberjs output into json."

It looks like that this is thrown on line 88 of cucumber.js.

I think this is happening because when my cucumber test suite finishes, it runs an After block, which in turn writes to sysout. This output in turn is then getting appended to the end of the JSON output, and thus is no longer valid JSON.

Somehow, the latest bump to 0.8.9 removed 0.8.8 from npm.

npm info grunt-cucumberjs is returning the below (note how 0.8.8 is missing). Was it unpublished perhaps??? (i'm running npm 3.4.1)

{ name: 'grunt-cucumberjs',
  description: 'Generates documentation from Cucumber features',
  'dist-tags': { latest: '0.8.9' },
  versions:
   [ '0.1.0',
     '0.1.1',
     '0.1.2',
     '0.1.3',
     '0.1.4',
     '0.1.5',
     '0.1.6',
     '0.1.7',
     '0.1.8',
     '0.2.0',
     '0.2.1',
     '0.3.0',
     '0.4.0',
     '0.4.1',
     '0.5.0',
     '0.5.1',
     '0.5.2',
     '0.7.0',
     '0.7.3',
     '0.8.0',
     '0.8.1',
     '0.8.2',
     '0.8.3',
     '0.8.4',
     '0.8.5',
     '0.8.6',
     '0.8.7',
     '0.8.9' ],
  maintainers:
   [ 'jozzhart <[email protected]>',
     'kushang <[email protected]>',
     'mavdi <[email protected]>' ],
  time:
   { modified: '2016-03-20T18:42:05.576Z',
     created: '2013-10-14T15:34:58.037Z',
     '0.1.0': '2013-10-14T15:35:00.676Z',
     '0.1.1': '2013-10-14T15:38:54.411Z',
     '0.1.2': '2013-10-14T16:05:34.639Z',
     '0.1.3': '2013-10-14T16:11:55.797Z',
     '0.1.4': '2013-10-14T16:14:38.602Z',
     '0.1.5': '2013-10-17T12:14:38.260Z',
     '0.1.6': '2013-10-17T16:09:23.385Z',
     '0.1.7': '2013-10-28T14:40:11.023Z',
     '0.1.8': '2013-12-11T17:07:00.052Z',
     '0.2.0': '2013-12-12T10:41:10.904Z',
     '0.2.1': '2013-12-12T14:34:06.977Z',
     '0.3.0': '2014-04-15T10:37:42.100Z',
     '0.4.0': '2014-07-28T10:27:30.606Z',
     '0.4.1': '2014-07-28T10:30:11.009Z',
     '0.5.0': '2014-11-27T13:09:24.684Z',
     '0.5.1': '2015-01-08T17:15:53.508Z',
     '0.5.2': '2015-03-16T02:04:56.027Z',
     '0.7.0': '2015-05-12T15:30:47.192Z',
     '0.7.3': '2015-07-28T08:33:02.205Z',
     '0.8.0': '2015-08-13T18:55:25.481Z',
     '0.8.1': '2015-11-18T18:20:36.813Z',
     '0.8.2': '2015-12-01T18:46:58.301Z',
     '0.8.3': '2015-12-02T17:46:29.510Z',
     '0.8.4': '2015-12-02T18:01:20.097Z',
     '0.8.5': '2015-12-17T20:38:07.548Z',
     '0.8.6': '2015-12-22T19:04:00.599Z',
     '0.8.7': '2016-01-07T18:17:51.571Z',
     '0.8.8': '2016-02-26T22:07:43.895Z',
     '0.8.9': '2016-03-20T18:42:05.576Z' },
  author: 'Mehdi Avdi <[email protected]>',
  repository:
   { type: 'git',
     url: 'git://github.com/mavdi/grunt-cucumberjs.git' },
  homepage: 'https://github.com/mavdi/grunt-cucumberjs',
  keywords: [ 'gruntplugin' ],
  contributors:
   [ 'Andrew Keig <[email protected]> (https://github.com/AndrewKeig)',
     'Jozz Hart <[email protected]> (http://jozzhart.com)',
     'Kushang Gajjar <[email protected]> (https://github.com/gkushang)' ],
  bugs: { url: 'https://github.com/mavdi/grunt-cucumberjs/issues' },
  readmeFilename: 'README.md',
  version: '0.8.9',
  licenses:
   [ { type: 'MIT',
       url: 'https://github.com/mavdi/grunt-cucumberjs/blob/master/LICENSE-MIT' } ],
  main: 'Gruntfile.js',
  engines: { node: '>= 0.8.0' },
  scripts: { test: 'grunt' },
  devDependencies:
   { cucumber: '^0.4.8',
     grunt: '~0.4.5',
     'grunt-cli': '~0.1.13',
     'grunt-contrib-clean': '~0.5.0',
     'grunt-contrib-jshint': '~0.10.0',
     'grunt-jsbeautifier': '^0.2.10' },
  peerDependencies: { grunt: '~0.4.5' },
  dependencies:
   { underscore: '~1.5.2',
     commondir: '~0.0.1',
     jsonfile: '^2.2.1',
     'js-base64': '^2.1.8' },
  gitHead: '4bd9b85a6acfb1ea059073fd3cbb988f6f880dd4',
  dist:
   { shasum: 'dd17be729e17de3a3d412d03daacdd94d1c79722',
     tarball: 'http://registry.npmjs.org/grunt-cucumberjs/-/grunt-cucumberjs-0.8.9.tgz' },
  directories: {} }

Thanks for accepting #104

Can you kick this up to npm please? (or let me know if there's something that I need to do...)

It goes straight to console.

Even with :

cucumberjs: {
  options: {
    format: 'json',
    output: 'cucumber.js',
    theme: 'foundation'
  },

I believe it is due to https://github.com/mavdi/grunt-cucumberjs/blob/master/tasks/cucumber.js#L70

Given the current logic, it could make sense to capture that beautiful json file we generate by default (with html reporter) and store it along.
That way we have a nice html report for human and a friendly json file for jenkins or other ci tools.

I need to test more but something like :

  grunt.file.write(options.output+'.json', JSON.stringify(featureOutput))

on line 181.

Maybe we could add an option save_json.

I would remove the reporter 'json' : it is not useful, as capturing the whole stdout is not a valid json anyway.

 $ node_modules/.bin/grunt cucumberjs > out   
 $ cat out | python -mjson.tool
 No JSON object could be decoded

The <script> and <style> elements use http:// urls which means they fail to load when the report is served over https because of mixed content security policy. This means the collapsible parts of the report don't work.

They should either omit the protocol part of the url or use https everywhere.

Can we use "grunt": "^1.0.1" ?

currently:
"peerDependencies": {
"grunt": "~0.4.5",
"cucumber": "^1.0.0"
},