sahava / gtm-datalayer-test Goto Github PK

More examples are needed, each with their own test spec and configuration JSON.

• Failing test
• Multiple browsers
• Multiple pages
• Complex structure using subset check only
• Complex JSON validation

Currently the only way (I have found) to test if an array (window.dataLayer) simply contains an object is to use the

"type" : "array
"not" : {
    "items" : {
        "not" : {
            "type" : "object",
            "properties" : ...

pattern as used in /lib/testUtils.js. However, this is not only really ugly, but it also lacks proper validation error messages. If the test fails, the only thing the reporter knows is that the spec validated against a "not" condition. The reporter doesn't know what part of the spec failed. That's why there's the "FAILED ON: " workaround, where I expose the object on which the test failed in the error message.

As far as I know, there's not much to do about this yet. There's the "contains" keyword coming up (hopefully) in a future JSON Schema draft, but even then I'm not certain if it's descriptive enough.

So we need to figure out how to properly do an "array contains..." so that the validation error is still expressive enough to be informative.

After errorless npm install stops npm test after http-server ./examples -s. On hard break appears an error

ERROR: connect ECONNREFUSED 127.0.0.1:4444 phantomjs at new RuntimeError (C:\gtm-datalayer-test\node_modules\webdriverio\build\li b\utils\ErrorHandler.js:143:12) at Request._callback (C:\gtm-datalayer-test\node_modules\webdriverio\build\l ib\utils\RequestHandler.js:330:43) at self.callback (C:\gtm-datalayer-test\node_modules\webdriverio\node_module s\request\request.js:185:22) at emitOne (events.js:116:13) at Request.emit (events.js:211:7) at Request.onRequestError (C:\gtm-datalayer-test\node_modules\webdriverio\no de_modules\request\request.js:877:8) at emitOne (events.js:121:20) at ClientRequest.emit (events.js:211:7) at Socket.socketErrorListener (_http_client.js:387:9) at emitOne (events.js:116:13).

Current master throws an error when installing from scratch:

/home/work/gtm-datalayer-test/node_modules/webdriverio/build/lib/launcher.js:162
throw _iteratorError;
^

Error: reporter "wdio-spec-reporter" is not installed. Error: Error: Cannot find module 'wdio-spec-reporter'
at Function.Module._resolveFilename (module.js:469:15)
at Function.Module._load (module.js:417:25)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Launcher.initReporters (/home/work/gtm-datalayer-test/node_modules/webdriverio/build/lib/launcher.js:137:41)
at new Launcher (/home/work/gtm-datalayer-test/node_modules/webdriverio/build/lib/launcher.js:73:31)
at Object. (/home/work/gtm-datalayer-test/node_modules/webdriverio/build/lib/cli.js:426:20)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at Launcher.initReporters (/home/work/gtm-datalayer-test/node_modules/webdriverio/build/lib/launcher.js:139:35)
at new Launcher (/home/work/gtm-datalayer-test/node_modules/webdriverio/build/lib/launcher.js:73:31)
at Object. (/home/work/gtm-datalayer-test/node_modules/webdriverio/build/lib/cli.js:426:20)
at Module._compile (module.js:570:32)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)

When using

capabilities: [{
    browserName: 'firefox'
}]

the Firefox browser starts, but it just stays on the first page. The browser doesn't navigate to the test page, nor does it run any of the tests.

Seems to have been an issue earlier, too: http://stackoverflow.com/questions/34536329/webdriverio-firefox-does-not-navigate-to-url-specified

Let's start thinking of getting rid of wdio. The only reason I chose wdio was because I wanted a test runner up as fast as possible.

Without wdio, we could setup Selenium, phantomjs, mocha, chai using e.g. Grunt to do the package and script handling.

Then, we could have more freedom to incorporate more complex interactions between the client and the server, such as adding https://github.com/google/data-layer-helper and run assertions when dataLayer.push() commands are registered. This way we could test each dataLayer.push() against the JSON as they come in, rather than the pretty complex method currently used to test the dataLayer as one big JSON lump.

Navigation support needs to be added. Basically anything that triggers a dataLayer.push() should be something you can write a test to.

Will need to figure out how to represent navigation in the JSON configuration, though.

Some way to block requests to any given endpoint (e.g. google-analytics.com).

Alternatively, use URL parameter to make blocking easier via e.g. GTM.

Hi Simo,

If you find it to be a good fit, I'd like to include some functionality we've been working on here at Napkyn Inc. We're a heavy user of gsheets for documenting QA results. As we begin rolling out our internal testing tools, we're looking for tools in the community that we could contribute some functionality to. Our contribution would be the gsheet scripts we use to munge the incoming data as well as the code we use to send the data to the sheets (integrated into your framework). I look forward to hearing your thoughts.

All the best,
Adam