dowjones / fiveby

make selenium tests easier to setup, write, and execute

Fiveby

All the things you expect from a robust testing framework by neatly packaging: WebDriverJS, mocha, and should chai with a little glue and zero magic:

var fiveby = require('fiveby');

fiveby(function (browser) {
  return describe('Google Search in ' + browser.name, function () {
      it('should work', function () {
        browser.get('http://www.google.com');
        var searchBox = browser.findElement(by.name('q'));
        searchBox.sendKeys('awesome');
        return searchBox.getAttribute('value').then(function (value) {
          'awesome'.should.equal(value);
        });
      });
    });
});

Run it (quick)

npx mocha tests/** --delay --timeout 30000

OR (recommended)

Add gulp and some convention to make it even more powerful: slush-fiveby. slush-fiveby is a simple fiveby project generator/example.

What's unique about fiveby?

• Cleanly allows mocha and webdriverjs to coexist
• MUCH simpler configuration and less boilerplate code
• environment properties
• conveniences: api cleanup, spins up a selenium server if not provided, closes the browser for you, etc ...
• Sends test traffic to a HAR file
• more

Configuration (fiveby-config.json)

{
  "implicitWait": 5000,
  "hubUrl": null,
  "browsers": {
    "firefox": true,
    "chrome": {
      "version": "37.0.2062.103",
      "chromeOptions": {
          "args": ["--disable-extensions", "--headless"]
        }
    },
    "phantomjs": true
  },
  "disableBrowsers": false
}

disableBrowsers and hubUrl are optional, disableBrowser defaults to false.

• Use phantomjs 2.0: http://phantomjs.org/download.html (support exists but new users should use --headless)

English?

Have little to no experience with end to end testing?

Ok, this tool will allow you to write a bit of javascript that will open any browser (or mobile app), emulate user behavior via a few simple commands, and then verify what's displayed onscreen is correct. You can compile large suites of these tests and easily run them against many different browsers at once and get nice reports. It can be run with something like jenkins to automate further. Or use any of the popular SaaS providers like:

Pre-reqs

• node.js
• mocha cli (you can use npm scripts to avoid this)
• java (for selenium)
• the appropriate webdriver
• you can avoid installing the server, webdriver, and browser deps by using docker, see docker-compose.yaml @slush-fiveby

See docs folder for even more details!

fiveby's Issues

failing in an afterAll hook stops all test execution

I identified (proved) that this is happening in my 2.0 release research. This will definitely be fixed in 2.0 and may be release along with more critical bug fixes (if any) in 1.1.x

add keywords

automation, selenium, test, testing, webdriver, webdriverjs, mocha, should, bdd, environment, e2e, end-to-end, gulp, har, promise ...

?

should not assigned to variable

This line requires should but doesn't assign it to a variable thus causing any assertions like this to fail:

should.exist(somethingThatShouldExist)

Is there a reason for not assigning a variable to should?

switch to envprops, the evolved version of embedded props

https://github.com/areusjs/envprops 😄

Nemo evaluation

Hi @scott2449 I just watched your presentation at Node Summit. You and I have spent the last few months working on similar products (see: http://nemojs.com)! Given your experience, I'd love to get your feedback on Nemo. If you want to ping me today and meet in person (I'll introduce myself if I see you around), please do! @mattedelman / @nemojs_news (Twitter)

enhance documentation

A living list of things that need to be added to the docs:

• show local config override
• show phantom config/usage
• add something about selendroid and ios-driver

pixelmatch

It would be really cool to integrate this into fiveby somehow. An example at the very least to do a pixelmatch between two sets of e2e tests.

https://github.com/mapbox/pixelmatch

How can I connect to SauceLabs using the fiveby-config.json file?

I am trying to config fiveby to use with SauceLabs, but am not having a lot of success. I’m a bit green, so was wondering if you could point me in the right direction to get me past this.

I tried to use this for the value of hubUrl in my config file, but was getting authentication errors:
“hubUrl”: “http://SAUCE_USERNAME:[email protected]:80/wd/hub”,

If I use SauceConnect and add a gulp task to start SC, then set hubUrl to “http://127.0.0.1:4445”, then I don’t get authentication errors, but the tests fail (the slush generated tests) with error:
“TypeError: undefined is not a function”

What is the proper way to configure this (ideally without having to use SauceConnect)?

Consider mocha delay flag

mochajs/mocha#362

This may resolve some the reporter and test definition issues. In fact this may allow for yet another refactor of the way fiveby chains files&suites.

FYI @adamgruber

Upgrade to selenium 2.45 as default

https://github.com/SeleniumHQ/selenium/blob/master/javascript/node/selenium-webdriver/CHANGES.md

• Lots of great fixes to server and JS
• Test if headers come through properly in HAR/Phantom with this change

formatting standards

add contributing.md and or .editorconfig to help contributors keep things clean

upgrade to mocha 2+

RFC: including selenium jar?

I want it to be easy to start fiveby (and not have to spin up a selenium server separately) but that comes at the cost of having to download the fairly large selenium jar.. maybe we should include a command that will install the jar, check for java, etc.. then in code we can just check for the file as well. And of course none if this is an issue when the dev provides a url.

ChromeDriver Error

When chrome driver is not installed (or on the path) fiveby just skips the tests and "succeeds". It should fail with an explanation.

more detail on rejection of nightwatch

I'm curious about the specific issues that kept you from considering nightwatch in favor of building your own library? I also built a selenium abstraction and get asked a lot about this. For me, the main thing that steered me away from nightwatch was it proscribed a test runner and assertion library. I wanted to have the freedom to choose between mocha, jasmine, or cucumber (just to name some). Also, based on my company's requirements, the parallelization mechanism wasn't flexible enough.

Still, nightwatch is attractive to managers and others who see the nice website, extensive API, and large team of contributors... Curious about your opinion. Thanks!