matthewmueller / component-test

Minimal configuration component test runner supporting browser testing, phantomjs, and saucelabs.

component-test

Minimal configuration component test runner supporting browser testing, phantomjs, and saucelabs.

Installation

npm install -g component-test

API

Browser

In browser testing

component test browser

Phantom

Test using phantomjs.

component test phantom

Saucelabs

Test using saucelabs.

component test sauce -b "iphone 6.0 Mac OS 10.8, firefox 5 Windows XP"

Travis

First download the travis gem:

gem install travis

Encrypt your credentials:

travis encrypt SAUCE=username:accessKey -r component/domify

Then add the browsers you want to test. Here's what the component/domify .travis.yml looks like:

language: node_js
node_js:
  - '0.10'
env:
  global:
    - secure: PT7QR0DCGcZ+JjH4kpsaGHPYyjdbKbm44Wj9nsHA/QgIG/FRBCvsAU1OyaWHF8KCP6p0TEfhgeW+vUlZfxlbpDa4eSR8kjqkcNTR1CWKqYfjjQzBFzibrmLE+gKOnIRxFhdwUjzatRc1A2B+jX/03HoIgsUhnG83xiinz21dBJI=
  matrix:
    - BROWSER="iphone 6.1 Mac 10.8"
    - BROWSER="ipad 5.1 Mac 10.8"
    - BROWSER="firefox 6 Mac 10.6"d

And here's the result:

TODO

• Offer higher-level API over saucelabs in separate repo (like zuul)
• Use Travis API for encryption (eg. mimic behavior of travis encrypt)
• Provide API for adding browsers to .travis.yml

License

(The MIT License)

Copyright (c) 2013 Matthew Mueller <[email protected]>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

component-test's Issues

code coverage?

thoughts on implementing code coverage reporting using yields/coverage?

cli api could resemble:

$ component test {browser,phantom} --coverage

anyway to change the default mocha.html ?

nice tool man!

it looks like there's no way to change the default mocha.html file ?

this what i see when i component test browser:

i think it's because i use mocha-matrix reporter...

fix window.onerror for IE9

browser: automatically close the browser if the tests pass

would be cool if the tests ran in the background as well (i don't know how to do that).
also, it should log in the console that the tests passed otherwise people will be like "WTF JUST HAPPENED"

investigate saucelabs test inconsistencies

Seems to be related to browser.eval in mocha-cloud.

Some of the errors:

{"message":"Error response status: 13. ","status":13,"cause":{"status":13,"message":"Job 635533192a704820afdeba73c33be9d4 has already finished, and can't receive further commands. It may have experienced an error. You can learn more at https://saucelabs.com/jobs/635533192a704820afdeba73c33be9d4","value":{"message":"Job 635533192a704820afdeba73c33be9d4 has already finished, and can't receive further commands. It may have experienced an error. You can learn more at https://saucelabs.com/jobs/63553...

and...

{"message":"Not JSON response","data":"ERROR Job 8af03cddfe86472e8c679e6f6ab9fb4f is not in progress. It may have recently finished, or experienced an error. You can learn more at https://saucelabs.com/jobs/8af03cddfe86472e8c679e6f6ab9fb4f"}

add yields/wd-browser

for a better API

options for mocha

Not sure if i'm missing something here but is there currently a way to pass options (timeout, reporter etc) to mocha? Would be cool if it were the exact same interface as mocha itself:

  component test phantom \
    --reporter spec \
    --timeout 10000 \
    --bail

I tried to use mocha.opts but this didn't work either.

allow custom build dirs

pass through arguments to phantom

integrate yields/gravy

way better implementation. will need some additional support (with the fixes i made for saucelabs)

use suite.title for page title

this is really just my ocd, but when using component test browser, i don't want the page title to always be "domify tests"

Add --tunnel option to component test <browser>

Useful for quickly opening up interactive sessions on saucelabs.

build before testing

I was confused as how this worked as I didn't do a component build yet.

The save bet would be to mention in the README that you should add add the command to your Makefile or issue make && component test browser.

More magic, this could internally run component install && component build or make, but that's likely to break e.g. one might depend on multiple steps in their makefile and others might have a makefile that doesn't even component build.

travis.ci testing is confusing/doesn't work for projects that don't have build.js committed

Might just need to update the docs on this one. Was confused until I was able to fix it with these changes: jb55/humanize-string@dd0c95d

browser: `component test` should default to `component test browser`

Feature Request: Support Folders for Test Scripts

I've got a component with it's functionality distributed throughout lib. As such, my test directory has the same tree defined, where each test file matches 1-1 with lib.

Currently, the glob pattern is test/*.js, which of course doesn't traverse all my test files. Could we either change the glob pattern to be test/**/*.js? (or at least allow some way to configure that glob pattern externally?)

Clearly document you are using localtunnel.me and not saucelabs connect

The fact you are using a 3rd party service to run the local tunnel affects privacy and security and should be clearly documented.

Also the commands should be component-test not component test

`component-test phantom` doesn't work on Windows

Spawn doesn't work for *.cmd on Windows.

browser: choose which browser to open

something like component test browser safari and component test browser "Google Chrome".

browser: kill the server after it's done

might take some work to figure out when all the relevant requests are done. don't like CTRL + Cing all the time.

return script errors from sauce labs and close connection

server returns wrong stuff when build.js or build.css is missing

because component doesn't create them if they're empty.
this is a server issue - don't use /*!

i can fix this after you merge my other PRs

FYI: I set free component-test name

Hi @matthewmueller,

Thanks for cool library!

Historically I borrowed component-test name in npm, and I've just renamed it to component-testem, because it makes more sense in case, what my library did. Feel free to rename yours, if you prefer ;)

All the best,
Aleksey

P.S.
npm site is cached previous version of the page https://npmjs.org/package/component-test, but you can make sure that it's free from terminal.

$ npm info component-test
npm ERR! 404 'component-test' is not in the npm registry.

Variables with the same name clash between test files

It'd be nice if the tool surrounded the contents of the test files in (function() { //code })() so users don't have to.

make sure we have a reasonable error when phantomjs isn't installed

Bring your own mocha.html

Suppose you have some html fixtures or custom css that you need during your tests.

Like here: https://github.com/Swatinem/scrollbars/blob/master/test/index.html

Would be cool if you could bring your own mocha.html or whatever that should be called.