arunoda / laika Goto Github PK

I'm trying to pass helper functions a la

server.eval(function(foo) {
  foo();
}, function() { console.log('bar') } );

Results in: TypeError: 'null' is not a function (evaluating 'foo()')

Forgive me if this is a naive approach, is there some other way to create helper functions that can be used throughout multiple tests()'s?

Laika should support, that Meteor app and package code as well as Laika test code is written in CoffeeScript.

When I run laika commmand, I'm greeted with four such error messages:

nodetime v0.8.12 error: [Error: Module version mismatch. Expected 11, got 1.] Error: Module version mismatch. Expected 11, got 1.
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at bindings (/Users/hubert/.meteorite/packages/nodetime/subhog/meteor-nodetime/71c155845b4d0c99ba471d1498d113cef499dbd6/.npm/node_modules/nodetime/node_modules/nodetime-native/node_modules/bindings/bindings.js:76:44)
    at Object.<anonymous> (/Users/hubert/.meteorite/packages/nodetime/subhog/meteor-nodetime/71c155845b4d0c99ba471d1498d113cef499dbd6/.npm/node_modules/nodetime/node_modules/nodetime-native/index.js:1:99)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)

After that, tests proceed normally. The nodetime package works smoothly in normal app run.

Current eventEmitter based test creation api just work. But it bit hard to use and see. Solution for this is to give some better tools to write tests

• evalSync(). support
• actions support
• default set of useful actions
• documentation

The first run actually injects the code, so I assumed it can't be it, but then it gets stuck at the loading part. Even if I give it a full minute.

At the second run, it starts up speedily fast, and just runs awesomely, also cleans up.

If I then run it again, it's like in the first try.

Ubuntu version: 12.10
Meteor version: Release 0.6.5
Laika version: 0.2.26

Also, not run with sudo, as I assumed it wouldn't matter.

meteor@multiply:~/repositories/chat$ laika

  injecting laika...
  loading phantomjs...
  loading initial app pool...
^Cmeteor@multiply:~/repositories/chat$ laika

  injecting laika...
  loading phantomjs...
  loading initial app pool...


  Levelling
    ✓ Create User at level 0 (1136ms)


  1 passing (1 seconds)

  cleaning up injected code

meteor@multiply:~/repositories/chat$

Any way I can provide more information than this?

I try to delay emitting events in my tests, something like this:

        client.eval(function() {
            Meteor.logout();
            Router.go("profile");
            setTimeout(function() {
                emit('checkUrl', {
                    current_path: Router.current().path,
                    expected_path: Router.path("home")
                });
            }, 5000);
        });

        client.on('checkUrl', function(dict) {
            assert.equal(dict.current_path, dict.expected_path);
            done();
        });

However it doesn't wait 5 minutes to emit. The test output shows it took about 1200ms and I get the undesired result for that test (as a result of not delaying the event).

I also tried using Meteor.setTimeout but the same issue.

In the following example the false value emitted is being received as null.
However, passing a true value instead of false is received okay.

suite('mySuite', function() {
  test('myTest', function(done, server, client) {

    client.eval(function() {
      emit('load', false);
    })
        .once('load', function( load ) {
          console.log( load );
          ( load ).should.be.false;
          done();
        });

  });
});

null
not ok mySuite myTest
  TypeError: Cannot read property 'should' of null
...

Hi! I just started learning Meteor this week, and I am trying to integrate laika as test framework. During this process I stumpled upon two questions:

1. I would like to do some kind of acceptance testing with asynronous actions, i.e. something like: "When user clicks button X, Y should happen". In code:

    test('in the server', function(done, server, client) {
      client.eval(function(){
        $("input[name=url]").val("http://github.com");
        $("button").click();
        // We have to wait until async operation is finished -> how?
        assert($("#output").text(), "This is github.com.");
        emit('done');
      });

      client.once('done', function(){
        done();
      });
    });

How do I solve this, do I have to use something like setTimeout?

1. I would like to use Sinon.js for mocking - how do I achieve this? As far as I see there is no package.json which is used by laika where I could ass Sinon as dependency.

Thx for you help!

Hi,

There was an issue previously reported on StackOverFlow:
http://stackoverflow.com/questions/17056068/meteorjs-laika-client-and-server-test-timout

I'm currently running nodejs 0.10.12 on MacOSX 10.7 and I am still experiencing the same issue.

Edit: I meant I'm on mac 10.8

Currently trying to get continuous integration to run properly on my local computer.
My machine is running Mac OSX 10.8

If I run the laika command in a folder that doesn't contain a meteor project (for example, if I misconfigured my CI configuration), laika will just hang on "loading phantomjs".

Edit:

I found this from your blog:

Replace test commands with following
METEOR_PATH=~/meteor laika -t 5000

The METEOR_PATH should be explained in the help.

I doesn't found any option to set the suite I want to run. Laika always run all the suites.

After the Future usage, any errors throws in the main callback gets ignored and simply system get closed with a stdout. So now mocha cannot track errors.

Fix

If the assert module is in the Fiber's main callback simply use futures to execute the actual function

Create common tasks as actions and make it can be extended pretty easily (easy to make pull requests)

With the amount of workload in the app, client takes some times to get load. That makes testing slow.

So, pre creating a client when the app is being adding to the page, may give some speed boost.

Can i create function/Unit test with 'laika' ? I have packages/modules written that are used by my Meteor.Methods to fetch and process data. I would like to test an entire suite of them using some sort of a match ?

I am trying to test reactivity, so it is expected to emit values more than once. How to treat this? Are there any alternatives for the once method? Where is this once method defined?

Make it possible to use following

assert.throws(function() {
    server.evalSync(function() {

        throw new Error('fail');
        emit('return');

    });
});

app.close() takes a callback as an argument to tell us db has been dropped.

But when we are closing them (using appPool) at the end, we don't care about this callback and it db does not get dropped.

Perhaps this is just my own confusion, but I am having problems getting the server to be able to assert inside the first server.eval of a test.

If I apply the patch below to hello-laika, I get a "Uncaught ReferenceError: assert is not defined":

diff --git a/tests/posts.js b/tests/posts.js
index f84bd68..b9ae2e5 100644
--- a/tests/posts.js
+++ b/tests/posts.js
@@ -4,6 +4,7 @@ var assert = require('assert');
 suite('Posts', function() {
   test('in the server', function(done, server) {
     server.eval(function() {
+      assert.equal(1, 1);
       Posts.insert({title: 'hello title'});
       var docs = Posts.find().fetch();
       emit('docs', docs);

Hi,

Would be great to do some repetitive startup/shutdown logic (like creating a user and logging in) in the hooks before/after running the actual tests. It would make the test code a lot cleaner too.

Could we pass the server and clients to these functions as well? Is the environment already running at that point?

Thanks.

For the testing app needs some sample data. But when inserting them at the runtime, causes tests to be run slower and occurs a timeout. (Since inserting data takes some time)

We need a support for fixtures from laika, that will be loaded before the tests.

related #25

First, great work. I'm really looking forward to using Laika!

However, first thing that struck me as soon as I read the docs was the fugly naming conventions of pretty much the whole API. Things such as 'ltest' and 'emit'/'once'/'on' doesn't seem very intuitive. Couldn't 'ltest' just be 'laika'. The value of laika would depend on the test style (tdd/bdd/etc.)

As for things such as 'eval' and 'evalSync'; well, it's just downright confusing. Seeing as 'eval' is commonly used to name functions that evaluate expressions in string values, not running a function. Not just in JS but in a whole bucketload of languages. Surely this should be something such as 'run' / 'runSync' seeing as we are running a test.

Thoughts?

I'm just trying to run the test: 'using both client and the server' as per the demo

test('using both client and the server', function(done, server, client) {
    server.eval(function() {
        Posts.find().observe({
            added: addedNewPost
        });

        function addedNewPost(post) {
            emit('post', post);
        }
    }).once('post', function(post) {
        assert.equal(post.title, 'hello title');
        done();
    });
    client.eval(function() {
        Posts.insert({title: 'hello title'});
    });
});

However I always get a timeout.

 1) Posts using both client and the server:
     Error: timeout of 2000ms exceeded at null.<anonymous>
(/usr/local/share/npm/lib/node_modules/laika/node_modules/mocha/lib/runnable.js:165:14)
at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)

The 'in the server' test passes ok. Maybe its something to do with observe()? Increasing the timeout period does not help. I'm not sure how to get some output to help narrow things down a bit.

Any ideas?

I'd really like to use laika as it looks like it will simplify the testing workflow. Thanks.

Rather than creating db for each test, try to pool databases and clean it after the use.

This may cause, tests to run faster.

I'm getting the following output. I have a couple migrations that wind up running since it's a fresh copy of the app, but they shouldn't be doing anyhting. I think the optimist and ECONNREFUSED problems are more relevant, although I might be wrong.

$ laika -d -t 20000

  injecting laika...
  loading phantomjs...
[app touch log]
[app touch log] Stand back while Meteorite does its thing
[app touch log]
Done installing smart packages
[app touch log]
[app touch log] Ok, everything's ready. Here comes Meteor!

[app touch log] [[[[[ ~/meteor/apps/spendflow ]]]]]

  loading initial app pool...
[app touch log] => Meteor server running on: http://localhost:21454/
[server log] Oops, looks like the Segment.io API key is missing! Analytics won't work until you fix that.
[server log] laika code injected and listening on: 18695
[server log] Running migration: 20130630_iso_dates
[server log] ====== INCOMES ======
[server log] ====== EXPENSES ======
[server log] Running migration: 20130630_iso_dates_2
[server log] ====== INCOMES ======
[server log] ====== EXPENSES ======
[server log] Running migration: 20130630_payment_metadata
[server log] Incomes...
[server log] Expenses...
[server log] Migration complete.
[server log]
[server log] /Users/kevin/.meteor/tools/4010e5731d/lib/node_modules/fibers/future.js:173
[server log]                                            throw(ex);
                [server log]                                  ^
[server log] Error: ENOENT, no such file or directory
    at Argv (/Users/kevin/.meteor/tools/4010e5731d/lib/node_modules/optimist/index.js:22:29)
    at Object.main (packages/webapp/webapp_server.js:428)
    at /Users/kevin/meteor/apps/spendflow/.meteor/local/build/programs/server/boot.js:180:27
[server log] Oops, looks like the Segment.io API key is missing! Analytics won't work until you fix that.
1..1
[server log] laika code injected and listening on: 13393
[server log] Running migration: 20130630_iso_dates
====== INCOMES ======
====== EXPENSES ======
Running migration: 20130630_iso_dates_2
[server log] ====== INCOMES ======
====== EXPENSES ======
not ok 1 Basic setup accounting.js exists
  Error: connect ECONNREFUSED

And my test looks like

$ cat tests/spendflow.test.js

var assert = require('assert');
var _ = require('underscore');

suite('Basic setup', function () {
  test('accounting.js exists', function (done, server, client) {
    client.eval(function () {
      emit('done', accounting);
    }).once('done', function (accountingObject) {
        if (! accountingObject) {
          // Make it fail test without ReferenceError
          accountingObject = null;
        }
        assert.equal(_.isObject(accountingObject), true);
        done();
      });
  });
});

Currently on laika 0.2.2 and nodejs 0.10.15

When one of my assertion fails, the rest of the tests fail as well with the same errror.

assert.js:92
  throw new assert.AssertionError({
        ^
AssertionError: false == true
    at ClientConnector.<anonymous> (/Users/username/meteor/website/tests/eventTest.js:42:11)
    at ClientConnector.EventEmitter.emit (events.js:95:17)
    at onCallback (/usr/local/share/npm/lib/node_modules/laika/lib/connectors/client.js:57:15)
    at Socket.<anonymous> (/usr/local/share/npm/lib/node_modules/laika/node_modules/node-phantom/node-phantom.js:199:7)
    at Socket.EventEmitter.emit [as $emit] (events.js:95:17)
    at SocketNamespace.handlePacket (/usr/local/share/npm/lib/node_modules/laika/node_modules/node-phantom/node_modules/socket.io/lib/namespace.js:335:22)
    at Manager.onClientMessage (/usr/local/share/npm/lib/node_modules/laika/node_modules/node-phantom/node_modules/socket.io/lib/manager.js:488:38)
    at WebSocket.Transport.onMessage (/usr/local/share/npm/lib/node_modules/laika/node_modules/node-phantom/node_modules/socket.io/lib/transport.js:387:20)
    at Parser.<anonymous> (/usr/local/share/npm/lib/node_modules/laika/node_modules/node-phantom/node_modules/socket.io/lib/transports/websocket/default.js:36:10)
    at Parser.EventEmitter.emit (events.js:95:17)`

My assertion looks like this:

        client.on("ready", function(ret){
            assert.ok(!ret.error);
            done();
        });

Looks like others have experienced this issue (http://stackoverflow.com/questions/17056068/meteorjs-laika-client-and-server-test-timout), namely:

Failure on test 2 of example 2 ("using both client and the server")

Error is "timeout of 2000ms exceeded"

I've tried suggested fixes in the SO question, including upgrading my Node.js version to no avail. Does anyone have a workaround? Otherwise, seems like a great test suite, please keep up the great work!

I observed that when running Laika, a lot of mongo databases are created and not deleted.
I have to run the following code at shell every some while:

$ mongo
> var dbs = db.getMongo().getDBNames()
for(var i in dbs){
    db = db.getMongo().getDB( dbs[i] );
    if(db.getName() == "test" || db.getName() == "admin" || db.getName() == "local")
        continue;

    print( "dropping db " + db.getName() );
    db.dropDatabase();
}

Note: I am not using the options that makes testing much speeder recommended on docs.

For debugging purposes, would be interesting to have a way to see what the Meteor is printing on the console during the execution of Laika.

A --debug option to use with node-inspector would be even better.

In your article at sitepoint, you state the following:

"you can send some results back to test with emit() and catch it with server.once or server.on"

What's the difference? Thanks.

http://www.sitepoint.com/test-driven-development-with-meteor/

As seen on extendind Laika example, we can extend Laika to use Collections in the same way we would use Meteor API, but extendind we have to declare Collection inside of each test.

Can we make Collection natively available inside test?
Can we also support fixtures?

When I run the laika command on my app, it hangs on loading phantomjs info. I get no message error, nor any reason why things don't work.

It's not a problem with phantom - when trying in the sample app (https://github.com/arunoda/hello-laika), it goes past this steps (and hangs later).

I might be missing something here but does laika not work with node v0.8.16. I tried running and i get this odd error.

` injecting laika...
loading phantomjs...
loading initial app pool...

buffer.js:434
throw new Error('Unknown encoding');
^
Error: Unknown encoding
at Buffer.toString (buffer.js:434:13)
at Object.fs.readFileSync (fs.js:236:33)
at Object.getMeteoriteNode (/usr/local/share/npm/lib/node_modules/laika/lib/helpers.js:42:33)
at detectConfig (/usr/local/share/npm/lib/node_modules/laika/lib/app_pool.js:83:35)
at new AppPool (/usr/local/share/npm/lib/node_modules/laika/lib/app_pool.js:16:19)
at module.exports.run.mochaOptions.reporter (/usr/local/share/npm/lib/node_modules/laika/bin/_laika:87:17)
at start.queue (/usr/local/share/npm/lib/node_modules/laika/node_modules/qbox/lib/qbox.js:38:5)
at Array.forEach (native)
at QBox.start (/usr/local/share/npm/lib/node_modules/laika/node_modules/qbox/lib/qbox.js:37:10)
at QBox.tick (/usr/local/share/npm/lib/node_modules/laika/node_modules/qbox/lib/qbox.js:58:10)`

add to package.json

Hi, I have a problem with Laika: node version 0.10.15, I have update phantomjs.
$> laika -d -t 10000
Doen´t matter timeout that I pass to Laika, always fail.
Error: timeout of 10000ms exceeded
at null. (/opt/local/lib/node_modules/laika/node_modules/mocha/lib/runnable.js:165:14)
at Timer.listOnTimeout as ontimeout

Try to use node-phantom-simple[0]. Which has some good communication pattern with phantom binary.

[0] - https://github.com/baudehlo/node-phantom-simple

I'm trying to figure out how to remove Laika, everytime my meteor server restarts it says "laika injected and listening on: someport".

I think it's causing some issues, but I can't be sure until I can get it to not run.

Currently fibers re-installed if needed, laika asks us to re-run laika.
That cause tests to fail in CI env like travis

So make it re-run tests automatically

This works in 0.2.8, but not 0.2.9:

$ TEST=1 laika tests/revisions.js

  injecting laika...
  loading phantomjs...
  loading initial app pool...
  cleaning up injected code


fs.js:654
  return binding.readdir(pathModule._makeLong(path));
                 ^
Error: ENOTDIR, not a directory 'tests/revisions.js'
    at Object.fs.readdirSync (fs.js:654:18)
    at scanFiles (/Users/xx/xx/xx/xx/node-0.10.12/lib/node_modules/laika/bin/_laika:112:22)
    at AppPool.onAppPoolReady (/Users/xx/xx/xx/xx/node-0.10.12/lib/node_modules/laika/bin/_laika:101:7)
    at AppPool.EventEmitter.emit (events.js:92:17)
    at App.createInitialPool (/Users/xx/xx/xx/xx/node-0.10.12/lib/node_modules/laika/lib/app_pool.js:30:12)
    at App.EventEmitter.emit (events.js:95:17)
    at Socket.<anonymous> (/Users/xx/xx/xx/xx/node-0.10.12/lib/node_modules/laika/lib/app.js:50:14)
    at Socket.EventEmitter.emit (events.js:117:20)
    at Socket.<anonymous> (_stream_readable.js:736:14)
    at Socket.EventEmitter.emit (events.js:92:17)
$ 

Meteor Uses localStorage to store logged-in user info.
PhantomJS pages share localStorage values. So It is not possible to have clients with diff users logged right now.

Possible Solutions

Try loading pages with different hostnames (EASY)

• possible to use 3 clients with commonly available hostnames (localhost, 127.0.0.1, 0.0.0.0)

Proxy Meteor App to load from different port

Option 1

• create websocket proxy in different ports (EASY)

Option 2

• only proxy static pages
• modify html to set DDP_DEFAULT_CONNECTION_URL with the original address (for SockJS and WS)

Currently even though a Server is requested a server is fetching from the pool and get closed without using it.
When doing unit tests this makes it is very inefficient.

Eg:-

test('on and emit', function(done) {
  var ev = new EV();
  var cnt = 0;
  ev.on('evt', function(v1, v2) {
    cnt ++;
    assert.equal(v1, 10);
    assert.equal(v2, 20);
  }); 
  ev.emit('evt', 10, 20);
  ev.emit('evt', 10, 20);

  assert.equal(cnt, 2);
  done();
});

Error's occured in the server code tracked in a nasty way inside lib/app.js. Need to move it to ServerConnector

There is no reporting for client side errors. PhantomJS onError can be used for this.

I can see that phantom initialization within laika is made with a single parameter " {'load-images': false}". In case you have a webapp that uses local storage we can't add phantomjs standard parameter i.e.: "--local-storage-path=/some/path".
Does anyone has a workaround for this ?

Allow laika to accept METEOR_PATH env variable with custom installed meteor version so, it could find the correct node binary

Need more tests to cover laika specific features.
Try to write tests in laika itself.

executing laika in a hidden directory (for example '.laika') will make it hang on loading initial app pool....
It looks like the meteor server is not starting correctly.

Recreate using:

git clone https://github.com/arunoda/hello-laika.git .hello
cd .hello
laika

We can do some phantomjs based optimizations.

• ask phantomjs to not to load images[0]
• ask phantomjs to cache[0]
• ask phantomjs to not to load css[1]

[0] - https://github.com/ariya/phantomjs/wiki/API-Reference#command-line-options
[1] - http://stackoverflow.com/questions/9486377/how-can-i-control-phantomjs-to-skip-download-some-kind-of-resource