massive-oss / massiveunit

munit is a cross-platform unit testing framework for Haxe. Utilises metadata markup for test cases and includes tools for generating, compiling and running test cases from the command line.

MUnit is a metadata driven unit testing framework for cross-platform Haxe development.

It includes tools for creating, updating, compiling and running unit test cases from the command line.

MUnit supports Haxe 3+

Installation

To install you must have Haxe installed

Then just use haxelib to download the latest version

haxelib install munit

To check that it is all installed and to view the help run:

haxelib run munit

Please note: After upgrading you may be required to update the TestMain.hx in existing project before being able to test

To install latest build from git:

haxelib git munit https://github.com/massiveinteractive/MassiveUnit.git master src

Features

Cross Platform

MUnit has been designed for cross platform Haxe development. It currently supports js, swf, as3, neko, c++, java, cs, python, php and hl and the tool chain works on Windows, OSX and Linux. To run js tests with NodeJS, add -lib hxnodejs or -D nodejs.

Test Metadata

Test cases use Haxe metadata to simplify creating tests (and avoid needing to extend or implement framework classes).

@Test
public function testExample():Void
{
	Assert.isTrue(true);
}

Asynchronous Tests

Unlike the default haxe unit test classes, MUnit supports asynchronous testing

@AsyncTest
public function asyncTestExample(factory:AsyncFactory):Void
{
	...
}

Tool Chain

MUnit is way more than just a unit test framework. It includes a command line tool for working with munit projects to streamline your development workflow.

• Setup stub test projects in seconds
• Auto generate test suites based on test classes in a src directory
• Compile and run multiple targets from an hxml build file
• Launch and run test applications in the browser or command line (neko)
• Save out text and junit style test reports to the file system for reporting and ci
• Auto generate stub test classes (and/or target classes)
• Integrated code coverage compilation with MCover

Documentation

For detailed user guides refer to the wiki

The API documentation is available on the haxelib project page.

How to contribute

If you find a bug, report it.

If you want to help, fork it.

To install latest build from git:

haxelib git munit https://github.com/massiveinteractive/MassiveUnit.git src

If you want to make sure it works, make sure to run the bash script (build.sh) and check that the tests all pass on all platforms:

haxelib run munit test -coverage

cd ../test
haxelib run munit test -coverage

New since 2.0.0

Haxe 3 Support

Some APIs have changed to ensure compatibility with both Haxe 2.10 and Haxe3.

Note: Support for Haxe 2.09 and Haxe 2.08 have been dropped

New since 0.9.5.x

C++ Target

MUnit now compiles/runs c++ targets

haxelib run munit test -cpp

Updates to TestMain to work with MCover 1.4.x

Note: You may be required to update the TestMain.hx in existing project
before being able to run `munit test -coverage`.

Code Coverage customisation

Munit now supports custom mcover settings when compiling via munit test -coverage.

re-run haxelib run munit config in a project to set coverage packages and coverage ignored classes

Thanks to tynril for adding these options.

New since 2.0.x

Haxe 3 Support

Munit now supports Haxe 3 RC.

There aren't any new features in this release, however there are several breaking changes to internal APIS and Interfaces to ensure compatibility with both versions of Haxe.

To compile tests aginst Haxe 3, you may need to delete any references to js.Dom in the generated TestMain class in your project.

New since 0.9.4.x

Report Command

Convert munit summary reports into different format(s). As of 0.9.4.0 there is only one supported format (TeamCity)

haxelib run munit report [format] [dest] [-coverage percent]

Example:

The following example generates a teamcity-info.xml report in the project directory (.)

haxelib run munit report teamcity . -coverage 85

Note: You must run and generated summary reports prior to executing the report command.

New since 0.9.3.x

Better CI support

Get error exit code when tests on one or more platforms fail

haxelib run munit test -result-exit-code

Note: haxelib currently doesnt return exit codes > 0 on OSX (see issue)

Workaround for issues with nekotools server HTTP POST via a simple SummaryReportClient

var httpClient = new HTTPClient(new SummaryReportClient())
runner.addResultClient(httpClient);

New since 0.9.2.x

Rich HTML output for JavaScript and Flash targets (see RichPrintClient)

var client = new RichPrintClient();

Seamless support for MCover code coverage

haxelib run munit test -coverage

Commands for generating stub test classes on demand

haxelib run munit create package.FooTest -for package.Foo  

CI friendly options for munit config command

haxelib run munit config -default
haxelib run munit config -src path/to/src -hxml path/to/test.hxml
haxelib run munit config -file path/to/my/custom/config.txt

Support for assertions inside async tests

public function someAsyncTest(factory:AsyncFactory)
{
	Assert.isTrue(false);
}

Support for custom runner html templates and resources

Run 'munit config' to set template and resources directories

For full list of recent changes see the change log

Credits

This project is brought to you by Dominic and Mike from Massive Interactive

massiveunit's Issues

Enable writing of config properties file through single command

Currently it's only possible to set up the munit config file by running through a series of prompts. Open this up so it can be generated through a single command. e.g.

 munit config -src src/main,src/js -test src/test -output build -report report

Node.js client

I would be greatly interested in a Node.js client. It should be fairly straightforward, since it's a command line program.

I've looked at the code to do this myself, but it's not clear where I would specify the new client. Could you give me some pointers?

Dion

Exit code should be 1 when tests fail

At the moment it seems that the haxelib munit command exits with a value of 0 (command was successful) regardless of whether the tests pass or not. It might be a good idea using an exit code of 1 (catch-all for all errors) when the tests fail.

The reason I'm asking is because I'm integrating Munit in my automated release flow and I obviously want to abort the release if tests don't pass.

There's this comment on the wiki:

"Save out junit style test reports to the file system for reporting and ci"

So there might be a simple / better way of achieving this?

Cheers,

Juan

Support headless browsers

It would be good to support headless browsers like Phantomjs etc.

I might end up looking into the myself, but if you've already got information about how to achieve this, then that would be a good start.

Test runner not respecting targets in edited test.hxml file

Description:
If a target is removed from test.hxml after it has been run, and then is run again, the target will still be picked up.

Steps to reproduce:

1. Create a test.hxml with targets js,as2,as3,neko
2. Run

munit test

3. Remove one of the targets from test.hxml
4. Run

munit test

Actual Results:

The removed target is still run.

Expected Results:

Only the targets described in test.hxml should be run.

Workaround (if any):

If you explicitly state the targets you want to test then this is not an issue. e.g.

munit test -as3 -as2

Additional Notes (if any):
Looking at the code it appears than when no target is specified all targets are added and then not normalized when targets are read again from the hxml file. It could be the runner is just running the old test file from the last run.

support for NME?

I can't figure out how to run this to test a haxenme project? (which uses hxcpp)

on an unrelated note:
for some reason I am testing for AS2, AS3 and javascript, even though in my test.hxml, I have this:

Flash 9+

-main TestMain
-lib munit
-lib hamcrest
-cp src

-cp test
-swf-version 9
-swf build/as3_test.swf
--next

JavaScript

-main TestMain
-lib munit
-lib hamcrest
-cp src

-cp test
-js build/js_test.js

--next

Neko

-main TestMain
-lib munit
-lib hamcrest
-cp src

-cp test
-neko build/neko_test.n

Tests Cause MUnit/Neko Server to Fail

My question is about some tests in my small project here: https://www.dropbox.com/s/v71ylqkgi8bqifa/LegendOfHamza.zip

It's a simple project. It has game code (com.deengames.legendofhamza) and some framework code (com.blastcube). The framework code is just a game class, screen class, and some helper methods for adding text fields and bitmaps.

I also have some MUnit tests in the test directory; inside ScreenTest.hx, I have commented out the bottom two tests. Both of these tests, if uncommented, cause the tests to run and finish, but when the browser launches to view the results, it says that it cannot connect to the server.

I'm not sure why this would be the case. Is this a bug?

Note that in either case (with or without commenting these tests out), running the tests eventually gives me the message that "ERROR: Local results server appeared to hang so test reporting was cancelled." When I have the tests commented out, the browser does launch properly with the results though.

It should be possible to run the unit test without a client.

Right now the TestRunner requires a client to run the code. However: I just wanted to start the tests for debugging in IntelliJ. Unfortunately intelliJ starts all generated swfs in the FlashPlayer and just throws a ExternalInterface-Not-Available exception.If I set the client to "null" it throws an Error while trying to add null. I added a if(client!=null) check to the TestRunner. Seemed to do the trick (but isn't quite elegant)

Error in Firebug console

Hi !

I have the below error flooding my Firebug console on http://localhost:2000/tmp/runner/index.html

TypeError: massive.munit.util is undefined
[Break On This Error]   

...rId = window.setInterval("massive.munit.util.Timer.arr[" + this.id + "].run();",...

It seems to happen at line 4737 in generated js:

massive.munit.util.Timer = function(time_ms) {
  this.id = massive.munit.util.Timer.arr.length;
  massive.munit.util.Timer.arr[this.id] = this;
  this.timerId = window.setInterval("massive.munit.util.Timer.arr[" + this.id + "].run();",time_ms); // line 4737
}; 

Is that a known issue ?

I'm testing munit for the first time and appart from that and a server hang happening when not deleting the default ExampleTest.hx template, it seems to work fine.

Cheers !

Thomas

Headless testing for js and flash targets?

Is there a way to run the tests for javascript and flash targets without having my browser pop up every time?

Especially with flash this is really annyoing, since I am using Linux and deny to use chrome, I have no chance to run flash in the browser at all.

Another point is, I want to use travis.ci for my haxe-projects, but in order to do that, I have to rely on headless testing.

For headless javascript you could probably use phantom.js.

AbstractTest - Testing interfaces and abstract superclasses

Description

It is possible to write tests for an interface, which must be passed by all implementations of the interface. However, munit currently tries to add these abstract tests by themselves, when they should only be run by subclasses of the tests, which work with concrete implementations.

Code to replicate issue

interface IAbstract {
  function returnTrue():Bool;
}
class Concrete implements IAbstract {
  public function returnTrue():Bool {
    return false;
  }
}

class IAbstractTest {
  public var instance:IAbstract;
  @Test
  public function testReturnTrue():Void {
    Assert.isTrue(instance.returnTrue());
  }
}

Current Workarounds

• Add a separate source folder that is not managed by munit, so it will not add the tests, but they can still be seen and subclassed
• Handle TestSuite manually, because munit will automatically add all classes in the test src directory.

Proposed Solution

Add a bit of metadata that allows a class to be overlooked when checking for test classes. I suggest @AbstractTest.

Example

@AbstractTest
class IAbstractTest
{
  public var instance:IAbstract;
  @Test
  public function testReturnTrue():Void {
    Assert.isTrue(instance.returnTrue());
  }
}

Mock/Stub support?

Sometimes I want to use 'fake' object while testing. For example, I wrote a logger which output strings to stdout, but I don't want these strings appear when I run unit tests. It will be very helpful to be able to replace sys.print sys.println or trace with my own fake implementation.

Another common case is to test a method that invoke third party API, especially database or network. It may slow down the test, so I want to use a fake API returning fixed value.

Does munit (have a plan to) support it?

0.9.4.1

Minor big fixes and resolved issues

Run separate test method

This feature exists in FlexUnit framewourk and it's wery helpfull in debug.
It's sorely lacking feature in this framework

Munit example app hangs on RUN after swf8 target when running multiple targets

This issue only occurs for swf8 test runner when run alongside other targets.

THE Command line hangs prior to results being captured.

Running the swf8 target only (using the -swf arg) works as expected.

Removing the swf8 test from the bin directory prior to running runs all other targets correctly.

Method for including additional scripts in runner?

I'm building some Haxe code which sometimes requires JQuery as an external library -- what is the recommended way to include Jquery have the js tests load Jquery when generating the html? Is one expected just to update the templates globally - and also copy the required external files into place?

Awesome work - thanks!

junit test reports missing after upgrading to 0.9.2.1

Executing
haxelib run munit test
does not seem to produce junit outputs in report directory anymore. report/test only contains a results.txt file with the content
END

After downgrading to 0.9.1.2 (via haxelib set) output works again. Using haxe 2.08 on Windows 7.

Adobe AIR support

Hi guys,

Compiling for Flash assumes the app is going to be launched within a browser, which is not the case for AIR apps.

I guess that to support AIR you would need:

• allow defining where the AIR / FlexSDK is (to be able to launch ADL)
• provide a default air-descriptor.xml
• allow custom descriptor (in case ANE files are required, for example).

Makes sense?

Massive test runner doesn't seem to report test-results to munit application

Whenever I my as3 tests in a browser, the browser tells me the correct test results:

PASSED
Tests: 36  Passed: 36  Failed: 0 Errors: 0 Ignored: 0 Time: 0.05

But the command-line output at the end shows a different result:

PLATFORMS TESTED: 0, PASSED: 0, FAILED: 0, ERRORS: 0, TIME: 1.06555

This prevents me from using the --kill-browser option.

Support for running JS tests in multiple browsers

Consider this a feature request:

It would be great to be able to launch multiple browsers for testing JS, to see if the code is cross-browser compatible.

There is already the -browser name attribute, perhaps we can have -browser firefox:chrome:opera etc? Also an ability to specify this in the .munit config file would be helpful. I would be happy to help implement this if there's interest in supporting this.

Regression: trace statements no longer appear in print client output

Trace statements used to appear in the print client output and the rich print client after each test class

TRACE: CalculatorTest.hx|46| something

@After code causes test runner to freeze

My question is about my small project here: https://www.dropbox.com/s/v71ylqkgi8bqifa/LegendOfHamza.zip

It's a simple project. It has game code (com.deengames.legendofhamza) and some framework code (com.blastcube). The framework code is just a game class, screen class, and some helper methods for adding text fields and bitmaps.

I also have some MUnit tests in the test directory; notably, in BaseScreenTest.hx, I have a tearDown function with an @after annotation. I noticed that uncommenting this annotation causes the "haxelib run munit test" test runner to freeze in the middle of some tests, and eventually give me a message that the test runner hanged.

I'm not sure why this is. Is this a bug? This also happened when I didn't have this as a base class, but instead had this method inside my ScreenTest.hx class.

--no-inline

It would be useful if you could run haxelib run munit test -as3 --no-inline, and let the --no-inline parameter be passed through to the resultant haxe command. (Perhaps other parameters could also be passed through this way - or even a -params '--no-inline -etc -D eg' parameter?)

Use name of swf from hxml file

Currently the swf file names are tied to as3_test.swf or as2_test.swf. This is mandatory and is not consistent with other targets i.e. JS or neko can be called something else entirely. A fix for this would be to test for as3_ or as2_ if none are found then find and use any swf similarly to other targets or even better, parse the test.hxml for the actual name of the swf to run.

Allow sync asserts in async tests.

I would really like to use sync asserts in async tests please:

@Test("Async")
public function testAsyncExample(factory:AsyncFactory):Void
{
    Assert.isFalse(false); //throws: massive.munit.AssertionException
    var handler:Dynamic = factory.createHandler(this, function() {
        Assert.isFalse(false);
    }, 300);
    timer = Timer.delay(handler, 200);
}

MUnit Results

Class: ExampleTest .
massive.munit.AssertionException: Assertion(s) were made before async test returned at ExampleTest#testAsyncExample

CPP Crash

Hey

If I use haxe.resource for in a before method for enough tests it eventually breaks. Could someone please help me out?

I've created a project demonstrating the problem.

https://github.com/thomasuster/resourceStress
munit test -cpp

Add number of assertions to test ratio to reports

Add the ability to see how many assertions vs tests (ratio) there are. It helps spot if there are multiple assertions if trying to keep to "...unit tests should fail for exactly one reason, that’s why you should be using one assert per unit test.".

It could be simply to add the number of assertions to the end of result.

PASSED
Tests: 1785 Passed: 1785 Failed: 0 Errors: 0 Ignored: 0 Time: 3.321 Assertions: 1785

Errors when attempting to perform unit testing on empty project

From what I have read it seems that munit should generate an example project when the following commands are entered:

$ mkdir ExampleProject
$ cd ExampleProject
$ haxelib run munit config
$ haxelib run munit test

However, the following error message appears:

Massive Unit - Copyright 2013 Massive Interactive. Version 2.0.2
   haxe -main TestMain -lib munit -lib hamcrest -cp src -cp test -swf-version 11 -swf build/as3_test.swf
   haxe -main TestMain -lib munit -lib hamcrest -cp src -cp test -swf-version 8 -swf build/as2_test.swf
   haxe -main TestMain -lib munit -lib hamcrest -cp src -cp test -js build/js_test.js
   haxe -main TestMain -lib munit -lib hamcrest -cp src -cp test -neko build/neko_test.n
   haxe -main TestMain -lib munit -lib hamcrest -cp src -cp test -cpp build/cpp_test
      haxelib run hxcpp Build.xml haxe -Dhamcrest="1" -Dhaxe3="1" -Dhaxe_release="1" -Dhaxe_ver="3." -Dmconsole="1" -Dmcover="1" -Dmlib="1" -Dmunit="1"
      Creating obj/darwin//__pch/haxe/hxcpp.h.gch...
      g++ -Iinclude -c -fvisibility=hidden -O2 -I/usr/lib/haxe/lib/hxcpp/3,0,2//include -DHX_MACOS -m32 -DHXCPP_VISIT_ALLOCS -frtti -o obj/darwin//__pch/haxe/hxcpp.h.gch /usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h
   /usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:14:20: error: typeinfo: No such file or directory
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:15:20: error: stdint.h: No such file or directory
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:29:20: error: string.h: No such file or directory
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:33:19: error: wchar.h: No such file or directory
In file included from /usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:193:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:205:21: error: algorithm: No such file or directory
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:19: error: ‘int64_t’ does not name a type
In file included from /usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:185:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Object.h:59: error: declaration of ‘operator new’ as non-function
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Object.h:59: error: expected ‘;’ before ‘(’ token
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Object.h: In member function ‘void hx::ObjectPtr<O>::CastPtr(hx::Object*)’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Object.h:152: error: must #include <typeinfo> before using typeid
In file included from /usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:186:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hxString.h: At global scope:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hxString.h:20: error: declaration of ‘operator new’ as non-function
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hxString.h:20: error: expected ‘;’ before ‘(’ token
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hxString.h: In member function ‘int String::compare(const String&) const’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hxString.h:106: error: ‘strcmp’ was not declared in this scope
In file included from /usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:192:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Anon.h: In static member function ‘static hx::ObjectPtr<hx::Anon_obj> hx::Anon_obj::Create()’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Anon.h:43: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Anon.h: In static member function ‘static hx::ObjectPtr<hx::Anon_obj> hx::Anon_obj::Create(const Dynamic&)’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Anon.h:44: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Anon.h: In static member function ‘static Dynamic hx::Anon_obj::__CreateEmpty()’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Anon.h:46: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
In file included from /usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:193:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: At global scope:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:85: error: ‘size_t’ does not name a type
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In member function ‘void Array_obj<ELEM_>::sort(Dynamic)’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:401: error: ‘stable_sort’ is not a member of ‘std’
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In member function ‘Dynamic Array_obj<ELEM_>::map(Dynamic)’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:619: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
In file included from /usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:194:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Class.h: In function ‘bool hx::TCanCast(hx::Object*)’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Class.h:154: error: must #include <typeinfo> before using typeid
In file included from /usr/lib/haxe/lib/hxcpp/3,0,2//include/hxcpp.h:201:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In static member function ‘static Array<T> Array_obj<ELEM_>::__new(int, int) [with ELEM_ = Dynamic]’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:504:   instantiated from ‘Array<ELEM_>::Array(const Dynamic&) [with ELEM_ = Dynamic]’
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Operators.h:251:   instantiated from here
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:556: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In member function ‘Array<T> Array_obj<ELEM_>::concat(Array<T>) [with ELEM_ = Dynamic]’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:412:   instantiated from ‘Dynamic Array_obj<ELEM_>::__concat(const Dynamic&) [with ELEM_ = Dynamic]’
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Boot.h:12:   instantiated from here
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:567: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In member function ‘Array<T> Array_obj<ELEM_>::copy() [with ELEM_ = Dynamic]’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:413:   instantiated from ‘Dynamic Array_obj<ELEM_>::__copy() [with ELEM_ = Dynamic]’
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Boot.h:12:   instantiated from here
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:575: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:576: error: ‘memcpy’ was not declared in this scope
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In member function ‘Dynamic Array_obj<ELEM_>::iterator() [with ELEM_ = Dynamic]’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:415:   instantiated from ‘Dynamic Array_obj<ELEM_>::__iterator() [with ELEM_ = Dynamic]’
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Boot.h:12:   instantiated from here
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:404: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In member function ‘Array<T> Array_obj<ELEM_>::slice(int, Dynamic) [with ELEM_ = Dynamic]’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:422:   instantiated from ‘Dynamic Array_obj<ELEM_>::__slice(const Dynamic&, const Dynamic&) [with ELEM_ = Dynamic]’
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Boot.h:12:   instantiated from here
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:586: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In member function ‘Array<T> Array_obj<ELEM_>::splice(int, int) [with ELEM_ = Dynamic]’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:423:   instantiated from ‘Dynamic Array_obj<ELEM_>::__splice(const Dynamic&, const Dynamic&) [with ELEM_ = Dynamic]’
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Boot.h:12:   instantiated from here
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:594: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In member function ‘Dynamic Array_obj<ELEM_>::map(Dynamic) [with ELEM_ = Dynamic]’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:427:   instantiated from ‘Dynamic Array_obj<ELEM_>::__map(const Dynamic&) [with ELEM_ = Dynamic]’
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Boot.h:12:   instantiated from here
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:619: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h: In member function ‘Array<T> Array_obj<ELEM_>::filter(Dynamic) [with ELEM_ = Dynamic]’:
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:428:   instantiated from ‘Dynamic Array_obj<ELEM_>::__filter(const Dynamic&) [with ELEM_ = Dynamic]’
/usr/lib/haxe/lib/hxcpp/3,0,2//include/hx/Boot.h:12:   instantiated from here
/usr/lib/haxe/lib/hxcpp/3,0,2//include/Array.h:602: warning: no corresponding deallocation function for `void* operator new(long unsigned int)'
Called from ? line 1
Called from BuildTool.hx line 1529
Called from BuildTool.hx line 673
Called from a C function
Called from BuildTool.hx line 708
Called from BuildTool.hx line 842
Called from BuildTool.hx line 885
Called from BuildTool.hx line 194
Uncaught exception - Error creating pch: 1 - build cancelled
Error: Build failed
Error: Error compiling hxml for cpp
Target cpp /Users/macuser/Haxe/ExampleProject/build/cpp_test/

I can confirm that openfl is properly building projects for flash and mac targets. It just seems to be munit which is not working for me.

Is there something that I am perhaps missing?

Thanks

no tag for version 2.0.0

If possible, could you tag all official releases to lib.haxe.org?

StackTrace on Error

No one chance to get stack trace with line numbers

TestRunner.hx

if (!Std.is(e, MUnitException))
{
    // Tried using stack trace (both flash9 and haxe.Stack.exceptionStack()
    // to get the line number of where the exception originated but neither
    // provide this information. Just going with providing the originating 
    // class + function.
    // mike 8/12/10
    e = new UnhandledException(e.toString() + " at " + result.location, null);
}

result.error = e;

How to get original e or e.stackTrace() from ITestResultClient realization (i.e. without modification your framework)?
It's matter, beacouse "className#methodName" is not enouth to error localization.

Wish to be:

+ var originalError = e;
if (!Std.is(e, MUnitException))
{
    // Tried using stack trace (both flash9 and haxe.Stack.exceptionStack()
    // to get the line number of where the exception originated but neither
    // provide this information. Just going with providing the originating 
    // class + function.
    // mike 8/12/10
    e = new UnhandledException(e.toString() + " at " + result.location, null);
}
result.error = e;
+ result.originalError = originalError;

API documentation is broken

The README states:

The API documentation is available on the haxelib project page.

However, the project page states:

Project munit version 2.0.0 has no documentation

A google search for "munit api documentation" doesn't turn up anything.

missing massive.mcover.munit.client.MCoverSummaryReportClient

when I try to run my tests with -coverage I get:

test/TestMain.hx:28: characters 44-103 : Class not found : massive.mcover.munit.client.MCoverSummaryReportClient

I cannot find that class in the mcover repo.

Openfl and munit

I've transitioned my project from NME to Openfl which runs beautifully, but when trying to run munit I receive the error...

"You cannot access the flash package while targeting cpp".

How do I get around this? Is this an hxcpp problem or munit?

Throwing exception in AsyncTest ends test but doesn't cancel aysnc timeout

Description

Throwing an exception inside an async test catches the error, but doesn't remove the AsyncDelegate timeout. This causes the test runner to report the async error after the test has finished (often once other test classes have started executing).

Code to replicate issue

@AsyncTest
public function shouldCancelAsyncTimeoutInTestRunner(factory:AsyncFactory)
{   
    var handler:Dynamic = factory.createHandler(this, handler, 50);
    throw "exception";
}

function handler()
{
    Assert.isTrue(true);
}

Proposed Solution

To prevent this, TestRunner.executeTestCase should check inside catch statement for asyncPending and then be able to cancel a AsyncDelegate.

massive.munit.TestRunner.executeTestCase

try
{
    ...
}
catch(e:Dynamic)
{
    if(async && asyncPending)
    {
        //cancel delegate
    }

    ...
}

Currently there is no way to access the generated delegate outside of the test being executed. AsyncFactory.createHandler should either hash generated AsyncDelegates using the asyncDelegateCount or at least store the most recent delegate in an internal property so that they can be retrieved for canceling.

massive.munit.async.AsyncFactory

//add functionality to retrieve the current AsyncDelegate instance

The AsyncDelegate will also require a public method for canceling the internal timer:

massive.munit.async.AsyncDelegate

public function cancel():Void
{
    timer.stop();
}

include another haxelib

I use another library for haxe called polygonal (http://code.google.com/p/polygonal/)

one of my class uses it and i want to unit test that class, unfortunately munit keeps complaining

I think I can add to classPaths the path to haxelib classes (which polygonal resides) but I don't know how? Is it comma separated? what if there is a white space in between? in quotes?

Local results server appeared to hang

I ran into the "nekotools server" issue while running the openfl unit tests.
You already described this bug here : https://github.com/massiveinteractive/MassiveUnit/wiki/Known-Issues#http-post-causes-nekotools-server-to-hang

We finally succeeded after removing the call to sys.io.Process('nekotools') and launching nekotools-server in a parent level bash script

#!/bin/bash -x
pwd=$(pwd)
cd /cygdrive/c/projects/munit/src
nekotools server &
SERVER_PID=$!
cd $pwd

haxelib run munit test

kill -15 $SERVER_PID    

Don't ask me why it works in this case ! Calling the server from io.Process may imply a different context of execution ... :/

Run under debug mode

About last changes:

#if debug
isDebug = true;
#else
isDebug = false;
#end

IMHO Is't not future - it's a bug. Why you forbided all test running in debug mode?

But this bug is not fatal:
public function new(resultClient:ITestResultClient)
{
super(resultClient);
isDebug = false;
}

Fail to run tests as non-priveleged user on Linux

Hi,

I was getting this error.

> haxelib run munit run
Massive Unit - Copyright 2012 Massive Interactive. Version 0.9.2.2
Called from ? line 1
Called from massive/munit/MunitCommandLineRunner.hx line 43
Called from massive/munit/MunitCommandLineRunner.hx line 41
Called from massive/munit/MunitCommandLineRunner.hx line 60
Called from massive/neko/cmd/CommandLineRunner.hx line 109
Called from massive/neko/cmd/CommandLineRunner.hx line 258
Called from massive/munit/command/RunCommand.hx line 311
Called from massive/neko/io/File.hx line 745
Called from /usr/lib/haxe/std/neko/io/File.hx line 63
Called from /usr/lib/haxe/std/neko/io/File.hx line 54
Uncaught exception - [file_open,/usr/lib/haxe/lib/munit/0,9,2,2/tmp/runner/tests-complete.js]

Basically a permissions error - it's trying to write to a priveleged location. If I create /usr/lib/haxe/lib/munit/0,9,2,2/tmp and give it "777" permissions, it works.

I'm assuming this is using mlib's create temp directory / create temp file helpers? If so, could these methods point to a folder in the users directory by default?

p.s. Not sure if you guys test linux as a platform. I'm just getting started but it mostly seems to be okay other than minor things like this. Thanks heaps :)

Can't install

$ haxelib install munit
Downloading munit-2,0,2.zip...
Download complete : 1312 bytes in 0.3s (3.3KB/s)
Invalid Zip Data

Icons dont appear in result client for AS3

This is due to the munit 'run' command shutting down the temporary nekotools server before the images are loaded by the browser.

This is only an issue for AS3 because the swf runner relies on ExternalInterface to update the javascript result client in the browser, and synchronous calls via ExternalInteface are inconsistently slow. The more data being printed to the browser client (lots of failures, exceptions and coverage info), the bigger this lag can be.

For some reason this occasionally results in a race conditions issue and by the time the browser DOM updates and goes to load the icons the server has already shut down and the urls dont exist anymore.

Potential workaround on current version branch: moving all the icons into a single spritesheet that is loaded up front so they are already cached by the browser.

Excluding classes from code coverage.

Hi!

I was looking all through the project and can't seem to figure out if it's possible to exclude classes from coverage report when using munit test -coverage. I have a project almost completely covered by tests and inability to exclude a few utility classes that I don't want to test and thus bring the coverage to 100% is driving OCD-ish me a little insane. If it is possible, I would be very grateful for the directions on how to do it.

Thanks!

Error running tests on windows

I am getting the following error when trying to run my tests on a windows machine:

Called from ? line 1
Called from massive/munit/MunitCommandLineRunner.hx line 43
Called from massive/munit/MunitCommandLineRunner.hx line 41
Called from massive/munit/MunitCommandLineRunner.hx line 59
Called from massive/munit/Config.hx line 30
Called from massive/munit/Config.hx line 64
Called from massive/munit/Config.hx line 71
Called from massive/munit/Config.hx line 94
Called from massive/neko/io/File.hx line 103
Called from massive/neko/io/File.hx line 66
Called from massive/neko/io/File.hx line 256
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 548
Called from massive/neko/io/File.hx line 1047
Uncaught exception - Invalid field access : split

JS: Timer issue when using AsyncTest.

I'm running into issues when trying to run async tests, where the timer throws the following error.

Uncaught TypeError: Cannot read property 'Timer' of undefined

It seems that this is the offending line of code https://github.com/massiveinteractive/MassiveUnit/blob/master/core/src/massive/munit/util/Timer.hx#L96 where it can't locate the Timer class.

Code to reproduce here:

@AsyncTest
    public function when_creating_a_stream__should_calling_emitWithDelay(asyncFactory:AsyncFactory) : Void {
        var handler:Dynamic = asyncFactory.createHandler(this, function(){
            Assert.isTrue(true);
        }, 12);

        Timer.delay(function() {
            handler();
        }, 10);
    }

Report output gets overwritten when testing multiple targets

The contents of the report directory is overwritten each time a different target's test-runner is executed.

Ideally the reports (and runner) for each target would stored in a target sub directory within the report directory so that multiple targets could be run at once and the results maintained.

Detect non-debug Flash Player

MUnit currently assumes tests are run in the Flash Debug Player, particularly where it comes to exception handling. We should detect this before tests and warn the user that unexpected behaviour may result, or abort the tests all together.

We should also be able to run tests in the release player, but that is less of a blocker.

CPP debug broken?

I'm able to run
munit test -as3 -debug
But unable to run
munit test -cpp -debug

Is this just me?

File system manipulation in tests has haxe/lib/munit/0,9,2,1 as current working directory

I was setting up some filesystem based neko tests and noticed that executing the line
neko.FileSystem.createDirectory("fstest");
would create fstest as subdirectory of haxe/lib/munit/0,9,2,1. This could cause problems with tests deleting files in that directory or polluting it.

For now I manually use neko.Sys.setCwd to set the cwd to the munit build directory, but I think this should be done automatically.

Exceptions testing bug

Try to run test case:

package ;
import flash.errors.ArgumentsError;
import massive.munit.Assert;

class ThrowBugExample {
public function new() {
}

@Test
public function test() {
    try {
        throwError();
        Assert.fail("Mast throws exception");
    } catch (error:ArgumentsError) {
        Assert.isTrue(true);
    }
}

public function throwError() {
    throw new ArgumentsError();
}
}

But unexpectable out is getted:

Class: ThrowBugExample
massive.munit.UnhandledException: VerifyError: Error # 1014:
Can't find class ArgumentsError. at ThrowBugExample#test

FAILED
Tests: 1  Passed: 0  Failed: 0 Errors: 1 Time: 0.028

May be it's haXe bug. Just I not insured jet.

Installation instructions are not working

This doesn't smell like a problem with munit itself, but it's pretty curious. When I run

haxelib install munit

I get the following error:

Invalid string length

I am able to install other libraries using haxelib. This only happens with munit. I'm on Mac OS 10.7.5. My haxelib version is 1.03 and my haxe compiler version is 2.10. I have neko installed. I immediately tried reinstalling Haxe 2.10 using the OSX automatic installer (http://haxe.org/download), but this didn't fix the problem.

I'm not sure how libraries are indexed my haxelib, but is it possible that there's some bug in how haxelib is tracking this project?

Testing silently fails when macro stack overflow occurs.

@:macro static function l() {
    l(); // stack overflow
    return null;
}

@Test
public function stackOverflow() {
    l();
}

Compiling this as test yields no error message at all. Non-test compilations correctly report the occuring stack overflow. All other macro related errors seem to be reported correctly.