m-mcgowan / spark-unit-test Goto Github PK

To allow tests to be flashed locally in an automated environment, an "enter DFU" command is required. This will reset the core and restart it in DFU mode automatically when this Spark function is called:

cmd("enterDFU")

To allow external devices to monitor the progress of the tests (e.g. a ci server), the test progress and test results should be posted to the cloud.

• skipped, passed, failed and count variables that reports these values.
• have a state variable, with values
  • waiting: the test hasn't started yet, waiting for external input to start the test runner
  • running: tests are executing
  • complete: all tests have executed
• when the test runner state is changed, post an event. This notifies clients that the test suite is available (state=waiting), is running (state=running) and when complete (state=complete).
• a command function to allow external control of the test, with these commands:
  • start: start the test (if not already running.)

At present, test output is available only via the Serial port.

The LED in the spark could be used to give an indication of the test result so far:

• green: all tests so far have passed
• orange: tests have passed or have been skipped
• red: there are failed tests

The LED should flash while tests are being executed, and then shift to a solid color when all tests have been executed.

Hi,

I am developing on Windows 10, in Particle Dev. I have added unit-test.h to my project folder, included the header in my .ino file, and tried compling the project. However, I am running into a lot of errors, including system.bootloader() not recognized. Am I missing a step?

Thank you

The spark unit test library supports selecting which tests are to be run and which are skipped.

This functionality can be made dynamic by exposing this as functions in the cloud. This will allow the build server to select which tests should be run.

This builds on the cmd function from #3 and adds these arguments:

• exclude=<pattern>: exclude tests matching the given glob pattern
• include=<pattern>: include tests matching the given glob pattern

The include/exclude feature is described in the arduino-unit documentation.

At present the test report is human-readable text. Some additional markup would make this easier to parse on a machine.

Consider using json or some other structured markup.

A function can be used to specify which output format is used. This is then used to set a flag in the Test class so that output is adjusted accordingly.

Hello :)

It would be awesome to have a repository with very basic things, like Hello World, pressing buttons and etc to get a grip of how Unit Tests works for Spark.

If you can create a simple example, I could help writing more.

As a quick fix to get the tests working, I combined everything into a single header.

It's now time to undo that by starting with the original repo and then reapplying my changes.

Local executors can get the test report via the Serial interface.

Remote test executors don't have this option. At present there is no easy way to pass a large amount of data via the cloud. Here are the options:

• use Spark.publish(): maximum event size is 64 bytes and maximum of 1 per second, 4 in a burst.
• open a TCP server: this could work, but need to find out our public IP address for the remote client to connect to. There are services on the internet that can provide this. But too much effort!
• use spark variable: copy say) 600 bytes form the log to a variable. Use a function to advance the pointer (and variable content) to the next 600 bytes. Function returns -1 when no more content (which then wraps around and starts from the beginning). This is fairly easy to code, test and use.
• write to github directly: too tightly coupled to a specific use-case.

I'll implement using a variable and a function to advance the start point. Interaction like this:

• call update function - this fetches the N bytes from the circular buffer, writes them to the variable and returns N. This advances the pointer in the circular buffer.
• if circular buffer has 0 characters available, check test status flag to see if test is complete - when complete return -1, else return 0. This allows client code to continue retrying when the log is still open.

Although a little hacky, this allows relatively small streams (<1MB) to be pushed through the cloud relatively quickly.

At present, there's boilerplate code required to test up a test application:

void setup()
{
    Serial.begin(9600);
}

void loop()
{
    if (Serial.available()) {
        char c = Serial.read();
        if (c=='t') {
            static bool run = false;
            if (!run) {
                Serial.println("Running tests");
                run = true;
            }
        }
    }
    if (run) {
        Test::run();
    }
}

It would be more convenient to code these as macros so that users can get typical test setup and loop behaviour without copying and pasting code.

e.g.

UNIT_TEST_APP();

This would delegate to unit_test_setup() and unit_test_loop(). Advanced test cases are free skip the UNIT_TEST_APP macro and instead call these functions directly from setup and loop.