haoliangyu / fachi Goto Github PK

• Error handling: the test case should fail on error but the hook should still run (#2)
• suite vs test:
  • Suite should run tests, logging (#3), and handle errors.
  • Test should handle the underline benchmark framework.
• Typings
• Documentations

https://github.com/microsoft/folio

A typical test file in fachi may contain multiple test suites and each test suite may contain test cases.

suite('search', function () {
  before(function () {
    // ...
  })

  after(function () {
    // ...
  })

  bench('by id', function () {
    // ...
  });
})

In the hierarchy, file -> suite -> test, items in each level are separate, and the execution result of a file/suite/test doesn't affect others in the same level. This provides the isolation mechanism in error situations, for example,

• the syntax error in a test file should not affect the execution of a second file,
• the file I/O in a]the suite before hook should not affect the execution of another suite in the same file,
• the error from a bug in a test case should not affect the execution of another test in the same suite.

The emergent error in a file pauses the execution of that file. So as the suite (hooks) and the test cases. After printing the error message, then it should move on to run the next file until all are done.

When the CLI tool runs, it should print out the log messages for each file, suite, and case. The format should look like

/my-project/benchmarks/search.js

  search
    * by id x 4,161,532 +-0.99% (59 cycles)
    * by name x 4,231,578 +-0.89% (56 cycles)

  search-string
    * by org x 5,234,786 +-0.95% (73 cycles)

The logging logic should be wrapped into a Logger class.