japa / core

The core of Japa. It includes all the primitives to build a test runner

@japa/core

The core of the Japa tests runners

This repo contains the code for the core of the japa tests runner. You can use it create your tests runner, just like @japa/runner.

You can view the documentation on https://japa.dev

---

core's Issues

Can't start adonis server when migrated to @japa/runner

Description

I'm trying to migrate to new Japa version. According to migration guide I've copy-pasted configuration from japaFile.ts to bin/test.ts. Now I can't start http server.
I'm using ts-node and running node -r ts-node/register/transpile-only bin/test.ts
App root is at C:\dev\thermotex\back

bin/test.ts

process.env.NODE_ENV = 'testing';
process.env.ADONIS_ACE_CWD = join(__dirname, '..');
sourceMapSupport.install({handleUncaughtExceptions: false});

async function startHttpServer() {
  const {Ignitor} = await import('@adonisjs/core/build/src/Ignitor');
  process.env.PORT = String(await getPort());
  await new Ignitor(join(__dirname, '..')).httpServer().start();
}

Package version

@japa/runner: 2.0.3

Error Message & Stack Trace

Console output

 Error

 Cannot find module '@ioc:Adonis/Core/Env'
Require stack:
- C:\dev\thermotex\back\env.ts
- C:\dev\thermotex\back\node_modules\@adonisjs\application\build\src\Application.js
- C:\dev\thermotex\back\node_modules\@adonisjs\application\build\index.js
- C:\dev\thermotex\back\node_modules\@adonisjs\core\build\src\Ignitor\index.js
- C:\dev\thermotex\back\bin\test.ts

 at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] C:/dev/thermotex/back/node_modules/@cspotcode/source-map-support/source-map-support.js:679
  674|                }
  675|                request = newRequest;
  676|            }
  677|          }
  678|
> 679|          return originalValue.call(this, request, parent, isMain, options);
  680|        }
  681|      }
  682|      if (onConflictingLibraryRedirect) {
  683|        sharedData.onConflictingLibraryRedirectArr.push(onConflictingLibraryRedirect);
  684|      }

   1  Object.<anonymous>
     C:/dev/thermotex/back/env.ts:15

   2  Module.m._compile
     C:/dev/thermotex/back/node_modules/ts-node/src/index.ts:1455

   3  Object.require.extensions.<computed> [as .ts]
     C:/dev/thermotex/back/node_modules/ts-node/src/index.ts:1458

Testing API requests with Japa.

Description

Hi guys sorry to bother your work. I'm trying to use Japa to test some requests of my API. How could I use it like in Adonis ?

import test from 'japa'

test.group('Unit/Founder', (group) => {
  group.beforeEach(() => {})

  test('should store a summoner and return status code 200', async (assert, client) => {
    const region = 'br1'
    const summonerName = 'iLenon7'

    const response = await client.post(`/summoners?region=${region}&${summonerName}`)

    response.assertStatus(200)
  })
})

Package version

🎎 Japa Version - ^3.0.1

Relevant Information

If that is not possible to do with Japa I apologize for my ignorance, I'm new with programming, and If you have any tips of what I can use to make this happen! Thank you very much and have a great day!

getting access to the IoC container inside japa tests

I'm using japa for my functional tests.
My tests are in a dir outside the scope of the Adonis IoC. (they're in the /test dir)

.
├── ace
├── app
├── bootstrap
├── CHANGELOG.md
├── config
├── database
├── gulpfile.js
├── node_modules
├── package.json
├── providers
├── README.md
├── resources
├── server.js
├── storage
├── test
└── yarn.lock

meaning I can't do this in my test files

const Database = use('Database')
const User = use('App/Model/User')

How would I include the IoC into my test so that I can access the IoC?

💡 - Optional sorted / sequential test files

Description

Some test files need other test files to be run first, so making the sort optional would be interesting.

Package version

"devDependencies": {
  "japa": "^3.1.1",
}

Relevant Information

I was thinking about a Pull Request for this feature, adjusting the "configure" function and the "Loader/loadFiles", so making filesPaths.sort() optional and default true.

Ace Command

I've followed the instructions on the adonis cookbook to integrate japa with AdonisJS and it fits my needs really well.

I think one thing might me improved, though: the command for running the tests.

I've created an ace command for that, available below, so I'm able to run the tests with node ace test instead of node -r @adonisjs/assembler/build/register japaFile.ts.

/commands/Test.ts

import { BaseCommand } from '@adonisjs/core/build/standalone'
import { execFile } from 'child_process'

export default class Test extends BaseCommand {

  public static commandName = 'test'
  public static description = 'Run all tests.'

  public static settings = {
    loadApp: false,
    stayAlive: true,
  }

  public async run () {
    const child = execFile('node', ['-r', '@adonisjs/assembler/build/register', 'japaFile.ts'], {
        env: {
            ...process.env,
            FORCE_COLOR: 'true',
        },
    });
    if (child.stdout) {
      child.stdout.on('data', function(data) {
        console.log(data.toString().trim()); 
      });
    }
  }
}

Please let me know if that's a valid approach, I'm able to make a pull request given some instructions (mainly on the currently not implemented npm invoke, I guess).

Allow to put Node.js flags in the payload of configure ?

Description

Hi, i'm working on a experimental branch to switch over ESM on one of my project. But my test fail because i use japa.configure to run multiple tests with multiple files (because ESM is still under a flag ... so the new syntax fail)

Can you allow to put Node.js flags in the payload of configure ?

Example:

japa.configure({
    flags: ["--experimental-modules"],
    files: [
        // join(__dirname, "communication.js"),
        // join(__dirname, "utils.js"),
        // join(__dirname, "test.js"),
        join(__dirname, "wca.js")
    ]
});

Package version

2.0.10

Best Regards,
Thomas

nodemon conflicts

I'm trying to perform tests and through a custom script by nodemon, when I try to run the command I created japa:watch, it manages to run the test but right after the error below, I tried to remove the folder node modules and install again, but none of that worked

**package json command that iam trying to change to use with nodeman
**

"japa": "node -r @adonisjs/assembler/build/register japaFile.ts",
    "japa:watch": "nodemon --watch 'test/specs/**/*' --ext json,ts --exec npm run japa --quiet"

## Package version

npm: '6.14.15',
node: '14.17.6',

Error Message & Stack Trace

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] japa: `node -r @adonisjs/assembler/build/register japaFile.ts`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] japa script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\Lucas\AppData\Roaming\npm-cache\_logs\2021-09-29T00_46_54_039Z-debug.log

Logs

0 info it worked if it ends with ok
1 verbose cli [
1 verbose cli   'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'run',
1 verbose cli   'japa'
1 verbose cli ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prejapa', 'japa', 'postjapa' ]
5 info lifecycle [email protected]~prejapa: [email protected]
6 info lifecycle [email protected]~japa: [email protected]
7 verbose lifecycle [email protected]~japa: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]~japa: PATH: C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin;C:\Users\Lucas\www\facebook-api\node_modules\.bin;C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin;C:\Users\Lucas\www\facebook-api\node_modules\.bin;C:\Users\Lucas\bin;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\local\bin;C:\Program Files\Git\usr\bin;C:\Program Files\Git\usr\bin;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Users\Lucas\bin;C:\Python39\Scripts;C:\Python39;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn;C:\Program Files (x86)\Microsoft SQL Server\150\Tools\Binn;C:\Program Files\Microsoft SQL Server\150\Tools\Binn;C:\Program Files\Microsoft SQL Server\150\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn;C:\Program Files\Azure Data Studio\bin;C:\Program Files\nodejs;C:\ProgramData\chocolatey\bin;C:\Program Files\Git\cmd;C:\Program Files\Docker\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\Users\Lucas\AppData\Local\Microsoft\WindowsApps;C:\Users\Lucas\AppData\Local\Programs\Microsoft VS Code\bin;C:\Program Files\Azure Data Studio\bin;C:\Users\Lucas\AppData\Roaming\npm;C:\Program Files\Git\usr\bin\vendor_perl;C:\Program Files\Git\usr\bin\core_perl
9 verbose lifecycle [email protected]~japa: CWD: C:\Users\Lucas\www\facebook-api
10 silly lifecycle [email protected]~japa: Args: [
10 silly lifecycle   '/d /s /c',
10 silly lifecycle   'node -r @adonisjs/assembler/build/register japaFile.ts'
10 silly lifecycle ]
11 silly lifecycle [email protected]~japa: Returned: code: 1  signal: null
12 info lifecycle [email protected]~japa: Failed to exec japa script
13 verbose stack Error: [email protected] japa: `node -r @adonisjs/assembler/build/register japaFile.ts`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\index.js:332:16)
13 verbose stack     at EventEmitter.emit (events.js:400:28)
13 verbose stack     at ChildProcess.<anonymous> (C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\lib\spawn.js:55:14)
13 verbose stack     at ChildProcess.emit (events.js:400:28)
13 verbose stack     at maybeClose (internal/child_process.js:1055:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
14 verbose pkgid [email protected]
15 verbose cwd C:\Users\Lucas\www\facebook-api
16 verbose Windows_NT 10.0.19041
17 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "run" "japa"
18 verbose node v14.17.6
19 verbose npm  v6.14.15
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] japa: `node -r @adonisjs/assembler/build/register japaFile.ts`
22 error Exit status 1
23 error Failed at the [email protected] japa script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

Inject parameters into test function

Description

Is there a way to inject parameters into the test function? Something like this:

test('Test description', (assert, context) => {
  ...
})

Package version

3.1.1

Set default timeout for all tests

Description

I would like to change the default timeout for all my tests. This saves me from having to manually edit the timeout for each test. The database I am testing on is very slow and as a result all my tests take long to complete however this is not an error it's the expected behaviour.

OpenAPI HTTP Testing in Adonis with Japa

Description

I'm using AdonisJS for creating a microservice that's utilizing an Open API contract. I'd like to use Japa for testing with Open API (e.g. example below), but it doesn't appear the default japa package has the plugins field available for using the Open Api plugin defined here.

test('get users', ({ assert }) => {
  const response = await supertest(baseUrl).get('/users')
  assert.isValidateApiResponse(response) // This should probably be isValidApiResponse in README.md of japa/assert
})

I've tried using @japa/runner and @japa/assert directly in Adonis, but went down a rabbit hole and am wondering if this feature will be available soon directly in japa? I've configured my test server following this article: https://docs.adonisjs.com/cookbooks/testing-adonisjs-apps

Package version

  "devDependencies": {
    "@adonisjs/assembler": "^5.4.2",
    "@types/supertest": "^2.0.12",
    "adonis-preset-ts": "^2.1.0",
    "eslint": "^8.10.0",
    "eslint-config-prettier": "^8.5.0",
    "eslint-plugin-adonis": "^2.1.0",
    "eslint-plugin-prettier": "^4.0.0",
    "execa": "5.1.1",
    "get-port": "5.1.1",
    "japa": "^4.0.0",
    "pino-pretty": "^7.5.3",
    "prettier": "^2.5.1",
    "supertest": "^6.2.2",
    "typescript": "~4.5",
    "youch": "^3.1.1",
    "youch-terminal": "^2.1.3"
  },
  "dependencies": {
    "@adonisjs/core": "^5.5.3",
    "@adonisjs/repl": "^3.1.9",
    "openapi-backend": "^5.3.0",
    "proxy-addr": "^2.0.7",
    "reflect-metadata": "^0.1.13",
    "source-map-support": "^0.5.21"
  }

Really love Adonis, great work!

[Feature suggestions] Mock Functions

Package version

@japa/runner v2.5.1

Describe the bug

Note : this is not a bug, this is a feature suggestion (unless it already exists, then its my bad 😅, please refer the doc fo it if possible)

Hi,

The feature im suggesting is the possiblity to mock functions, let suppose i have a class User:
`Class User {
...methods and properties

public someMethode(userId: number) {
    // some logic
}

public getUserInfo(userId: number) {
    if (condition) {
        this.someMethode(userId)
    }

    // rest of function logic
}

}`

i define a test for User::getUserInfo:

`
test.group("User.getUserInfo", () => {
const instance = new User();

test("success", async ({ assert }, userId) => {
    const userInfo = instance.getUserInfo(userId)

    // the different assertions
}).with([
    1,
    15975,
    579846
])

});
`

is there a way to mock User::someMethode that is going to be called inside the method getUserInfo, throught the test, so i can have an information that it was actually called.

thank you in advance for the response, and please correct me if this already exists and/or if its not technically possible.

Reproduction repo

No response

Run only one test

Hello!
How to run only one test from multiple tests?

For example, I need run test with name 'not load relation with null value in foreign key'.
Hot to can i run it? Command cli argument?

Thx!

Can't start tests on windows

Description

Unable to run tests on Windows

Package version

    "@japa/expect": "1.1.3",
    "@japa/runner": "2.0.5",
    "@japa/spec-reporter": "1.1.11",

Error Message & Stack Trace

{ filePath: 'C:/Users/kozac/Dev/kitlink/test/homepage.spec.ts' }

   Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
  
   at Object.importer c:/Users/kozac/Dev/kitlink/bin/test.ts:26
    21|      files: ['./test/**/*.spec.ts'],
    22|      plugins: [expect()],
    23|      reporters: [specReporter()],
    24|      importer: (filePath) => {
    25|        console.log({filePath})
  > 26|        return import(filePath)
    27|      },
    28|    },
    29|  })

Relevant Information

Can group initialization be asynchronous?

In my test case, I want to create multiple instances of a model in a database. And run a separate test for each:

test.group('Status code must mee 2xx or 3xx', async (group) => {
  let links = await ShortLinkFactory.createMany(50)

  links.forEach(link => {
    test(link.id, async (assert) => {
      await supertest(link.url)
        .get('/')
        .expect((res) => {
          assert.isAbove(res.statusCode, 200)
          assert.isBelow(res.statusCode, 400)
        })
    })
  })
})

Of course I can do everything in one test, but then I do not get a beautiful and convenient log that conveniently displays which links returned the expected result and which failed

Where to post some questions ?

Hello,

I've a question about Japa on adonis but I think it could be a general question for Japa, where can I post it?

---

Here my code in /todos:

<h1 class="text-2xl font-bold text-cyan-700">Tâches de la journée</h1>

Do I have to write a test to check if the text is in a h1? And how to check that the text is in a h1 without considering tag attributes?

  test('title must be in a h1', async ({ client }) => {
    const response = await client.get('/todos')

    response.assertStatus(200)
    response.assertTextIncludes(
      '<h1>Tâches de la journée</h1>'
    )
    // Maybe we could use a regex, or a querySelector or I'm wrong on what to test?
  })

Is functional tests only for behavior without considering styles? And how to test styles?

Handling events on tick 1 (constructor) doesn't work

Hi,

What i try to achieve :
I want to catch a event triggered on tick 0 (constructor most of the time so). This should be triggered on tick 1 (because of internal nextTick of events package). So when my test catch this event, it's done().

The problem :
I prepared a simple example to demonstrate you the problem

const test = require('japa');
const events = require('events');

class testJapa extends events {
    constructor() {
        super();
        this.emit('construct');
    }
}
let eventWrapper = new testJapa();

test('eventWrapper construct',function(assert,done) {
    eventWrapper.on('construct',function() {
        console.log('construct triggered!');
        done();
    });
});

Ok it's fail.

Error: Test timeout, ensure "done()" is called; if returning a Promise, ensure it resolves.

I know, you are a good developer so the test handler has been nextTick to ensure asynchronous execution.

So i update my code for

constructor() {
    super();
    process.nextTick( () => {
        this.emit('construct');
    });
}

Result : *Same error as before

So i made some tests and decide to try this :

let eventWrapper;
setImmediate(() => {
    eventWrapper = new testJapa();
});

And no the error is : TypeError: Cannot read property 'on' of undefined

So we have to trigger a another event from the construct one (and the second will be catched, because he is on tick 2 now).

Waiting for your feedback on the subject.

Best Regards,
Thomas

Asynchronous datasets / datasets not supported anymore? Multiple chained asynchronous test dependencies

While having an asynchronous group callback would make this streamlined and very simple, I fail to see how to have multiple asynchronous test dependencies.

Additionally, counting on typescript, the example for typescript intellisense in the documentation seems broken as the typings don't allow this.

Description

I have two datasets, one being asynchronously mocked, and another being asynchronously built partly based on the first dataset.

On a "simple" approach where test groups would be asynchronous, I would simply need to have line 1 with my first await, and line two with my second await.

The documentation tells me to use the .with() property after a test() call, which per typescript's compiler, doesn't exist.

Property 'with' does not exist on type 'RunnerTest'.ts(2339)

The typescript-intellisense-specific documentation tells me to use the .with() property on a test() that would've only been called with a single parameter (the test name).
Trying to do so raises multiple errors.

Expected 2 arguments, but got 1.ts(2554)
[index.d.ts(29, 45): ]()An argument for 'callback' was not provided.
Property 'with' does not exist on type 'RunnerTest'.ts(2339)

Is this feature still supported ?

If not, how would I expect to have asynchronous dataset initialization that wouldn't require repeated boilerplate code?

Package version

• ^4.0.0

Match all tags in filter

Description

Currently when providing multiple tags to the config.filters.tags, any test that matches at least one of the tags will be run. It would be helpful if we could have the option to run tests that match all of the tags. I originally built a plugin to achieve this behavior via .skip() but the default reporter logs skipped tests to the console, as it should of course. So alternatively, would it be possible to filter out a test in a plugin so that custom filters can be implemented?

Package version

"@japa/expect": "^1.1.4",
"@japa/run-failed-tests": "^1.0.7",
"@japa/runner": "^2.0.7",
"@japa/spec-reporter": "^1.1.12",

Relevant Information

async function (_, runner) {
  runner.onSuite((suite) => {
    suite.onTest((test) => {
      if (!doesTestMatchAllTags(test, tags)) test.skip()
    })
    suite.onGroup((group) => {
      for (const test of group.tests) {
        if (!doesTestMatchAllTags(test, tags)) test.skip()
      }
    })
  })
}

need example of running a test

I see the examples, but what would the command look like when running a test with japa?

I assume: $ japa test1.js

Seems like this should be right after:
https://github.com/thetutlage/japa#installation

How get dynamic title for each test with array strings in dataset?

Description

Documentation says:

The dynamic values (aka interpolation) works by wrapping the property name inside single curly braces {email}.

It work, if I have array of objects like [{email: '1'}, {email: '2'}, {email: '3'}]. But in my case dataset is just array of strings ['1','2','3']. How I can get Dynamic title for each test ?

test('validate collection - { ??? }')
  .with(async () => Object.keys(await lookupCollections()))

Add Suite layer to Refiner

Version 5.0.17-0

I assumed that a suite would be another layer in the Refiner, but I was a bit surprised to find that it wasn't.

I was able to hack in userland a way to add suites to the runner only when it's listed, but it feels like it should already be included. Especially when we want to open the runner up to plugins.

Let me know your thoughts, thanks!

Question: How to do transactions of Lucid during testing

Prerequisites

How do one does transactional testing of Lucid models in test cases.

Description

As japa is capable of running multiple testcases in parallel, it would be very dangerous that the data set of testing is polluting the data set of another test case. And parallelization also means that the migrations can't be done for each test group.

I tried the following but failed

LoginController.ts

import { HttpContextContract } from '@ioc:Adonis/Core/HttpContext'

export default class LoginController {
  /**
   * Login action
   * @param request
   */
  public async login ({request, auth}: HttpContextContract) {
    const {email, password} = request.all()
    return await auth.use('api').attempt(email, password)
  }
}

Japa test file

import test from 'japa'
import supertest from 'supertest'
import Route from '@ioc:Adonis/Core/Route'
import Admin from 'App/Models/Admin'
import fakerStatic from 'faker'
import Database, { TransactionClientContract } from '@ioc:Adonis/Lucid/Database'

const BASE_URL = `http://${process.env.HOST}:${process.env.PORT}`

test.group('Login Controller', (group) => {
  let transactionCtx: TransactionClientContract

  group.beforeEach(async () => {
    transactionCtx = await Database.transaction()
  })

  group.afterEach(async () => {
    await transactionCtx.rollback()
  })

  test('ensure login works with valid credentials', async (assert) => {
    const adminModel = new Admin()
    adminModel.useTransaction(transactionCtx)
    const admin = adminModel.fill({
      email: fakerStatic.internet.email(),
      name: fakerStatic.name.firstName(),
      password: 'password',
    })
    await admin.save()

    const result = await supertest(BASE_URL)
      .post(Route.makeUrl('api.v1.admin.login') as string)
      .set('Content-Type', 'application/json')
      .set('Accept', 'application/json')
      .send({email: admin.email, password: 'password'})
      .expect(200)
    assert.containsAllKeys(result.body, ['token', 'type'])
  })
})

Package version

{
  "name": "nodejs-backend",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "build": "node ace build",
    "start": "node ace serve --watch --debug",
    "lint": "eslint . --ext=.ts"
  },
  "devDependencies": {
    "@adonisjs/assembler": "^2.0.0",
    "@types/jsdom": "^16.2.4",
    "@types/supertest": "^2.0.10",
    "adonis-preset-ts": "^1.0.4",
    "eslint": "^7.10.0",
    "eslint-plugin-adonis": "^1.0.15",
    "execa": "^4.0.3",
    "get-port": "^5.1.1",
    "japa": "^3.1.1",
    "jsdom": "^16.4.0",
    "pino-pretty": "^4.2.1",
    "supertest": "^5.0.0",
    "ts-node": "^9.0.0",
    "typescript": "^4.0.3",
    "youch": "^2.1.0",
    "youch-terminal": "^1.0.1"
  },
  "dependencies": {
    "@adonisjs/ace": "^6.0.0",
    "@adonisjs/auth": "^4.3.1",
    "@adonisjs/core": "^5.0.0-preview-rc",
    "@adonisjs/fold": "^6.0.0",
    "@adonisjs/lucid": "^8.4.4",
    "@gauravgango/adonisjs-stomp-driver": "1.0.0-beta.20",
    "luxon": "^1.25.0",
    "mysql": "^2.18.1",
    "phc-argon2": "^1.0.10",
    "phc-bcrypt": "^1.0.3",
    "proxy-addr": "^2.0.6",
    "reflect-metadata": "^0.1.13",
    "source-map-support": "^0.5.19"
  }
}

Error Message & Stack Trace

Login Controller

  ✖ ensure login works with valid credentials
    Error: expected 200 "OK", got 400 "Bad Request"

Relevant Information

It appears the request happening through japa is not on same node request where the main test case is running hece the request is not in same transactional reference

Filter by group title also executes "root" tests

Description

Hey !

I just noticed that when filtering by a group name, tests that are not in a group are also executed.

Exemple :

import { test } from '@japa/runner'

test.group('Maths.add', () => {
  test('hey', ({ assert }) => assert.isTrue(true))
})

test('root', () => console.log('Hey'))

When executing npm run test --groups Maths.add, the test root is also executed

Is this the desired behavior? It seems a bit weird to me, and I don't see anything related to it in the documentation
( By the way it would be awesome if we could have an Algolia search 😋 )

Package version

@japa/runner 2.0.8

---

If you confirm that this is not the desired behavior, I can take care of it and suggest a PR 👍

Custom order for running tests

Description

There is a pattern to use a custom order for tests?

Actually, I'm using this syntax:

1_a_test.spec.ts
2_b_test.spec.ts
3_c_test.spec.ts
4_d_test.spec.ts

In VSCode, my files are showed up in the right order, but when Japa tries to run my tests, they're not in the right order.

What's the order Japa try to run tests? I didn't found this clearly in the docs.

Skip test by default when calling ".skip()" without parameter

Description

At this point, when calling .skip() without a boolean parameter the test won’t be skipped. For example, the following test fails even though it’s marked as skipped:

test('fails', async ({ expect }) => {
  expect(true).toBe(false)
}).skip()

// the test above runs and fails when calling `.skip()` instead of `.skip(true)`

My suggestion is to change the default behavior of .skip) to skip the test when not providing a parameter value. I guess the API of the test class is thoughtfully designed in the way it is. Maybe my suggested change (default the skip parameter to true) can be considered as a useful default.

Package version

I’m using "@japa/runner": "~1.2.0"

Additional Information

I’m happy to create a pull request adding my suggested functionality.

Error in group functions is swallowed and not shown

Description

When an exception is thrown in group.setup(), nothing is shown and the test doesn't run. See the error below:

group.setup(async() => {
    app = await setupApplication();
    await setup(app);
});

Package version

2.0.11

Error Message & Stack Trace

event = "group:end"
payload = [
  {
    phase: 'setup',
    error: Error: Cannot find module '@adonisjs/repl'
    Require stack:
    - /Users/mgalbusera/Documents/development/denise/adonis5-jwt/test-helpers/__app/noop.js
        at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
        at resolveFileName (/Users/mgalbusera/Documents/development/denise/adonis5-jwt/node_modules/resolve-from/index.js:29:39)
        at resolveFrom (/Users/mgalbusera/Documents/development/denise/adonis5-jwt/node_modules/resolve-from/index.js:43:9)
        at module.exports (/Users/mgalbusera/Documents/development/denise/adonis5-jwt/node_modules/resolve-from/index.js:46:47)
        at Object.resolveFrom (/Users/mgalbusera/Documents/development/denise/adonis5-jwt/node_modules/@poppinss/utils/build/src/Helpers/resolveFrom.js:25:39)
        at Registrar.loadProvider (/Users/mgalbusera/Documents/development/denise/adonis5-jwt/node_modules/@adonisjs/fold/build/src/Registrar/index.js:50:25)
        at Registrar.collect (/Users/mgalbusera/Documents/development/denise/adonis5-jwt/node_modules/@adonisjs/fold/build/src/Registrar/index.js:68:59)
        at Registrar.register (/Users/mgalbusera/Documents/development/denise/adonis5-jwt/node_modules/@adonisjs/fold/build/src/Registrar/index.js:97:9)
        at /Users/mgalbusera/Documents/development/denise/adonis5-jwt/node_modules/@adonisjs/application/build/src/Application.js:530:41
        at Application.registerProviders (/Users/mgalbusera/Documents/development/denise/adonis5-jwt/node_modules/@adonisjs/application/build/src/Application.js:524:9) {
      code: 'MODULE_NOT_FOUND',
      requireStack: [Array]
    }
  }
]

PS: I don't have @adonisjs/repl since I'm in a library and not an actual app.

Possible to gather test coverage?

Description

I really like the framework and the simplicity - currently use jest, the only thing I’m unclear on is how to gather test coverage with Japa. Is that something that’s possible or unsupported?

Add the ability to use Dataset property in the title

Hello @thetutlage. Thanks for making this awesome tool.

Description

Jest allow us to customize the test title of a dataset for each row, would be awesome if Japa allow to use the dataset row to customize the test title. This would improve the coding and reader experience.

With JEST you can customize the test title with the dataset property using the $ symbol with the property name, maybe Japa can use the same strategy.

https://jestjs.io/docs/api#testeachtablename-fn-timeout

name: String the title of the test block.

• Generate unique test titles by positionally injecting parameters with printf formatting:
  • %p - pretty-format.
  • %s- String.
  • %d- Number.
  • %i - Integer.
  • %f - Floating point value.
  • %j - JSON.
  • %o - Object.
  • %# - Index of the test case.
  • %% - single percent sign ('%'). This does not consume an argument.
• Or generate unique test titles by injecting properties of test case object with $variable
  • To inject nested object values use you can supply a keyPath i.e. $variable.path.to.value
  • You can use $# to inject the index of the test case
  • You cannot use $variable with the printf formatting except for %%

Question: How to run tests as an authenticated user

Is there a way to test api endpoints that require auth?
Example:
I want to test the creation of a organization in my api endpoint, but to create it I need to be an authenticated user

I could not find it in the docs, right now what I'm doing is call the login endpoint and save the response cookies to use in future requests but it is a very ugly way to do it specially when having tests in different files that doesn't run in a controlled order
¿Is there a better way to do it ?

Need spy function in assert

I think, good idea, if you are extend original chai of lib as https://github.com/chaijs/chai-spies

expect(spy).to.have.been.called.with('foo');

Catch throw async/await

Hi!

I write test for async function where throw exception.

  test('Should ignore invalid task file types', async (assert) => {
    const Helpers = getHelpers('badFile')
    const scheduler = new Scheduler(Helpers)
    const fn = async () => {
      await scheduler.run()
    }
    await assert.doesNotThrow(fn)
  })

And catch test error:

...
✖ Should ignore invalid task file types (3ms)
...
1. Should ignore invalid task file types
  expected [Function: fn] to be a function
  async () => {
    await scheduler.run()
  } => undefined

How do i write test in correct view?
Thx!

Running multiple files does not work

While running individual test works perfectly, I'm unable to run multiple test files. My test are written in .ts and setting up a index.js file to configure multiple test doesn't work.

Content of index.js

`
require('ts-node').register();

const { configure } = require('japa');

configure({
files: ['/*.ts'],
experimentalEsmSupport: false,
});

`
Note
experimentalEsmSupport was set to both true and false and return the same result. On the command line. ZERO TESTS EXECUTED was printed.

Package version

4.0.0

Error Message & Stack Trace

ZERO TESTS EXECUTED Printed

can't get Japa to work with new version of Adonis 5

Hi guys,
i just installed japa to write some tests for my repositories/service classes
I'm getting this error while trying to run with node.

[Feature] Support global before each and after each

Description

I usually change a lot of things in my database during the tests, and placing the same beforeEach blocks in every test group is a little bit frustrating. I really would like to prepare auth, clean users, etc, before every test.

Change Private methods and properties to Protected to use Japa classes as Base classes

Version 5.0.17-0

First things first, I love the new version, this package is headed in the right direction and look forward to it getting even better!

I'm working on my own version of the Vow test runner for AdonisJS since it hasn't been ported over for version 5.

I would like to have the freedom to build on top of the Japa classes by extending them on my own custom classes.
Problem is that most of methods and properties are private instead of protected, so they can't be overridden or accessed, even when they are extended.
It makes the custom classes unwieldy trying to find ways around that with a bunch of extra clutter.

Since the idea of Japa is to build on top of it, it would make sense for the class to be extendible with the methods and properties available for the developer to override and access.

Of course, there's a time and place for private methods and properties, but closer to the end user.

Thanks!

passing an async function into tests, throws syntax error

using
node v6.9.4
japa v1.0.0

trying to use async functions as shown in the examples, but get a syntax error.

const test = require('japa')

test.group('API', group => {
  group.beforeEach(() => {

  })

  test('Async test', async (assert) => {
    const result = await new Promise(function (resolve, reject) {
      resolve('done')
    })

    console.log(result)
    assert.isTrue(true)
  })
})

/home/landan/www/adonuxt_test/tests/functional/tests/api.js:28
  test('Async test', async (assert) => {
                           ^
SyntaxError: Unexpected token (
    at Object.exports.runInThisContext (vm.js:76:16)
    at Module._compile (module.js:542:28)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at requireDir (/home/landan/www/adonuxt_test/node_modules/require-dir/index.js:122:33)
    at Object.<anonymous> (/home/landan/www/adonuxt_test/tests/functional/test.js:1:85)

Question - Is there a parallel or concurrent run support ?

Is there a parallel or concurrent run support ?

listening to Events

Is there an example of how to use these?
https://github.com/thetutlage/japa#events

I'm looking to send test results to a webhook in another app.

Cannot Access Assert from JapaFile.ts

Error Message & Stack Trace

node -r @adonisjs/assembler/build/register japaFile.ts

I'm trying to run the following commands given in this issue #9
Assertion looks like it has been renamed to Assert

import { Assert } from 'japa/api';
import chaiSpies from 'chai-spies';
Assert.use(chai-spies)

Error: Cannot find module 'japa/api'
Require stack:
- /Users/projordan/workspace/platform-backend/japaFile.ts
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:999:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/projordan/workspace/platform-backend/japaFile.ts:6:1)
    at Module._compile (node:internal/modules/cjs/loader:1097:14)
    at Module._compile (/Users/projordan/workspace/platform-backend/node_modules/pirates/lib/index.js:99:24)
    at Module._extensions..js (node:internal/modules/cjs/loader:1151:10)
    at Object.newLoader [as .ts] (/Users/projordan/workspace/platform-backend/node_modules/pirates/lib/index.js:104:7)
    at Module.load (node:internal/modules/cjs/loader:975:32)

Is it possible to create nested groups?

I searched the docs and previous issues and couldn't find any details: is it possible to have sub-groups? For example:

test.group('Third Party Integration', group => {

  test.group('Retrieving data', () => {
    test('calls the api', () => {
       ...
    });
  });

  test.group('Submitting updates', () => {
    test('calls the api', () => {
       ...
    });
  });

});

would output

Third Party Integration
  Retrieving data
    > calls the api
  Submitting updates
    > calls the api

This is how mocha works when I nest describe() blocks. I think I can live without this, but it's helpful once you start having lots of tests, and I want to be sure it's not possible at the moment. When I nest test.group inside itself, it just ignores the top-level group.

Can we define timeout on the groups?

Having to define timeout on every single test seems kinda odd.

Broken documentation link

The link on readme is broken.
https://japa.adonisjs.com/core/api/overview

JUnit or xUnit 2 Test Results

Description

Are there any plans on adding JUnit or xUnit Reports to japa? This will allow us to display some statistics about each particular test that has passed or failed in certain CI/CD pipelines. Notably Azure Dev Ops Pipelines is one of them.

I noticed that in the main documentation you worded that the results were displayed with the "default" list reporter, so I am assuming that there is a way to provide an alternate reporter that will be able to generate out the results in the relevant format?

TypeError: assert is not a function

I get this when i try to use the most basic form of assertion :

test('something simple', (assert) => {
  assert.plan(2)

  assert(something)
  assert(somethingElse)
});

While it is documented in Chai, and in node.js core assert module itself that the usage of assert(test [, message]) is equivalent to assert.equal(true, test[, message]) (and is less verbose)

"Packages installation failed" Error on Windows Server 2019 and Windows 11

Package version

4.0.0

Describe the bug

Unable to install japa on Windows Server 2019 and Windows 11 following the Installation and Usage guide in the documentation.

Executing npm init japa@latest in the project root and following the prompts results in the following error after the bin folder is created.

> Select the tests reporter · @japa/spec-reporter
> Select the assertion library · @japa/expect
> Select additional plugins · No items were selected
> Select the project type · TypeScript ESM
> Want us to create a sample test? (y/N) · false

CREATE: bin/test.ts
CREATE: bin/japa_types.ts


× Packages installation failed

The above error happens regardless of what options are chosen during the installation prompt.

Reproduction repo

No response

How to test informations in headers ?

Hello, i am developing a test that search one user in database with base about id of this user, i am passing this information in headers
so.. how can i realize the test about it ?

because i can't find this information in docs