Hey, thanks for the plugin!

I'm getting "No tests found" when running from the root of an umbrella project, but running inside each app works, am i missing something?

Hi there!

This commit: a9b61bb
broke the plugin updating automatically for me, as the submodule was not getting updated correctly by git. I fixed it manually by going into ~/.local/share/nvim/lazy/neotest-elixir and running git submodule update iex-unit, which seems to have set everything straight. Is there any other way to reference that submodule in a way that won't break installs?

Thanks!

🗺️ Background

I just setup elixir in neovim. Running tests with neotest works but debugging isn't working yet. I have test debugging working for C# but I'm guessing my dap setup could be bad? Or maybe it isn't what neotest-elixir expects?

I'm using the Gilded Rose to learn Elixir (source)

🐛 Problem

lua require('neotest').run.run({strategy = 'dap'}) errors with:

Adapter doesn't support chosen strategy.
neotest-elixir: ...are/nvim/site/pack/packer/start/neotest/lua/nio/init.lua:105: The coroutine failed with this message:
...ker/start/neotest/lua/neotest/client/strategies/init.lua:55: attempt to index local 'instance' (a nil value)
stack traceback:
        ...ker/start/neotest/lua/neotest/client/strategies/init.lua: in function 'run'
        .../pack/packer/start/neotest/lua/neotest/client/runner.lua:129: in function '_run_spec'
        .../pack/packer/start/neotest/lua/neotest/client/runner.lua:89: in function <.../pack/packer/start/neotest/lua/neotest/client/runner.lua:88>
Press ENTER or type command to continue

This neotest issue seems to suggest that this problem involved the neotest adapter (source)

🏗️ Configs

  use({
    "nvim-neotest/neotest",
    requires = {
      {
        "jfpedroza/neotest-elixir",
      },
    }
  })

dap.adapters.mix_task = {
  type = 'executable',
  command = require("mason-core.package"):get_install_path()..'/bin/elixir-ls-debugger',
  args = {}
}

dap.configurations.elixir = {
  {
    type = "mix_task",
    name = "mix test",
    task = 'test',
    taskArgs = {"--trace"},
    request = "launch",
    startApps = true, -- for Phoenix projects
    projectDir = "${workspaceFolder}",
    requireFiles = {
      "test/**/test_helper.exs",
      "test/**/*_test.exs"
    }
  },
}

local neotest_ok, neotest = pcall(require, 'neotest')
if not neotest_ok then
  return
end

local function keymap(key, cmd, description)
  vim.keymap.set({"n", "v"}, key, "<cmd>"..cmd.."<cr>", { desc = description })
end

keymap("<leader>tcr", "lua require('neotest').run.run(vim.fn.expand('%'))", "[t]est [c]lass [r]un")
keymap("<leader>tcd", "lua require('neotest').run.run({vim.fn.expand('%'), strategy = 'dap'})", "[t]est [c]lass [d]ebug")
keymap("<leader>tr", "lua require('neotest').run.run()", "[t]est [r]un")
keymap("<leader>td", "lua require('neotest').run.run({strategy = 'dap'})", "[t]est [d]ebug")
keymap("<leader>tsr", "lua require('neotest').run.run({suite = true})", "[t]est [s]uite [r]un")
keymap("<leader>tsd", "lua require('neotest').run.run({suite = true, strategy = 'dap'})", "[t]est [s]uite [d]ebug")
keymap("<leader>tw", "lua require('neotest').summary.toggle()", "[t]est [w]indow")

local adapters = {}
if packer_plugins["neotest-dotnet"] and packer_plugins["neotest-dotnet"].loaded then
  table.insert(adapters, require("neotest-dotnet"))
end
if packer_plugins["neotest-elixir"] and packer_plugins["neotest-elixir"].loaded then
  table.insert(adapters, require("neotest-elixir"))
end
neotest.setup({
  adapters = adapters,
  diagnostic = {
    enabled = true
  }
})

If you don't have much time, I'd happily become a maintainer.

I want to add more tests to make maintenance of this repository easier. Additionally, I would like to remove support for mix_test_interactive because neotest's watch feature is really great.

Yo:

not sure if this is a problem with this plugin or neotest per se.

On a fresh compile step, test running can hang for a longgggggggg time with no indication that stuff is compiling.

I have no idea what to suggest here but I think it's kind of a problem.

Hey! I just started looking into the Neotest library and was about to start working on an elixir adapter for it. Figured I'd do a search and found you had just begun one!

I'd love to help out, what all is left/is there a list of things you have left to finish?

If test name contains \n symbols, then such tests are not found by the adapter:

  test "new\nline" do

In case of multiline (heredoc strings), UI representation is not great:

test """
  new
  line
  """ do

I suggest replacing new lines with spaces, as it is done in Elixir for tests with --trace option https://github.com/elixir-lang/elixir/pull/11728/files

how to print Logger output on output-panel?
I'm using the config below but there is no any logger output printed

        config = function()
            local neotest = require("neotest")

            neotest.setup({
                adapters = {
                    require("neotest-elixir"),
                },
                log_level = vim.log.levels.DEBUG,
            })
      end

First, thanks for this plugin. Running file per file is working very well

What am I trying to accomplish

I have habits to run all tests in a project. In other language like go and c#, I can do that with require("neotest").run.run(vim.fn.getcwd()) command.

What I am expecting

I am expecting to see tests running succefull

What I am experiencing

All tests are failling with error

error: syntax error before: '='
    │
  1 │ defmodule <%= inspect context.web_module %>.<%= inspect Module.concat(schema.web_namespace, schema.alias) %>RegistrationLiveTest do
    │             ^
    │
    └─ _build/dev/lib/phoenix/priv/templates/phx.gen.auth/registration_live_test.exs:1:13

How to reproduce

Make a file ~/.config/nvim-elixir/init.lua file with the following content. It is just install minimal config with lazy.nvim

local lazypath = vim.fn.stdpath("data") .. "/lazy/lazy.nvim"

if not (vim.uv or vim.loop).fs_stat(lazypath) then
 vim.fn.system({
   "git",
   "clone",
   "--filter=blob:none",
   "https://github.com/folke/lazy.nvim.git",
   "--branch=stable", -- latest stable release
   lazypath,
 })
end

vim.opt.rtp:prepend(lazypath)

local lazy = require("lazy")

lazy.setup(
 {
   "nvim-neotest/neotest",
   event = "VeryLazy",
   dependencies = {
     "nvim-neotest/nvim-nio",
     "nvim-lua/plenary.nvim",
     "antoinemadec/FixCursorHold.nvim",
     "nvim-treesitter/nvim-treesitter",
     "jfpedroza/neotest-elixir",
   },
   config = function()
     require("neotest").setup({
       adapters = {
         require("neotest-elixir")
       },
     })
   end,
 }
)

vim.keymap.set("n", "<space>tra", function() require("neotest").run.run(vim.fn.getcwd()) end, { desc = "run all tests" })
vim.keymap.set("n", "<space>trf", function() require("neotest").run.run() end, { desc = "run file tests" })

Then, create a new phoenix project and run tests with the following commands :

mix phx.new helloworld
cd helloworld
mix deps.get
mix ecto.create # Should be optional
mix ecto.migrate # Should be optional
mix test # All tests should work
NVIM_APPNAME=nvim-elixir nvim # start neovim with the init.lua from earlier
:e test/helloworld_web/controllers/page_controller_test.exs # just to open a file with tests
:Neotest summary # to see all tests
:lua require("neotest").run.run(vim.fn.getcwd()) # all tests are failling
:lua require("neotest").run.run() # file tests are succesful

Notes

I tried to understand how elixir-tools.nvim works, but I was unable to find any clues of what could be the origin

The idea is to read the file that is used by ExUnit to write failed tests, so next time when you open summary of tests, you can already see failed tests even though they were run from the CLI.
https://github.com/elixir-lang/elixir/blob/927b10df80ee1c1c7396e68efe00d06bc3e80420/lib/mix/lib/mix/tasks/test.ex#L751-L767

In the example test file below, "test 1" is detected, but "test 2" is not. As soon as you convert "test 2" to a full do/end block, it then is detected. Convert "test 1" to a do: block, and it is no longer detected.

describe "my tests" do
  test "test 1" do
    assert true
  end

  test "test 2", do: assert false
end

#17 (comment)

Hi! I am trying to run my tests in a docker container, however I am getting these errors because it can't find the formatter.ex and json_encoder.ex in my container:

-r : No files matched pattern /Users/steve/.local/share/nvim/lazy/neotest-elixir/neotest_elixir/json_encoder.ex
-r : No files matched pattern /Users/steve/.local/share/nvim/lazy/neotest-elixir/neotest_elixir/formatter.ex

Is there a way to override these paths somehow so I can mount the files and point to the new location in the container?

Here is how I am setting it up:

require("neotest").setup({
  log_level = vim.log.levels.DEBUG,
  adapters = {
    require("neotest-elixir")({
      post_process_command = function(cmd)
        return vim.tbl_flatten({
          { "docker", "compose",     "exec", "-i" },
          { "-w",     "/app" },
          { "-e",     "MIX_ENV=test" },
          { "app" },
          cmd })
      end,
    }),
  },
})

Or do you have a better recommendation for how to run these in docker instead?

Hi 👋

Firstly, thank you so much for this plugin. It's what made me switch over from vim-test to neotest. 💙

Not sure if this question is a better fit here or in the neotest repo, but I don't see the output of IO.inspect (or IO.puts FWIW) on the test output window when some assertion fails. Maybe I have the wrong expectations for the output? Missing a configuration?

Hello

Thank you for this awesome adapter.
I'm trying to integrate it into my workflow and here is the error I've found.
Steps to reproduce:

• install elixir, erlang and asdf

• $ asdf global elixir system
  

• $ asdf global erlang system
  

• try to run some test using neotest plugin
• the test is failed and output is the following

** (SyntaxError) /home/artur/.asdf/shims/mix:3:18: syntax error before: '.'
    |
  3 | exec /home/artur/.asdf/bin/asdf exec "mix" "$@" # asdf_allow: ' asdf '
    |                  ^
    (elixir 1.14.0) lib/code.ex:1245: Code.require_file/2

everything is OK if I specify elixir version from asdf and not the system one.

We have a quite huge testsuite, which generates several tests from given metadata.

The testfiles are marked with a "not ran" or "skipped" symbol (I guess, see screenshot) while the parent folder is marked as failed.

The testfiles do not contain much more than a use and a copule of module attributes, from which a test matrix is built and then the describe and test implementations are generated via macro magic.

Perhaps a mix test --trace can be used to get a more accurate list of tests or at least the parent is not marked as failed?

defmodule MyTest do
  use ExUnit.Case

  test inspect(&MyModule.hello/1) do
    # ...
  end
end

^Such tests are not found.
The same, but with interpolation, works okay:

  test "#{inspect(&MyModule.hello/1)}" do

When I run neotest.run.run({suite = true}), it seems that neotest-elixir is also trying to compile and run test files from my dependencies. In the case below, a failing compilation of a dependencies' test is causing the whole suite to break. I assume this is not expected behaviour...

Perhaps we can filter out all tests in mix_root/deps/. I'm not sure how complicated this is with umbrella apps and so forth.

Such tests are not recognized by the adapter:

  test ~s|text with quotes " and '| do

Some time ago, module A had doctests, but now it doesn't have them and doctest A instruction is still in tests.
When I run tests for the whole file, then everything is green.
If I rerun test only for doctest, then it becomes red. Output is the following:

Excluding tags: [:test]
Including tags: [line: "4"]

All tests have been excluded.

Finished in 0.03 seconds (0.03s async, 0.00s sync)
23 tests, 0 failures, 23 excluded

Randomized with seed 487001

If I rerun tests for the whole file, then everything is green again.

You may ask why I rerun tests for doctests :) If another test in that file fails, and I run tests for the whole file, then doctest becomes red as well! I try to rerun doctests and consistently see it as red in neovim.

Expected behaviour: consider doctest instructions without tests as successful.

I noticed, that if I have opened an umbrella project, and I'm in the root of it, then tests are launched using not full paths.
Let's say we have the following structure:

apps/
  foo/
    test/
      foo_test.exs
  bar/
    test/
      bar_test.exs
  ... # other projects

and when I ask to run tests for foo_test.exs, then the following command is executed:

mix test test/foo_test.exs

and mix tries to find the file in all the projects and reports these messages for every umbrella project:

Paths given to "mix test" did not match any directory/file: test/foo_test.exs
==> bar

it would be great if for umbrella projects the following command is executed:

mix apps/foo/test/foo_test.exs

Such approach would increase the speed of running tests.

I'm trying out neotest but I always get the "No tests found" response when run with run(). If I try to use:

require("neotest").run.run(vim.fn.expand("%"))

The tests gets a "spinner" icon in the summary split, but no tests are run and nothing is printed in the output panel.

I've tried with a fresh project from mix new with a file like this:

defmodule ExNewTest do
  use ExUnit.Case
  doctest ExNew

  test "greets the world" do
    assert ExNew.hello() == :world
  end
end

The neotest rust adapter works well. I've tried to update the treesitter grammar and tried various elixir LSPs and now I'm a bit lost.

> nvim --version
NVIM v0.10.0-dev-3135+g7acf39dda
Build type: Debug
LuaJIT 2.1.1713484068

Hey 👋

rocks.nvim has the ability to install luarocks packages and automatically install dependencies, including tree-sitter parsers.

I was wondering: Would you be open to automating releases and publishing this plugin to luarocks.org?
See also our wiki page on plugin support.

I'd be more than happy to help if you have any questions or issues.

Due to project setup constraints, I'm unable to run tests from the project root on my umbrella project, I have to cd to apps/sub_app and then run the test. I wonder if there's a way to configure the adapter in a way that it runs tests from the umbrella app root.

I tried to do that by modifying post_process_command to run the test through mix cmd --app my_app test path/to/test.exs. Here's my version of the run command:

elixir -r /Users/piacsek/.local/share/nvim/lazy/neotest-elixir/neotest_elixir/json_encoder.ex -r /Users/piacsek/.local/share/nvim/lazy/neotest-elixir/neotest_elixir/formatter.ex -S mix cmd --app my_app mix test --formatter NeotestElixir.Formatter --formatter ExUnit.CLIFormatter path/to/my/test.exs

But I when I run this command, the following output appears:

09:57:37.361 [error] CRASH REPORT Process <0.1422.0> with 0 neighbours crashed with reason: call to undefined function 'Elixir.NeotestElixir.Formatter':init([{include,[]},{exclude,[external,es_integration]},{max_cases,16},{seed,178117},{failures_manifest_file,...},...])
.....
Finished in 0.07 seconds (0.07s async, 0.00s sync)
5 tests, 0 failures

The tests pass, but the formatter fails to parse the results, making neotest consider the test as a failure.

PS: I'm currently trying to alter my project in a way that I can run any test from the root of the project, but having that as a configuration could come in handy not just to me but to other people in a similar situation.

Hey there, thanks for the great plugin!

Bit in a hurry right now, so just wanted to let you know that it seems that doctests are not fully supported (not listed in test summary, no indicator in signs?)
Let me know if I can help with this!

Creating an issue from the TODO item to have discussion

I explored if using the ETF format could be an option. There exists a lua parser for it. But sadly this seems to make use of lua 5.3 constructs which is very hard to get it working on neovim (lua 5.1). More specifically, the parser uses

• bit shift operators << which seems to be supported under a different module / syntax ref
• string.unpack function for which all the existing options falls short in different ways
  • lua-struct does not return the bytes read as the second return value. Based on an issue comment on the repo, it seems like a simple change to return it.
  • lua-pack is not a drop-in replacement. returns values in base-2 formats 🤷🏻
  • lua-compat-5.3. This is possibly the only option that seemed to work, but it messes up with global values and that breaks somewhere in plenary code

Curious if there are any other ways / options to achive this

Hi,

I am using Lazy to load plugins into NVIM and I am getting no feedback when running

lua require("neotest").run.run()

Here is my NVIM config for testing. I had previously installed vim-test, but then saw the developer was moving to a new project, so I installed that instead.

return {
  {
    -- wrapper for running tests of different granularities   You • Mon 22 May 09:55:56 2023 • comments and removes modules
    -- 'vim-test/vim-test',
    {
      'nvim-neotest/neotest',
      dependencies = {
        'nvim-lua/plenary.nvim',
        'nvim-treesitter/nvim-treesitter',
        'antoinemadec/FixCursorHold.nvim',
        'jfpedroza/neotest-elixir'
      },
      config = function()
        require('neotest').setup({
          adapters = {
            require('neotest-elixir')
          }
        })
      end
    },
    'jfpedroza/neotest-elixir',
  }
}

When I attempt to run the tests, it does nothing and I see no running tests. I am running the tests inside a vanilla scaffolded phoenix project.

Should I see a popup on the screen with test results?

Not sure why. I have a multiline describe do.. block. I attach a screenshot.

It also succeeds in the console

Here's my setup:

require("neotest").setup({
  adapters = {
    require("neotest-elixir")({}),
  }
})

vim.keymap.set('n', '<leader>tr', function() require('neotest').run.run() end)
vim.keymap.set('n', '<leader>ta', function() require("neotest").run.run(vim.fn.expand("%")) end)
vim.keymap.set('n', '<leader>td', function() require("neotest").run.run({strategy = "dap"}) end)
vim.keymap.set('n', '<leader>tp', function() require("neotest").output_panel.toggle() end)

First of all, thank you very much for your work, which has given me a new experience in testing elixir in nvim. But I don't know if you have the same feeling that doing Unit Testing in elixir is much slower than in other languages, I know it's not caused by neotest or Lua.

The main reason is that every time you run a mix test you need to start the whole app, whereas if we just run the test in iex, it would be much faster.

Here is a project I generated with mix new elixir_demo, you can see that even the simplest project that is brand new takes 1.5 seconds to run each test using the command line, whereas with iex it just takes 0.015 seconds, which is a 100x improvement, and if we could do that, then developing elixir projects in Nvim would be a superb experience.

References:

• ExUnit tests from a module instead of command line/mix - Questions / Help - Elixir Programming Language Forum
  https://elixirforum.com/t/exunit-tests-from-a-module-instead-of-command-line-mix/22485
• TestIex — Easier Test Driven Development in Elixir | by Daniel Olshansky | Medium
  https://olshansky.medium.com/testiex-easier-test-driven-development-in-elixir-761cd1a9f11
• How to speed up the start of mix test? - Questions / Help - Elixir Programming Language Forum
  https://elixirforum.com/t/how-to-speed-up-the-start-of-mix-test/43498/15