Possibility to clump together tests from several files about doctest

[ag91]

Good point, I didn't need this yet because my extension are usually a single file.
I would go for a separate function for this, so your first option because this is a completely new feature.

Thinking about it, the design issue seems to be that doctest doesn't return data but just prints out something:
if you were to receive an output like '(:file "somepath/code.el" :pass 1 :fail 1 :errors ()),
your new use case would be simple to implement being just a view of the data.

[doublep]

Thinking about it, the design issue seems to be that doctest doesn't return data but just prints out something

Well, changing that would be backward-incompatible (even if there is unlikely to be any code depending on this). Also, as it is now, users can query the current test counts using the recently added doctest-state (or previously by accessing package-private variables). If this use scenario is to be retained, we need counts to stay as a global state, otherwise you wouldn't be able to query them after the fact, you'd have to use the return value. And this means there has to be a well-defined point where counts are reset to zero.

[ag91]

Okay, I think we can go with your first proposal to add a new function running a suite then.
I am not so concerned about backward-compatibility because we can always deprecate current functions,
but we don't need that yet if there are ways around that are nice as well :)

[doublep]

Sorry for long silence, don't have much time for Elisp stuff.

I now noticed a really weird piece of code:

(defun doctest--reset-state ()
  "Reset doctest's current state."
  (when (eq major-mode 'emacs-lisp-mode) (eval-buffer))
  ...

First, it is semantically a wrong place. Why does ...reset-state, of all things, evaluate the buffer? Second, I'm not sure it should be done at all, at least when non-interactive. Yeah, it provides all the function definitions which are used in the test expressions, but evaluating the whole buffer potentially has a lot of consequences...

Is it fine if the new function relies on the caller to provide all the function definitions (with this being mentioned in the documentation, of course)? For example, the caller could require necessary Elisp features, which may or may not be byte-compiled. In comparison, eval-buffer always results in non-byte-compiled definitions.

[ag91]

well spotted! That must have been a shortcut: it is done also for doctest-defun and also there seems unnecessary.
I suspect that for stateful code, you may find useful to reset all vars to their init value via eval-buffer but I would probably add some sort of defcustom to enable this unsafe behavior on demand.

That seems fine to me, I could provide different default functions that eval-buffer or evaluate only the minimum necessary code to make user's life easier.

[doublep]

Implemented in PR #8.