snogge / emake.el

Test Elisp without the hoops

Emacs-Make

Test Elisp with services like Travis CI without the fuss of Cask – just you, your project, and (Emacs-)Make.

Things EMake does:

• parses, installs, and runs tests for your package
• provides all the power of Elisp to extend its capabilities on-demand

Things EMake will never do (or ‘reasons you may still need Cask’):

• manage your development environment or provide tools to do so
• provide ’bundler-like’ exec abilities (this includes Cask’s emacs and eval commands)

Why use EMake?

EMake maintains a tight focus on continuous testing – there’s nothing included in the base script deemed unnecessary for this purpose. This focus removes complexity that can cause false failures in your testing (such as interactions with CI images, Python incompatibilities, etc.). This reduction of complexity naturally leads to a more stable build.

Free of dependencies, EMake is faster to install. In my own projects, switching from Cask to EMake reduced build time by 90 seconds. While we surely shouldn’t be too concerned about some faceless VM’s time, we should be good stewards of the resources freely provided to us.

Because it relies only on existing tools, EMake is more pliable than Cask (at least, more obviously so). Want to use an environment variable to test MELPA-Stable separate from MELPA in your testing matrix? Just configure an environment variable in the matrix itself (or use any of Make’s facilities for more complicated logic). Want to use an external testing framework? Throw it in emake-test-runner-master-alist with some Elisp.

Anything you want to – do it. Want to change the world? There’s nothing to it.

Using EMake

A practical demonstration is available in the example submodule which is also hosted on GitHub and hooked up to Travis.

I recommend specifying the commit you want to use to prevent issues with builds broken by upstream changes. If a certain version works for you, there is no added value to receiving further updates until you specifically want them. It may be dangerous to use =master=! While it should always be stable, it may introduce backwards-incompatible changes.

For example:

EMAKE_SHA1=af25a9cd958d77d5126d62e8855a30097bea0d47
$(CURL) -O https://raw.githubusercontent.com/vermiculus/emake.el/$(EMAKE_SHA1)/emake.el

EMake itself is driven by a few environment variables:

PACKAGE_FILE

This is the Elisp file that contains the definition of your package (e.g., Author, Package-Version, Keywords, etc.)

PACKAGE_TESTS

This contains a space-delimited list of Elisp files to load before running tests. The files are loaded in the order they’re provided.

PACKAGE_LISP

This contains a space-delimited list of files to be considered part of the package.

PACKAGE_ARCHIVES

This contains a space-delimited list of package.el archives to use for resolving dependencies.

If your test suite has extra dependencies that shouldn’t be proper dependencies of the project as a whole, you can tell EMake what to do by configuring the following environment variables:

PACKAGE_TEST_DEPS

This contains a space-delimited list of package-names your test suite is dependent upon.

PACKAGE_TEST_ARCHIVES

These archives will be used to install the dependencies in PACKAGE_TEST_DEPS (and their dependencies, …). If not specified, PACKAGE_ARCHIVES will be used for this as well.

The entry point to EMake is the function emake and is intended to be invoked as follows:

emacs -batch -l emake.el --eval "(emake (pop argv))" target [args...]

Since this is just Elisp, other setup can be made by just evaluating some lisp in this invocation or loading an external file. You might want to set byte-compile-error-on-warn, for example, or maybe define new testing frameworks by extending emake-test-runner-master-alist. It’s just Elisp – no funny business!

To provide extra information, you can use EMAKE_DEBUG_FLAGS, which see.

EMake.mk

Since most package development is pretty similar across projects, EMake includes in its distribution a file called emake.mk. By downloading this file instead, you get instant access to the available targets (setup, compile, and test) and support for both the ERT and Buttercup testing frameworks as well as checkdoc and package-lint. All you have to do is set EMAKE_SHA1 and the PACKAGE_BASENAME variables. For example, a one-file package by the name of coffee-table.el would use PACKAGE_BASENAME=coffee-table. More complex environment setup (e.g., use of a coffee-table-pkg.el file) can be configured using the standard variables above.

EMAKE_SHA1 should be the SHA-1 of the commit you wish to use for testing. This is to remove the possibility of EMake changes introducing bugs in your builds. I recommend taking the most recent SHA-1 of the repository (unless, perhaps, you find yourself unluckily in the middle of a push – just check the commit date).

EMACS_VERSION should be set in your ~/.profile. (If you’re using exec-path-from-shell, don’t forget to add it to exec-path-from-shell-variables if you want to run EMake from Emacs.)

See this project’s own Makefile for an example.

Default Targets

EMake comes with a few default targets to give it some out-of-the-box functionality.

install

Invoking $(EMAKE) install parses PACKAGE_FILE to install all its noted dependencies (in the Package-Requires header) from PACKAGE_ARCHIVES.

You can instruct EMake to ignore calculated dependencies using PACKAGE_IGNORE_DEPS.

compile

Invoking $(EMAKE) compile byte-compiles all files in PACKAGE_LISP. You can provide the optional argument ~error-on-warn to instruct the byte-compiler to error-out on compilation warnings (like unused local bindings or non-namespaced variables).

test

Invoking $(EMAKE) test kicks off the automated tests for your project. If you’re using a framework that can’t discover test definitions for you, you can define PACKAGE_TESTS to be the file (or files) to load those definitions from before running the tests.

You can specify which framework to use with an additional argument: $(EMAKE) test ert tests with ERT (the default) and $(EMAKE) test buttercup tests with Buttercup. Other frameworks may be defined in or added to emake-test-runner-master-alist.

help

Shows documentation for all Makefile targets.

help-*

Shows documentation for an Emake target. For example,

make help-compile
[...] emacs -batch -l emake.el [...] help compile
emake: Running target "help" with function `emake-help' with arguments ("compile")
emake: Documentation of compile (function emake-compile)...
Compile all files in PACKAGE_LISP.
Several OPTIONS are available:

‘~error-on-warn’: set ‘byte-compile-error-on-warn’

----

This target uses the following environment variables:

    PACKAGE_LISP: space-delimited list of Lisp files in this package

emake: Documentation of compile (function emake-compile)...done

Extending EMake

Targets can be created (or overridden) by defining a function using the emake-target property in its declare form before calling the emake function.

For example, if custom.el contains a custom target defined so:

(defun my-function ()
  (declare (emake-target "my-cake"))
  (message "Yum!"))

and you invoke EMake as:

cake:
        emacs -batch -l emake.el -l custom.el --eval "(emake (pop argv))" my-cake

and run make cake, my-function will be executed after some output boilerplate. See emake--resolve-target for more details.

You may find emake-with-elpa, emake-project-root, and emake-package-desc helpful (along with the package-desc- family of cl-struct accessors provided by package.el).

If your target is generalized and generally useful, consider contributing it to this repository!