rrthomas / beetle

A Forth-oriented VM and debugger

Beetle

by Reuben Thomas [email protected]
https://github.com/rrthomas/beetle

Beetle is a simple virtual machine designed for the Forth language. It uses a byte-stream code designed for efficient execution which is binary portable between implementations. It has been implemented in C (for POSIX systems) and hand-optimised assembler (for ARM). The C implementation should run on any POSIX system; the assembler version runs pForth (see below) at up to half the speed of the corresponding native code compiler and generates more compact code. Beetle is designed to be embedded in other programs; a simple shell has been written to demonstrate this ability. In the C implementation, all memory references are bounds checked. An I/O library is implemented; access to native code routines is also possible, allowing Beetle and C programs to call each other.

This package comprises the definition of the Beetle virtual machine and an implementation in ISO C99 using POSIX APIs. Detailed documentation is in the doc directory; installation instructions follow.

The package is distributed under the GNU General Public License version 3, or, at your option, any later version.

THIS PROGRAM IS PROVIDED AS IS, WITH NO WARRANTY. USE IS AT THE USER’S RISK.

Installation and compatibility

Beetle should work on any POSIX-1.2001-compatible system. Beetle has been tested on x86_64 GNU/Linux with GNU C.

Previous releases were known to work on Acorn RISC OS 3, Digital UNIX V3.2, UNIX System V Release 4.0, ULTRIX 4.3, NetBSD 1.2, MSDOS 6, and Atari TOS 1.4.

Reports on compatibility, whether positive or negative, are welcomed.

Building from a release tarball

Perl and help2man are required to build from source. For building from git, see below.

To build Beetle from a release tarball, run

./configure && make && make check

For the bibliographies in the documentation to be built correctly, GNU Make should be used.

Building Beetle from git

The GNU autotools are required: automake, autoconf and libtool. Gnulib is also used, with a third-party bootstrap module; these are installed automatically.

To build from a Git repository, first run

./bootstrap

Then see "Building from source" above.

To build the PDF documentation, a comprehensive TeX system such as TeXLive is required. This is only necessary when building from Git, as pre-built PDFs are supplied in release archives.

Use

Run beetle (see beetle --help and shell.pdf for documentation). If you have rlwrap, you can run beetlei instead to get readline support.

Demo: Hello, world!

In tests/hello.txt is a command file for the shell that demonstrates its use as a crude assembler. Run the following commands to see it in action:

cd tests
beetle < ./hello.txt
beetle hello.obj

Documentation

The canonical documentation consists of:

• The Beetle Forth Virtual Machine
  The design of the Beetle Forth virtual machine is described. Essential reading for those programming or implementing the VM.
• An implementation of the Beetle virtual machine for POSIX
  A portable implementation of Beetle is described, with instructions for porting, compiling and running it.
• A simple shell for the Beetle virtual machine
  The user guide for Beetle’s shell.

The following documents contain extra material on Beetle’s design, but many details are out of date:

• An Introduction to the Beetle Forth Virtual Processor
  An introduction to the system; this is the best paper to read first. It was published in ACM SIGPLAN Notices February 1997.
• Beetle and pForth: a Forth virtual machine and compiler
  I developed Beetle for my BA dissertation project. (I used to refer to it as a “virtual processor”; I now use the now-standard term “virtual machine”.) My BA dissertation contains older versions of all the papers mentioned above, as well as a description of the project that produced them.
• Tradeoffs in the implementation of the Beetle virtual machine
  A hand-coded implementation of Beetle is described, and compared to the C version.
• Encoding literals in a portable byte-stream interpreter
  Various methods of encoding literal numbers in a byte stream are compared.

pForth

pForth is an ANSI Forth compiler that targets Beetle.

Running Beetle object files

The C implementation of Beetle allows a hash-bang line to be prepended to an object file, so that they can be run directly. A suggested line is:

#!/usr/bin/env beetle

A magic file for the file(1) command is also provided: beetle.magic. This file should be part of file >= 5.33.

Hand-written ARM assembler version

ARMbeetle.bas contains a hand-written ARM assembler version of Beetle, written in the BBC BASIC assembler (for RISC OS).

Bugs and comments

Please send bug reports (preferably as GitHub issues) and comments. I’m especially interested to know of portability bugs.

beetle's Issues

Remove `@PACKAGE@` where possible

The name is fixed, and the code is no longer feeding into SMite/Mit.

(+LOOP) should use the carry or overflow flag, not the sign bit

Add [increment] to [index] ; if [index] thereby crosses the [limit] - 1 to [limit] boundary, [...]

It's not clear what this means unless we know whether to interpret increment as signed or unsigned. For example, if increment is 1<<31, it could be interpreted as positive (unsigned) or negative (signed). Suppose index and limit are, say 5 and 0 respectively. If increment is interpreted as positive, then it does not cross the boundary, but if negative it does.

I think the fix is to specify that increment is always signed. It is not necessary to specify whether index and limit are signed or unsigned; just use cyclic arithmetic.

Use mijit-beetle when available

Surprisingly, although it does no optimization, mijit-beetle already makes beetle go faster. Time to run:

$BEETLE pforth make.fs

Beetle 2.1.1	Beetle 'master' branch	Beetle 'mijit' branch
6.84s	6.68s	4.71s
-0%	-2%	-31%

Remove memory manager

A nice idea, but really inefficient, and not really in tune with Beetle's mostly brutally-simple design.

A practical reason to remove it is that I'd like Beetle to be as fast as it reasonably can be, for comparison with SMite, and the current implementation of the memory manager slows down all virtual memory access horribly. (Given which it's impressive how much faster than SMite Beetle runs current pForth!)

Fails to link with mijit-beetle on aarch64

See rrthomas/mijit-beetle#3

Division of -2^31 by -1

Should raise an error.

It is not necessary to execute NEXT when beetle starts up.

"beetle.pdf", section 2.4 "Operation" says that A is set to 0 and then NEXT is executed. The latter is redundant, and could be deleted; if omitted the first call to single_step() or run() will execute NEXT anyway, because A is 0.

This simplification would be an observable change of the specification; the first call to single_step() will now execute "NEXT" where previously it would have executed the first instruction.

Stacks can grow in either direction

The Beetle spec currently says "downwards".

Add VM "core dumps"

On uncaught memory exceptions, dump object file that can be loaded into debugger.

libbeetle should depend on c-decls.h

If I edit the file mijit-beetle/c-decls.h then rebuild beetle, the build is incomplete. Please consider adding an extra dependency to the Makefile.

add explicit license

Would be nice to add explicit license instead of the symlink to ancient automake version. Also it would clarify confusion: is it GPLv3? or v2? or v2+? The automake has upgraded it's license which is a great thing but I'm not sure if it's applicable to this project as well.