Source: https://github.com/mbland/go-script-bash
A ./go script aims to abstract away many of the steps needed to develop (and
sometimes deploy) a software project. It is a replacement for READMEs and other
documents that may become out-of-date, and when maintained properly, should
provide a cohesive and discoverable interface for common project tasks.
For a five-minute overview of the framework, see Mike Bland's go-script-bash lightning talk at Surge 2016.
- Introduction
- Environment setup
- How to use this framework
- Feedback and contributions
- Installing Bash
- Open Source
- Prior work
The ./go script idea came from Pete Hodgson's blog posts In Praise of the
./go Script: Part I and Part II. To paraphrase Pete's
original idea, rather than dump project setup, development, testing, and
installation/deployment commands into a README that tends to get stale, or
rely on oral tradition to transmit project maintenance knowledge, automate these
tasks by encapsulating them all inside a single script in the root directory of
your project source tree, conventionally named "go". Then the interface to
these tasks becomes something like ./go setup, ./go test, and ./go deploy.
Not only would this script save time for people already familiar with the
project, but it smooths the learning curve, prevents common mistakes, and lowers
friction for new contributors. This is as desirable a state for Open Source
projects as it is for internal ones.
No. The ./go script convention in general and this framework in particular are
completely unrelated to the Go programming language. In fact, the
actual ./go script can be named anything. However, the go command from the
Go language distribution encapsulates many common project functions
in a similar fashion.
Of course, the danger is that this ./go script may become as unwieldy as the
README it's intended to replace, depending on the project's complexity. Even
if it's heavily used and kept up-to-date, maintenance may become an intensive,
frightening chore, especially if not covered by automated tests. Knowing what
the script does, why it does it, and how to run it may become more and more
challenging—resulting in the same friction, confusion, and fear the script was
trying to avoid.
The ./go script framework makes it easy to provide a uniform and easy-to-use
project maintenance interface that fits your project perfectly regardless of the
mix of tools and languages, then it gets out of the way as fast as possible. The
hope is that by making the right thing the easy thing,
scripts using the framework will evolve and stay healthy along with the rest of
your project sources, which makes everyone working with the code less frustrated
and more productive all-around.
This framework accomplishes this by:
- encouraging modular, composable
./gocommands implemented as individual scripts—in the language of your choice! - providing a set of builtin utility commands and shell command aliases—see
./go help builtinsand./go help aliases - supporting automatic tab-completion of commands and arguments through a
lightweight API—see
./go help envand./go help complete - implementing a quick, flexible, robust, and convenient documentation
system—document your script in the header, and help shows up automatically as
./go help my-command! See./go help help.
Plus, its own tests serve as a model for testing command scripts of all shapes and sizes.
The inspiration for this model (and initial implementation hints) came from Sam
Stephenson's rbenv Ruby version manager.
It's the ultimate backstage pass! It's the default shell for most mainstream UNIX-based operating systems, easily installed on other UNIX-based operating systems, and is readily available even on Windows.
Yes. It is an explicit goal to make it as easy to use the framework on Windows as possible. Since Git for Windows in particular ships with Bash as part of its environment, and Bash is available within Windows 10 as part of the Windows Subsystem for Linux (Ubuntu on Windows), it's more likely than not that Bash is already available on a Windows developer's system. It's also available from the MSYS2 and Cygwin environments.
Of course there are many common tools that may be used for managing project
tasks. For example: Make, Rake, npm, Gulp, Grunt, Bazel,
and the Go programming language's go tool. There are certainly more powerful
scripting languages: Perl, Python, Ruby, and even Node.js
is a possibility. There are even more powerful shells, such as the
Z-Shell and the fish shell.
The ./go script framework isn't intended to replace all those other tools and
languages, but to make it easier to use each of them for what they're good for.
It makes it easier to write good, testable, maintainable, and extensible shell
scripts so you don't have to push any of those other tools beyond their natural
limits.
Bash scripting is really good for automating a lot of traditional command line tasks, and it can be pretty awkward to achieve the same effect using other tools—especially if your project uses a mix of languages, where using a tool common to one language environment to automate tasks in another can get weird. (Which is part of the reason why there are so many build tools tailored to different languages in the first place, to say nothing of the different languages themselves.)
If you want to incorporate different scripting languages or shells into your
project maintenance, this framework makes it easy to do so. However, by starting
with Bash, you can implement a ./go init command to check that these other
languages or shells are installed and either install them automatically or
prompt the user on how to do so. Since Bash is (almost certainly) already
present, users can run your ./go script right away and get the setup or hints
that they need, rather than wading through system requirements and documentation
before being able to do anything.
Even if ./go init tells the user "go to this website and install this other
thing", that's still an immediate, tactile experience that triggers a reward
response and invites further exploration. (Think of Zork and the first
"open mailbox" command.)
The real question is: Where can't you run it?
The core framework is written 100% in Bash and it's been tested under Bash 3.2, 4.2, 4.3, and 4.4 across OS X, Ubuntu Linux, Arch Linux, Alpine Linux, FreeBSD 9.3, FreeBSD 10.3, and Windows 10 (using all the environments described in the "Will this work on Windows?" section above).
Actually, yes. See the Standalone mode section below.
Also see the following question...
Yes. You can share one copy of the go-bash-framework sources, and even have
common code in the lib/ directory, but set each script to use its own command
scripts dir.
This may be especially useful if you're writing a standalone program, in which one script provides the actual program interface, and the other provides the development-only interface.
The project's own ./go test command does it all. Combined with automatic
tab-completion enabled by ./go env and pattern-matching via ./go glob, the
./go test command provides a convenient means of selecting subsets of test
cases while focusing on a particular piece of behavior. (See ./go help test.)
The tests are written using mbland/bats, an optimized version of Sam
Stephenson's Bash Automated Testing System (BATS). Code coverage
comes from Simon Kagstrom's kcov code coverage tool, which not only
provides code coverage for Bash scripts (!!!) but can push the results to
Coveralls!
To run a ./go script that uses this module, or to add it to your own project,
you must have Bash version 3.2 or greater installed on your
system. Run bash --version to make sure Bash is in your PATH and is a
compatible version. You should see output like this:
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin15)
Copyright (C) 2007 Free Software Foundation, Inc.
If you do not see this, follow the instructions in the Installing Bash section later in this document.
Note: While Bash is required to run this framework, your individual command scripts can be in any other interpreted language installed on the host system.
First you'll need a copy of this framework available in your project sources.
The most expedient way to bootstrap your program is to use the go-template
file as a starting point (replacing curl with wget,
fetch, or whichever tool you prefer):
$ curl https://raw.githubusercontent.com/mbland/go-script-bash/master/go-template >./go
$ chmod ugo+rx ./goYou may rename this file whatever you wish (i.e. it doesn't have to be named
./go), update its documentation and variables to fit your project, and check
it into your project repository. See the go-template comments for details.
If you'd prefer to download a copy of the framework and check it into your sources, versioned archives are available from the go-script-bash Releases page. The archives for the current release are:
- https://github.com/mbland/go-script-bash/archive/v1.7.0.tar.gz
- https://github.com/mbland/go-script-bash/archive/v1.7.0.zip
You can also add this repository to your project as a Git submodule:
$ git submodule add https://github.com/mbland/go-script-bash <target-dir>
$ git commit -m 'Add go-script-bash framework'
$ git submodule update --initwhere <target-dir> is any point inside your project directory structure that
you prefer.
If you're not using go-template, create a bash script in the root directory of
your project to act as the main ./go script. This script need not be named
go, but it must contain the following lines, with @go "$@" as the last line
of the script:
. "${0%/*}/go-core.bash" "scripts"
@go "$@"where:
${0%/*}produces the path to the project's root directory based on the path to the./goscript${0%/*}/go-core.bashproduces the path to the framework'sgo-core.bashfile within your project's copy of the framework (adjusted to reflect where your copy ofgo-script-bashactually resides)scriptsis the path to the directory holding your project's command scripts relative to the project root (it can be any name you like)
The ./go script changes to the project root directory before executing any
commands. That means every command script you write will also run within the
project root directory, so every relative file and directory path will be
interpreted as relative to the project root.
Your project structure may look something like this:
project-root/
go - main ./go script
lib/ - publicly-exported modules (if the project is a go-bash-script plugin)
scripts/ (or bin/) - project (or plugin) ./go command scripts
lib/ - project-specific Bash library modules (see "Modules" section)
plugins/ - (optional) third-party command scripts (see `./go help plugins`)
.../
bin/ - plugin ./go command scripts
lib/ - publicly-exported Bash library modules (see "Modules" section)
go-script-bash/
go-core.bash - top-level functions
lib/ - publicly-exported Bash library modules (see "Modules" section)
libexec/ - builtin ./go command scripts
This structure implies that the first line of your ./go script will be:
. "${0%/*}/scripts/go-script-bash/go-core.bash" "scripts"The following variables are set by the framework based on the above example
(note there are many other variables set in go-core.bash and elsewhere; see
./go help vars):
_GO_ROOTDIR:/absolute/path/to/project-root_GO_CORE_DIR:/absolute/path/to/project-root/scripts/go-script-bash_GO_SCRIPTS_DIR:$_GO_ROOTDIR/scripts_GO_PLUGINS_DIR:/absolute/path/to/project-root/plugins
For plugins, _GO_ROOTDIR and _GO_SCRIPTS_DIR will be scoped to the root
directory of the plugin installation; the other variables will remain the same.
See ./go help plugins for more details.
Each command script for your project residing in the scripts directory must
adhere to the following conditions:
- No filename extensions.
- It must be executable, with a
#!(a.k.a. "she-bang") line. The interpreter name will be parsed from this line, whether it is an absolute path (#!/bin/bash) or is of the form:#!/usr/bin/env bash. - If
scripts/parentis a command script, subcommand scripts must reside within a directory named:scripts/parent.d.
Scripts can use any interpreted language available on the host system; they
need not be written in Bash. Bash scripts will be sourced (i.e. imported into
the same process running the ./go script itself). Other languages will use the
PATH environment variable to discover the interpreter for the script.
See ./go help commands for details on the algorithm used to discover command
scripts for execution.
The builtin ./go help command will parse command script summaries and help
text from the header comment block of each script. Run ./go help help to learn
more about the formatting rules.
By evaluating the value of ./go env - within your shell, all builtin commands
and aliases provide automatic tab completion of file, directory, and other
arguments. If an implementation isn't available for your shell (within
lib/internal/env/), it's very easy to add one. Feel free to open an issue or,
better yet, send a pull request!
To learn the API for adding tab completion to your own command scripts, run
./go help complete. You can also learn by reading the scripts for the builtin
commands.
If you wish to use the framework to write a standalone program, rather than a
project-specific development script, set _GO_STANDALONE in your top-level
script to prevent alias commands, builtin commands, and plugin commands from
showing up in help output or from being offered as tab completions. (help
will still appear as a top-level tab completion.) All of these commands will
still be available, but users won't be presented with them directly.
_GO_STANDALONE also prevents the script from setting PWD to _GO_ROOTDIR,
enabling the script to process relative file path arguments anywhere in the file
system. Note that then you'll have to add _GO_ROOTDIR manually to any
_GO_ROOTDIR-relative paths in your own scripts.
There are a number of possible methods available for sharing code between command scripts. Some possibilities are:
- The generally preferred method is to use
. $_GO_USE_MODULESto source optional library modules; see the Modules section. - Include common code and constants in the top-level
./goscript, after sourcinggo-core.bashand before calling@go. - Source a file in the same directory that isn't executable.
- Source a file in a child directory that may not have a name of the form:
parent.d. - Source files from a dedicated directory relative to
$_GO_ROOTDIR, e.g.:. "path/to/lib/common.sh"
- Subcommand scripts can source the parent command via:
. "${BASH_SOURCE[0]%.d/*}"
Any script in any language can invoke other command scripts by running
./go <command> [args..]. In Bash, however, you can also invoke the @go
function directly as @go <command> [args...].
The @go, @go.printf, and @go.print_stack_trace functions are available to
command scripts written in Bash, as Bash command scripts are sourced rather than
run using another language interpreter.
A number of global variables defined and documented in go-core.bash, all
starting with the prefix _GO_, are exported as environment variables and
available to scripts in all languages (along with the global COLUMNS
environment variable). Run ./go vars to see them all along with their values,
and run ./go help vars for more details.
You can add third-party plugin command scripts to the plugins subdirectory of
your scripts directory. Run ./go help plugins for more information.
You can import optional Bash library code from the core framework, third-party
plugins, or your own project's scripts directory by sourcing the
_GO_USE_MODULES script. For example, to import the core logging utilities:
. "$_GO_USE_MODULES" 'log'Run ./go help modules and ./go modules --help for more information.
The core library log module provides functions for standard logging
facilities. For example:
@go.log INFO Hello, World!
@go.log ERROR Goodbye, World!For more information, run ./go modules --help log.
The assertions and helpers from the test suite have been extracted into the
lib/bats libraries. While these are not modules you can import with
_GO_USE_MODULES, they are completely independent of the rest of the core
framework and you may source them in your own Bats tests. (Whether or not these
will ever become a separate library remains an open question.)
Variables, helper functions, and assertions for testing features based on the
core framework are available in the lib/testing directory. The lib/bats-main
library makes it easy to write a ./go test command script with the same
interface and features as the core framework's ./go test command.
Read the comments from each file for more information.
The kcov-ubuntu module provides the run_kcov function that will download and
compile kcov, then run kcov with the original ./go command line
arguments to collect test coverage. Only available on Ubuntu Linux for now,
hence the name. Run ./go modules --help kcov-ubuntu for more information and
see scripts/test for an example of how it may be used.
Feel free to comment on or file a new GitHub issue or otherwise ping @mbland with any questions or comments you may have, especially if the current documentation hasn't addressed your needs.
If you'd care to contribute to this project, be it code fixes, documentation updates, or new features, please read the CONTRIBUTING file.
If you're using a flavor of UNIX (e.g. Linux, OS X), you likely already have a suitable version of Bash already installed and available. If not, use your system's package manager to install it.
On Windows, the Git for Windows, MSYS2 and Cygwin distributions all ship with a version of Bash. On Windows 10, you can also use the Windows Subsystem for Linux.
Once you've installed bash, your PATH environment variable must include
its installation directory. On UNIX, you can add it in the appropriate
initialization file for your shell; look up your shell documentation for details.
On Windows, in most cases, you'll use the terminal program that ships with Git
for Windows, MSYS2, or Cygwin, or you'll invoke the Windows System for Linux
environment by entering bash in a built-in Command Prompt window. These
terminals automatically set PATH so that Bash is available.
However, if you want to use the Git, MSYS2, or Cygwin bash from the built-in
Command Prompt window, open the Start menu and navigate to Windows
System > Control Panel > System and Security > System > Advanced system
settings. Click the Environment Variables... button, select PATH, and
add the directory containing your bash installation. The likely paths for each
environment are:
- Git:
C:\Program Files\Git\usr\bin\ - MSYS2:
C:\msys64\usr\bin\ - Cygwin:
C:\cygwin64\bin\
To use one of these paths temporarily within a Command Prompt window, you can run the following:
C:\path\to\my\go-script-bash> set PATH=C:\Program Files\Git\usr\bin\;%PATH%
# To verify:
C:\path\to\my\go-script-bash> echo %PATH%
C:\path\to\my\go-script-bash> where bash
# To run the tests:
C:\path\to\my\go-script-bash> bash ./go test
It should not be necessary to set Bash as your default shell. On Windows,
however, you may wish to execute the bash command to run it as your shell
before executing the ./go script or any other Bash scripts, to avoid having to
run it as bash ./go every time.
Most of the framework as-is does not require any other external tools. However, in order for the automatic command help and output formatting to work, you'll need the following utilities installed:
tput(ncurses) on Linux, OS X, UNIXmode.comshould be present on Windows
To use the get file builtin, either curl, wget, or fetch must be
installed on your system. get git-repo requires git, naturally.
This software is made available as Open Source software under the ISC License. For the text of the license, see the LICENSE file.
This is a Bash-based alternative to the 18F/go_script Ruby implementation.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent