dymayday / holochain-rust

The core Holochain framework written in Rust, a container API for that (with rust and node implementations), and hdk-rust library for Zomes

Holochain-rust

Travis: Circle CI: Codecov: License:

This is the home of the Holochain Rust libraries.

This code is loosely based on the Golang prototype.

Code Status: Rust version is alpha. Not for production use. The code is guaranteed NOT secure. We will aggressively restructure code APIs and data chains until Beta.

Releases happen weekly.

Holochain Links:	FAQ	Developer Docs	White Paper

Overview

This holochain-rust repository implements a number of distinct yet overlapping aspects of the Holochain framework.

1. A library for the core Holochain functionality for defining and running DNA instances: core
2. A library and syntax for use in Rust based development of Zomes within DNAs, called Holochain Development Kit: hdk-rust
3. A library for managing instances and connecting them to interfaces: conductor-api
4. A Rust based Conductor that uses the conductor_api: conductor
5. A nodejs based Conductor for running tests: nodejs-conductor
6. A command line developer tool: hc
7. A sample application that we use to demonstrate the current functionality and drive development: app-spec

Core

The core folder contains the code that implements the core functionality of Holochain. It is the aspect that takes in an application DNA, and an agent, and "securely" (NOT secure during alpha) runs peer-to-peer applications by exposing the API functions to Zomes. It draws on other top level definitions and functions such as dna, cas_implementations, agent, and core_types.

HDK Rust

Holochain applications have been designed to consist at the low-level of WebAssembly (WASM) running in a virtual machine environment. Most languages will be easiest to use with some stub code to connect into the WASM runtime environment due to the nature of WASM. That is the main reason why a "Developer Kit" for a language exists. It is a library, and a syntax, for writing Zome code in that language.

hdk-rust is a solid reference implementation of this, that enables Zomes to be written in the Rust language (the same, somewhat confusingly, as Holochain Core).

Within this repository, some aspects cross over between core and hdk-rust, such as core_types, since they get stored into WASM memory in core, and then loaded from WASM memory, within hdk-rust. Related, wasm_utils is used on both sides to actually perform the storing, and loading, of values into and from WASM memory.

Other HDKs and language options

Any language that compiles to WASM and can serialize/deserialize JSON data can be available as an option for programmers to write Holochain applications.

An HDK for Assemblyscript is under experimental development at hdk-assemblyscript.

We expect many more languages to be added by the community, and there is even an article on how to write a kit for a new language.

Conductor API

Core only implements the logic for the execution of a single DNA. Because the Holochain app ecosystem relies on DNA composibility, we need to be able to load and instantiate multiple DNAs. We call an executable that can do this a Conductor. The first such Conductors we implemented were the Qt C++ GUI driven holosqape and the CLI driven hcshell Conductor which we used for running javascript based tests.

These gave us the experience from which we abstracted the conductor_api crate which specifies and implements a standard way for building conductors, including specifying the various interfaces that might be available for executing calls on a particular DNA, i.e. websockets, HTTP, Unix domain sockets, carrier pigeon network, etc...

If you need to implement your own conductor, conductor_api should provide you with the needed types and functions to do so easily.

To implement a conductor in a C based language, the core_api_c_binding [NEEDS UPDATING] code could be used, such as HoloSqape does.

Rust Conductor

The conductor crate uses the conductor_api to implement an executable which is intended to become the main, highly configurable and GUI less conductor implementation that can be run as a background system service.

Nodejs Conductor

The nodejs_conductor directory implements a node package that creates a conductor that wraps the Holochain core Rust implementation so we can access it from node. This is crucial especially for creating a test-driven development environment for developing Holochain DNA. The hc command-line tool relies on it to run tests.

HC Command-line developer tool.

The cli crate implements our command line developer tool which allows you to create DNA scaffold, run tests, and finally package your DNA for running in a Conductor. For more details see the crate README.

App Spec Driven Development

We use a practice for coordinating additions and features that starts with adding a feature to a sample application so that we know we have a working example all the times. You can read about the details here

Documentation: The Book on Holochain

There is a work-in-progress book of documentation being written about holochain-rust. See the published version at the associated GitHub Pages for this repo, https://developer.holochain.org/guide/latest. See instructions for how to contribute to the book at doc/holochain_101/src/how_to_contribute.md.

Installation & Usage

Important: the instructions in this readme are for developers intending work on Holochain code-base itself, not Holochain application developers. If you want to use Holochain, please proceed to the instructions on the quick start installation guide: https://developer.holochain.org/start.html

The following instructions are for developing Holochain Core or the HDK itself

There are two components needed currently to run Holochain applications, the core (what's in this repo) and also the networking engine. You can install and work on core using the built-in mock network following the instructions below, but if you want to actually test out your apps using the real networking, you will have to install the networking component following the instructions in the README there. (Note: please see the instructions in the guide book for hc or the production Conductor for how to configure the tools to use and activate the networking component.

There are three approaches to building and testing Holochain: using nix-shell, make, docker:

Nix Shell (Supported: Ubuntu, Debian, Mac OS X & Nix OS)

The nix-shell command from the nixos team is the preferred way to work with Holochain.

NixOS is an entire operating system but the nix-shell is simply a tool to manage dependencies for an individual shell session.

To install nix-shell:

# basic deps needed on ubuntu/debian
apt-get update && apt-get install -y curl bzip2

# this installs on all (non-windows) systems
curl https://nixos.org/nix/install | sh

Follow any further instructions output to the terminal during installation.

The website has more details.

Running the nix-shell command from inside the root of the repository will detect and use the shell.nix file.

The nix-shell approach offers unique benefits:

• Dependencies are injected into a single shell session only
  • Minimal modifications to the host environment
  • No need to maintain/rerun/troubleshoot installation scripts
  • Further isolation from host environment can be achieved with nix-shell --pure
• Dependencies are hashed
  • "Dependency hell" is avoided
  • Nice parallels with the hashed zomes model
  • Security + reliability benefits
• Dependencies can be garbage collected with the nix-collect-garbage command
• Single "package manager" across most operating systems
• Ability to ship utility scripts in the shell.nix file
• Access to the nix functional programming language for dependencies/script management
• NixOS runs on HoloPorts so nix-shell provides similar behaviour/environment

If you have a nix friendly system, this is the fastest and most reliable way to develop and test.

Once in a nix-shell the hc-* prefixed bash commands support tab completion.

Note: The hc-test-all command builds and tests everything in core.

Note: The hc-install-* commands may write to the current user's home directory. Other commands that call cargo and npm may also write to the home directory. This is how cargo and npm work unfortunately. hc-**-flush commands delete relevant development artifacts.

Troubleshooting

Default nix-shell behaviour preserves some of the user's environment, simply adding to it rather than isolating from it.

This can cause problems if your user has cruft that conflicts with what nix is doing, e.g. existing cargo or npm installations/environment variables.

If you are seeing an issue in nix-shell that others are not seeing, try using our isolation script ./scripts/nix/pod.sh to debug the command.

For example:

./scripts/nix/pod.sh 'hc-build-wasm && hc-test'

or even:

./scripts/nix/pod.sh hc-test-all

Future deployments

In the future we plan to distribute binaries through nixpkgs. This would to enable the following:

# doesn't work yet... watch this space!
nix-shell -p holochain --run holochain ...

Make (Supported: Ubuntu, Debian & Mac OS X)

For Linux/OSX you can install the prerequisites directly into the host environment with:

cd path/to/holochain
. ./scripts/install/auto.sh

Note: the script will install homebrew on mac os x Note: the script will install dependencies with apt-get on linux

After the install script completes successfully, you can start local development using make

Running the make command will:

1. install (or update to) the correct version of rust
2. build all the rust libraries from the source code in this repository.
3. build and install the command-line tools.

Note: it's very important to use the rust version specified in the Makefile! Since we are using nightly rust builds, the language is changing rapidly and sometimes introduces breaking changes that we haven't adapted to yet. Don't just use the latest nightly.

Note: The installation script evolves over time alongside core. The installation script is idempotent. Rerun the script after each upgrade/downgrade.

Docker (Supported: Ubuntu, Debian, Mac OS X, Nix OS, Windows)

We support docker. The docker folder contains scripts to build and run docker images.

The holochain/holochain-rust:latest docker image is an alpine NixOS rebuilt nightly. The build process warms nix and incrementally compiles cargo/wasm/neon for faster feedback.

Windows

You will need to install rust manually.

Rustup https://rustup.rs/# is likely the best option.

The rust language moves very fast on the nightly channel.

It is very important to be using the correct nightly version.

Currently this is:

nightly-2019-01-24-x86_64-pc-windows-msvc

The nightly version we test/develop against can always be found in the .travis.yml file.

Running tests

Run:

. docker/run-test

or

make test

or

nix-shell --run hc-test

Note that there are also make commands for running the tests of just core, or the command-line line tools or app_spec separately:

make test_cli
make test_holochain
make test_app_spec
make build_nodejs_conductor

Building for Android

Note there is an article written on how to build Holochain for Android, read it here.

Upgrading

Upgrading to a new tagged release of Holochain may include new/changed system dependencies.

If not using nix-shell we strongly recommend rerunning ./scripts/install/auto.sh when upgrading core.

The script is designed to be idempotent. This means there is no harm re-running it.

Contribute

Holochain is an open source project. We welcome all sorts of participation and are actively working on increasing surface area to accept it. Please see our contributing guidelines for our general practices and protocols on participating in the community, as well as specific expectations around things like code formatting, testing practices, continuous integration, etc.

Some helpful links:

• Chat with us on our Chat Server or Gitter

License

Copyright (C) 2018, Holochain Foundation

This program is free software: you can redistribute it and/or modify it under the terms of the license p rovided in the LICENSE file (GPLv3). This program is distributed in the hope that it will be useful, bu t WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Note: We are considering other 'looser' licensing options (like MIT license) but at this stage are using GPL while we're getting the matter sorted out. See this article for some of our thinking on licensing for distributed application frameworks.