Update README

Intel Homomorphic Encryption Toolkit

Intel Homomorphic Encryption (HE) Toolkit is Intel's primary platform for delivering innovation around HE with the aim of providing both the community and industry with an intuitive entry point for Experimentation, Development and Deployment of HE applications. Intel HE Toolkit currently offers sample kernels and example programs that demonstrate varying operations and applications that can be built leveraging three major HE libraries Microsoft SEAL, PALISADE, and HElib. Moreover, Intel HE Toolkit demonstrates the advantages of using Intel® Processors through libraries such as the Intel HE Acceleration Library to utilize the latest Intel hardware features.

Quick Start

Want to get up and running quickly with Intel HE toolkit? Then follow our Quick Start guide.

Dependencies

Intel HE toolkit has been tested on Ubuntu 22.04

Must have system dependencies for the toolkit include,

python >= 3.10
pip
git

Further Python dependencies include,

toml
argcomplete (optional: tab completion)
docker      (optional: building docker containers)
pytest      (optional: running tests)
pytest-mock (optional: running tests)

For faster setup, a requirements.txt file is provided for recommended user python dependencies and a dev_reqs.txt is provided for all python dependencies listed above and pre-commit tools for development. Either file can install dependencies with

pip install -r <requirements-file>

However, to build anything useful with the toolkit, we recommend to additionally install the following system dependencies,

m4
patchelf
cmake >= 3.22
g++ == 11.x or clang == 14.x
pthread
virtualenv (optional if building the Logistic Regression Example)
autoconf   (optional if using PALISADE)
gmp == 1.5.x  (optional if using HElib)

Setup

To set up the toolkit users must first initialize the hekit command using

git clone https://github.com/intel/he-toolkit.git
cd he-toolkit
./hekit init --default-config

This will create a directory ~/.hekit in the user's home directory and create the default.config, or other user specified config file to this location. This directory will be where all components built and installed by hekit will be kept.

Moreover, the hekit command will be added to the user's PATH so as to enable the user to call the command from anywhere on their system. This modifies your shell's initialization script (currently only in bash).

Intel HE toolkit is primarily accessible through the hekit command. There are currently two ways of interacting with the toolkit: through a Docker build or directly on your system.

The hekit command

The hekit command is a command-line tool that can be used by the user to easily set up an HE environment in a configurable and intuitive manner.

The hekit command has a help option which lists all subcommands and flags

hekit -h

Moreover, each subcommand has a help option that can be invoked with hekit <subcommand> -h.

The hekit subcommands consist of utility commands such as check-dependencies and docker-build as well as commands for managing the fetching, building, and installation of user-defined projects using recipe files (more information found here). See the README for more detailed information on the usage of hekit.

Docker build (Recommended)

The recommended method is to use the Docker build and installation which builds the toolkit in its entirety including all HE libraries in a self-contained docker container running Ubuntu 20.04. This can be built through the hekit command

hekit docker-build

Additionally, the docker build can optionally be used via VS Code Server. See here for a detailed description on the usage and components of this build.

System build

Alternatively, one can build the toolkit's HE components using the following command

hekit install recipes/default.toml

This will build the toolkit project with the default settings. The toolkit will download and build all three HE libraries automatically with Intel HE Acceleration Library enabled.

Note: You will be responsible for installing all of the required dependencies.

The sample kernels and examples can also be built in a similar manner using

hekit build recipes/sample-kernels.toml

and

hekit build recipes/examples.toml

respectively.

To view a list of the components and their instances have been built/installed through hekit one can execute

hekit list

to list each component, their instance(s) and the status of the fetch, build, and install steps.

Kernels

Located in he-samples is a collection of software components built on Microsoft SEAL and PALISADE comprising sample kernels for operations performed homomorphically and example applications. The HE Samples are designed to enable quicker evaluation of HE on Intel platforms, serve as a learning tool for how to implement operations in different HE libraries, and provide examples of how these operations can be used to build applications based on HE technology for different use cases.

Sample kernels

The sample kernels are for complex HE operations, requiring multiple API calls such as Matrix Multiplication and Vector Dot Product. See the README for instructions.

Test sample kernels

The unit tests are a selection of unit tests meant for verifying the accuracy of the various sample kernels included in this project. See the README for more information.

Examples

The examples directory includes example applications built using HE technology. The primary purpose of these examples is to serve as a showcase of different use cases which can be implemented using HE. Moreover, these can be used as learning references and starting points for further development. The toolkit currently includes the following examples listed below.

Secure Query

The secure query example shows how it is possible to implement a key-value database using HE. This allows a client to perform lookups of values in the database without exposing the query to the server hosting the database and optionally the key-value pairs in the database as well. The secure query example is implemented using the SEAL BFV scheme. See the README for more details and instructions on how to run this program.

Logistic Regression

The transposed logistic regression example presents a scalable and fast method of logistic regression inference in HE. Using the SEAL CKKS scheme, the example will encrypt the model (bias and weight) and takes batches of encrypted data samples to perform the inference all within the HE domain. See the README for usage information.

Private Set Intersection

The Private Set Intersection (PSI) example computes the intersection of two given sets. The program computes a hash value for each entry of both the client and the server sets, then using the HElib BGV scheme, it encrypts the client set and computes the intersection, returning all the encrypted elements that are common to both sets. See the README for usage information.

New Examples

Using the hekit new command users can create example projects of their own. These can be entirely new projects or projects based on one of the provided examples listed above. For more on the usage of the hekit new command see here.

Known Issues

Running ./hekit init --default-config produces the error ValueError: Unknown shell 'sh' The hekit command currently only supports bash, please ensure the default shell is set to bash or alternatively set the environment variable export SHELL=/bin/bash.
There is a specific hardware configuration, AVX512DQ, which causes some of the PALISADE sample kernels to fail when building the HEXL library with PALISADE. This error seems independent of the HE Toolkit and is currently being investigated.
HE Toolkit SEAL examples currently do not work with clang-12 on ubuntu 20.04 and is under investigation.
HE Toolkit SEAL examples currently do not work with clang-12 and clang-13 on ubuntu 22.04 and is under investigation.

Third Party Plugins

Intel HE Toolkit provides the utilities to handle third party plugins that can be installed in the system to add new HE functionalities. See PLUGINS for details about how to interact with them or create new plugins.

Contributing

Intel HE Toolkit welcomes external contributions through pull requests to the main branch.

Please sign your commits before making a pull request. See instructions here for how to sign commits.

Before contributing, please ensure that you run pre-commit with

pre-commit install
pre-commit run --all-files

and make sure all checks pass.

Also please run

pytest tests

to make sure the tests pass.

We encourage feedback and suggestions via GitHub Issues as well as via GitHub Discussions.

Adding a New Command

The following steps allows to add a new subcommand to hekit as a command or tool. Below ACTION must be replaced by a word or set of words that described the functionality of the new command.

Create a new python file ACTION.py inside either the commands directory or the tools directory.

Create a function to set the parser of the subcommand (the subparser to hekit parser). The function must begin and end with set_ and \_subparser, respectively. Define the arguments of the command. The parameter of the function set_defaults(fn) must be set to the function defined in the next step. Check argparse for API reference information.

def set_ACTION_subparser(subparsers):
    """create the parser for the 'ACTION' command"""
    parser_ACTION = subparsers.add_parser("ACTION", description="ADD-SUBPARSER-DESCRIPTION")
    parser_ACTION.add_argument(
        "ARG1", help="ADD-ARG-DESCRIPTION"
    )
    parser_ACTION.add_argument(
        "ARG2", help="ADD-ARG-DESCRIPTION"
    )
    parser_ACTION.set_defaults(fn=NEW_FUNCTIONALITY)

Create the set of functions that implement the new functionality. The entry point must be a function that has args as parameter and it will use the arguments defined in the previous step.

def NEW_FUNCTIONALITY(args) -> None:
    """Executes new functionality"""
    if(args.ARG1):
        # code goes here
    elif(args.ARG2):
        # code goes here

hekit should automatically discover the function set_ACTION_subparser and enable the options of the new command.
Generic utilities or helper functions that can be used for several commands should be placed in utils.

Troubleshooting

Executable `cpplint` not found

Make sure you install cpplint: pip install cpplint. If you install cpplint locally, make sure to add it to your PATH.
/bin/sh: 1: pre-commit: not found

Install pre-commit. More info at https://pre-commit.com/.
When attempting to sign a commit
```
error: gpg failed to sign the data
fatal: failed to write commit object
```
Try adding export GPG_TTY=$(tty) to your shell initializer script such as ~/.bashrc.
When writing recipe files, there is a known issue when chaining bash commands such as
```
pre-build = "cd .. && ls"
```
where hekit does not know how to interpret the && symbol. To resolve this issue, wrap the commands in bash -c 'command1 && command2 && ...' to produce
```
pre-build = "bash -c 'cd .. && ls'"
```

Contributors

The Intel past and present contributors to this project, sorted by last name, are

Software	Version(s)
Linux	rhel fedora 8.8 kernel-5.4
Git	2.39
CMake	N/A
Compiler (GCC, Clang, etc.)	8.5
Python	3.11
Doxygen	N/A
HE Toolkit	main 859025f
Recipe File(s) Used	N/A

	m_context.reset(
	new seal::SEALContext(m_parms, true, seal::sec_level_type::none));

intel / he-toolkit Goto Github PK

he-toolkit's Introduction

Intel Homomorphic Encryption Toolkit

Contents

Quick Start

Dependencies

Setup

The hekit command

Docker build (Recommended)

System build

Kernels

Sample kernels

Test sample kernels

Examples

Secure Query

Logistic Regression

Private Set Intersection

New Examples

Known Issues

Third Party Plugins

Contributing

Adding a New Command

Troubleshooting

Contributors

he-toolkit's People

Contributors

Stargazers

Watchers

Forkers

he-toolkit's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs