artix75 / psyc

A C implementation of common Artificial Neural Networks

PsyC

PsyC is an open-source C library that allows building neural networks on POSIX systems such as Linux or macOS. It provides a collection of layer types and functionalities to facilitate the construction and training of neural networks.

Idea and Project Name

PsyC is the result of the fascination with artificial neural networks and their similarities to biological neural networks and the human brain. The project's name, PsyC, is derived from a combination of "psyche" (mind) and "C" (programming language), representing the intersection of the developer's background in psychology and their subsequent career as a programmer.

The project started in 2016 as a self-directed journey to explore artificial neural networks, mainly for autodidactic purposes. Recognizing the need for improved performance in neural network training, the developer opted to implement the library using the C programming language.

After a five-year pause, the project restarted in the summer of 2022.

Features

Supported layer types:

• Fully Connected (Dense) layers.
• Linear layers.
• Convolutional/Pooling layers.
• Normalization layers.
• Dropout layers.
• Embedding layers.
• Recurrent, LSTM, and GRU layers.
• Positional Encoding.
• Attention mechanism.

PsyC also provides additional features:

• Building models composed of multiple neural networks, such as encoder-decoder models
• Building transformers by utilizing Attention layers and enabling a flag that allows feeding the whole sequence as inputs

System Requirements

• POSIX-compliant operating system (Linux, macOS)
• zlib must be installed on your system

Supported hardware/libraries

PsyC can leverage certain libraries and hardware for accelerated computation.

• Apple® Accelerate Framework for optimal performance on Apple® Silicon CPUs.
• Intel® AVX instruction set support for improved performance on x86 CPUs.
• BLAS routines support can be provided by either Apple Accelerate Framework or GNU GSL Library if they're found. If none of the previous options are found, PsyC still provides its own partial implementation of some BLAS routines (but it's actually quite slow in this latter case).

The above libraries and hardware are automatically detected by PsyC and they're not dependecies, as PsyC can run without them.

Currently, there's no support for NVIDIA® CUDA®, but its support is planned for future releases.

Suggested Additional Libraries

• ImageMagick® for converting images to data that can be used as input to models.

Installation

1. Clone the repository:

   git clone https://github.com/artix75/psyc

2. Build the library:

   cd psyc
   make

   Note: Make sure you have the necessary dependencies installed.

3. Install:

   make install

   Note: by default, PsyC will be installed inside /usr/local. If you want to install PsyC to a different location, provide the PREFIX variable to the make command, like this.

   make PREFIX=/opt install

Testing

make test

Usage

After installing it, PsyC will provide:

• A dynamic library: libpsyc.so (libpsyc.dylib on macOS)
• A static library: libpsyc.a
• A command line tool: psycl
• An utility to build your own program with the PsyC lib: psyc-cc

Example:

1. Include the necessary headers in your source file:

   #include <psyc/all.h>

2. Create a neural network:

   PSModel *model = PSModelCreate("My Awesome Model");

3. Add some layers:

   PSAddLayer(model, FullyConnected, 784, NULL); /* Input layer, size: 784) */
   PSAddLayer(model, FullyConnected, 30, NULL); /* Hidden dense (FullyConnected) layer of size 30) */
   PSAddLayer(model, SoftMax, 10, NULL); /* Output softmax layer of size 10) */

   The code above creates a basic neural network composed of only Fully-Connected (Dense) layers and a Softmax layer used for the output layer. However, PsyC offers various types of layers (see the Features section). Here are some example codes:

   /* Add a convolutional layer */
   PSLayerDef conv_def = {.filter_width = 3, .stride = 1, .padding = 2};
   PSAddLayer(model, Convolutional, 0, &conv_def);
   
   /* Add LSTM or GRU layers */
   PSAddLayer(model, LSTM, 10, NULL);
   PSAddLayer(model, GRU, 10, NULL);
   
   /* Add an Attention layer */
   PSLayerDef attn_def = {.attention_heads = 4, .causal_attention = 1};
   PSAddLayer(model, Attention, 10, &attn_def);

4. Train your model:

   PSFloat training_data[] = {...};
   PSTraningOptions opts = {.learning_rate = 0.01};
   long datalen = (long) sizeof(training_data) / sizeof(PSFloat);
   PSTrain(model, training_data, datalen, NULL, 0, &opts);

5. Get predictions:

   Get raw predictions by forwarding inputs to the model and by reading the output layer's state:

   PSFloat inputs[] = {...};
   PSForward(model, inputs);
   PSLayer *output_layer = PSGetOutputLayer(model);
   PSFloat *outputs = PSGetOutputs(output_layer);

   Or get classification predictions:

   PSFloat inputs[] = {...};
   int predicted = PSClassify(model, inputs)

6. Finally, release the model:

   PSModelFree(model);

7. Compile your program by linking against the PsyC library, like this:

   PREFIX=/usr/local # Or whatever you used as PREFIX during install
   gcc -I$PREFIX/include -L $PREFIX/lib -lpsyc -o myprogram myprogram.c

   or, alternatively, you can use the provided psyc-cc utility:

   psyc-cc -o myprogram myprogram.c

Command Line Tool and Demos

PsyC can also be built as a command line tool (psycl). To see its usage:

psycl --help

Command Line Tool example:

psycl --layer fully_connected 784 --layer fully_connected 30 --layer fully_connected 10 --train --mnist --test --mnist

Demos

PsyC also provides several demos.

• MNIST demo
• CIFAR demo
• Recurrent networks demo
• Encoder-decoder demo (both with and without attention mechanism)
• GPT2 emulation demo

After building PsyC, you can find them inside the bin/ subdirectory that is found inside the PsyC source directory.

Documentation

PsyC ships with its own documentation that can be found inside the doc directory of the repository.
The documentation comes in three formats: HTML, Markdown and UNIX manual pages. After installing PsyC with make install, documentation will be installed too. UNIX manual pages will be properly installed into $PREFIX/share/man, while other formats will be installed inside $PREFIX/share/psyc directory. The index page of HTML documentation can be found at doc/html/index.html. The main UNIX manual page is psyc(7), so, after installing PsyC, it can be simply accessed with man psyc. Although the documentation is not complete and it's still in progress, most of the more commonly used functions and data structures are documented.

Future Roadmap

• GPU support (e.g., CUDA®) for enhanced performance on compatible systems
• OpenBLAS support
• Diffusion models, VAE, GAN, and so on.
• Bi-directional recurreent networks

License

PsyC is licensed under the BSD 3-Clause License.

psyc's Issues

How to train the LSTM network?

I need an example.