tchlux / roanoke-slm-workshop

Code and files for the Roanoke College Small Language Model (SLM) workshop in January 2024.

Living Code Workshop: SLMs in Python

Overview

Welcome to our GitHub repository, designed for the Roanoke College computer science department Living Code Workshop (slides)! This repository showcases the capabilities of small language models, demonstrating how they can be effectively used in various applications. Our Python script we wrote in the workshop provides a glimpse into the world of AI-driven text generation, parsing, and interaction.

Features

• Language Model Integration: Utilizes small language models for generating text, answering questions, and more.
• Customizable Grammars: Implements Llama grammars for constrained outputs, ensuring relevance and precision.
• Real-time Demonstrations: Includes live examples to illustrate the use of language models in practical scenarios.

Set Up

Clone this repository to your local machine using the following command:

git clone https://github.com/tchlux/roanoke-slm-workshop

To run the script, you will need to install the llama-cpp-python library. You can do this using pip:

python3 -m pip install --user llama-cpp-python

Then download your favorite small language model (examples below) and put it into this directory for easy access. For optimal performance, we recommend using one of the following models:

1. Mistral Instruct: A versatile model suitable for various instructional tasks. Model Link
2. StableLM Zephyr: A smaller, more efficient model for quick and compact responses. Model Link

Usage

The script allows you to interact with pre-trained language models. To run the script, navigate to the cloned repository's directory and execute:

python3 slm.py

There is an additional script that shows how to call the Google search API from code to fetch knowledge and real time information.

python3 google.py

Useful Links

• LM Studio: For a selection of local language models and a pretty user interface, visit lmstudio.ai.
• Hugging Face: Explore a wide range of models published on huggingface.com.

Contributing

Feel free to fork this repository and contribute by submitting a pull request. We appreciate any input!