jonboh / shai

A Shell AI assistant. It can help you write and explain commands, without having to leave your comfy terminal!

Shai. Shell AI Assitant

Shai is a Shell AI assistant. Its purpose is to help you interact with your machine through a command line interface. It has two main ways to achieve this.

• Command Generation: Shai allows you to quickly ask a LLM model to generate a command based on what it should do.
• Command Explanation: Given a command Shai will explain what a command will do and its side effects.

You'll probably interact with Shai primarily in its command generation mode.

You can also ask Shai to explain the command it just generated:

or a command that is currently in your buffer line:

Note that the explanation will not take into account your initial prompt. This is purposely done to avoid biasing the model explanation. In addition this can help double check that the model generated the correct command and detect model hallucinations.

The model explanations will briefly explain the command and enumerate a list of common side-effects. In addition, if the command is potentially dangerous the model will usually point it out.

As a backend you can use any of the following OpenAI models:

• GPT-3.5-turbo
• GPT-3.5-turbo-16k
• GPT-4
• GPT-4-32k

This might be anecdotal, but I've sometimes experienced lower latencies with the larger context models.

For the time being I don't have access to the gpt-4 models API, but they should work if your account does have access. All examples have been generated with GPT-3.5-turbo, so I expect the results from GPT-4 to be equivalent or better (specially when the request has several moving parts, like pipelines or redirection).

Support for other models is planned. See Current Status

Note that Shai is not intended to replace a solid knowledge of your system or its commands, but it can help ease the burden of memorizing every single flag and command syntax. The OpenAI GPT models are by no means foolproof, and the best results are usually achieved when you as an operator already know the terminology and capabilities related to the command you want to generate. It can also be a resource to explain commands taken from tutorials or forums.

Installation

Cargo

cargo install shai

Remember to add .cargo/bin to your PATH.

This method should be available for all platforms, but you'll need to have Cargo installed, use rustup to install it.

Note: See the next section to integrate shai and allow it to interact with your buffer line.

Arch Linux. AUR:

yay -S shai

Other

Use the latest release in this github repo. Put the binary in a location that is included in your PATH, source the corresponding integration shell script (see next section) in your rc file and you should be good to go.

Shell Integration

I've developed Shai with the aim to have it very closely integrated in the shell. I tend to use it similarly to fzf, so one key-shortcut away.

In this repository you will find the scripts needed to integrate Shai in your shell experience. The following list contains the shells that are currently supported and the planned ones.

• Bash
• Zsh
• Fish
• Nushell
• PowerShell
• Window CMD

In order for Shai to be available through shortcuts you'll need to source the corresponding script. These scripts provide a default binding, modify it to adapt it to your needs. All these scripts perform the same function, the setup a function that is called through the keybind. The function takes the current content of your shell buffer, forwards it to Shai, and if the command generation interface is being used and Shai is exited with Ctrl+a (accept), the buffer line is modified with the command generated. In most cases the function essentially highjacks the command edition capabilities of the shell treating Shai as a text editor.

Shell	Integration file
Bash	bash_assistant.sh
Zsh	zsh_assistant.zsh
Fish	fish_assistant.fish
Nushell	nushell_assistant.nu
PowerShell	powershell_assistant.ps1

If you just install the binary you can generate the integration script using shai

shai generate-script --shell <your-shell, one of {bash, zsh, fish, nushell, powershell}>
# e.g
shai generate-script --shell zsh > zsh_assistant.zsh
# then in your .zshrc
source zsh_assistant.zsh

Remember to source the resulting script in you rc, otherwise you won't have the shortcuts available.

Depending on what model you use you might need to provide the API key as an environment variable. For the OpenAI models you could set it with:

export OPENAI_API_KEY=$(<command_to_get_API_key>)
# if you have it on a text file
export OPENAI_API_KEY=$(cat ~/.secrets/chatgpt.key)

See here how to get your OpenAI API key.

How to use it

Shell Shortcuts

If you don't modify the shell integration script the shortcuts will be the following:

• Alt+s : Command Generation
• Alt+e : Command Explanation

You can change these shortcuts on the shell integration files. They have been chosen to avoid collision with already existing shortcuts in readline.

Shai Controls

• Enter : Send Prompt
• Ctrl+c : Exit Shai
• Esc : Cancel current request

Command Generation

When a command is generated the following controls are also available:

• Ctrl+a : Accept generated command
  • This will check that the model followed the format instructions, when the model provides a long answer this shortcut will retrieve only the text inside markdown code blocks.
• Ctrl+r : Accept generated command (raw)
  • This will accept the input from the model without any check
• Ctrl+e : Explain generated command
  • This will generate an explanation for the command that Shai has just generated.

These keybinds cannot currently be changed.

Examples

Git

Miscellaneous

Run bash on a running container:

Make a ssh tunnel:

Simple ffmpeg operations, or command modifications:

Discard command output:

Current Status

At the moment Shai is memoryless, when you send a prompt it is sent to the model without any context from your previous prompts. The purpose of Shai is not to be a conversational application but to quickly generate commands leveraging LLMs. I might implement conversational functionality later in the future if it improves the commands produced.

I plan to add support for other models like OpenAssistant once they provide an API or a way to run it locally is available (even if the requirements for the machine are steep, but within a high-end desktop). If you have pointers for other similar models that can be run locally or have an available API please, fill an issue and I will try to add support for it.

Context Awareness.

You can modify the assumed Operating System or distro to get more relevant results. This information is provided in the --operating-system option. You can modify your integration script with the appropriate value.

In the same way the --shell option lets the model know in which shell it is running, this can help the model use shell specific features. However, for modern shells like nushell this can actually confuse the model as it won't have a lot of information about this shell in its training data. In cases like this it might be better to let the model believe it is running on another shell (like bash).

Experimental Options

Initially I envisioned Shai as a more capable assistant to which context about the current state of your machine could be forwarded, and it would act accordingly. There are some options that are disabled by default:

Option	Description
pwd	Provides the model with the current working directory
depth	depth with which to run the tree command. It provides context about ther current directory and its content
environment	The list of environment variables set (only their name is passed to the model)
programs	The list of available programs to the model with which to complete the task

I have found that the performance of the GPT3.5 model is lacking in this respect. I have some hope that the GPT-4 (or future) models perform better, so the option to forward this context to the model is still available, if you find good results with it let me know!

Roadmap

In no particular order:

• Custom keybindings
• Prompt Overrides
• Add prompt history navigation.
• Add support for other models. (I'd like to make it easy to interact with a locally run model).
  • OpenAI
  • Local
  • OpenAssistant (once a public API is available)
  • Google Bard (once a public API is available)
• Add other shells
  • Bash
  • Zsh
  • Fish
  • Nushell
  • PowerShell
  • Window CMD

Acknowlegments

• fzf
  • fzf is the main inspiration in terms of the integration with the shell
• ChatGPT-rs
  • I did take some hints from this project to implement the streaming responses from OpenAI.

shai's Issues

Unable to install on macOS Sonoma

Trying cargo install shai, but I’m getting the following:

❯ cargo install shai
    Updating crates.io index
  Installing shai v0.1.1
error: failed to compile `shai v0.1.1`, intermediate artifacts can be found at `/var/folders/zy/17xcp3xx6vb3q0qnfh06185h0000gn/T/cargo-install7meZgt`

Caused by:
  failed to select a version for the requirement `clap = "^4.3"`
  candidate versions found which didn't match: 3.2.25, 3.2.24, 3.2.23, ...
  location searched: crates.io index
  required by package `shai v0.1.1`

I’m not that well versed in Go, but looks like a package management problem. My crates.io seems up to date when I search for packages:

❯ cargo search shai
shai = "0.1.1"                        # A shell AI assistant
[redacted]
❯ cargo search clap
clap = "4.4.6"                             # A simple to use, efficient, and full-featured Command Line Argument Parser
[redacted]

I also tried cloning the repo and updating Cargo.toml to use clap 4.4:

diff --git a/Cargo.toml b/Cargo.toml
index 18e0061..e50f524 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -15,7 +15,7 @@ exclude = ["assets/*"]
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 
 [dependencies]
-clap = { version = "4.3", features = ["derive"] }
+clap = { version = "4.4", features = ["derive"] }
 serde = { version = "1.0", features = ["derive"] }
 serde_yaml = "0.9.22"
 serde_json = "1.0"

But I’m having the same issue:

❯ cargo install --path .
  Installing shai v0.1.1 ([redacted]/shai)
    Updating crates.io index
error: failed to compile `shai v0.1.1 ([redacted]/shai)`, intermediate artifacts can be found at `/Users/remi/Sites/shai/target`

Caused by:
  failed to select a version for the requirement `clap = "^4.4"`
  candidate versions found which didn't match: 3.2.25, 3.2.24, 3.2.23, ...
  location searched: crates.io index
  required by package `shai v0.1.1 ([redacted]/shai)`

Not sure what’s going on here.

Failed to initialize input reader

Yup, I’m back.

This is probably not an issue in Shai, but affects it.

I followed the instructions in the README. Installed the necessary script to make it work in my shell as well. I’ve tried both in my default Zsh and in Bash.

First, in Zsh, when starting Shai, it exits immediately and outputs:

Failed to initialize input reader

Seems it cannot capture stdin or something.

I’m using Oh-My-Zsh. I thought maybe tmux, or atuin, or fzf was interfering. I started Zsh without no config and no plugin instead, except the script needed by Shai. I even tried different keybindings. Sadly, same error.

I tried the same in Bash. The UI does show up, and I can the commands after sending my prompt. But accepting the command just exits, and outputs nothing in the command line.

Sounds like this is an issue in crossterm not working with ttys on macOS as mentioned in an issue with rust-pager. I see Shai is using crossterm. I guess it isn’t that cross-platform after all. ¯\_(ツ)_/¯

For now, I can at least use Shai in Zsh with the following:

shai ask --operating-system macos --shell zsh --model open-aigpt4-32k --write-stdout