GithubHelp home page GithubHelp logo

gaffner / cataigpt Goto Github PK

View Code? Open in Web Editor NEW

This project forked from withcatai/catai

0.0 0.0 0.0 14.91 MB

the same loved alpaca model, wrapped in chatGPT like frontend. run your AI assistant locally

License: MIT License

JavaScript 41.75% TypeScript 43.75% CSS 2.00% HTML 1.62% Dockerfile 0.17% Svelte 10.70%

cataigpt's Introduction

CatAI Logo

CatAI

npm version npm downloads GitHub license


Run Alpaca model on your computer with a chat ui.

Your own AI assistant run locally on your computer.

Inspired by Dalai, Node-Llama, Alpaca.cpp

Installation & Use

npm install -g catai

catai install Vicuna-7B
catai serve

catai

Features

  • Auto detect programming language 🧑‍💻
  • Auto detect code block 📃
  • Click on user icon to show original message 💬
  • Real time text streaming ⏱️
  • Fast model downloads 🚀

Intro

You can use any Alpaca model as long as your computer can handle it.

catai install Vicuna-13B

You can also download a custom model like this:

catai install https://example.com/model.tar.bin --tag myModel

If you want to switch between models you can use catai use command.

catai use Vicuna-7B

You can use all the UIs from the client directory (default catai).

catai server --ui chatGPT

Cross-platform

You can use it on Windows, Linux and Mac.

This package from version 1.6.0 could depend on llama-node which supports:

  • darwin-x64
  • darwin-arm64
  • linux-x64-gnu
  • win32-x64-msvc

Memory usage

Runs on most modern computers. Unless your computer is very very old, it should work.

According to a llama.cpp discussion thread, here are the memory requirements:

  • 7B => ~4 GB
  • 13B => ~8 GB
  • 30B => ~16 GB

Configuration

You can change the configuration by edition the config.js file.

catai config --editor [editor]

After you change the configuration, you need to restart the server.

  • 💡To increase the model understanding, try change the context size.
  • 💡To increase the model output, try change the numPredict size.

List models

You can list all the models that you have installed.

catai list

Uninstall models

You can uninstall models that you don't need.

catai remove Vicuna-7B

Uninstall package

You can uninstall the package.

catai remove all # remove all downloaded data
npm uninstall -g catai

Good to know

  • All download data will be downloaded at ~/catai folder.
  • The download is multi-threaded, so it may use a lot of bandwidth, but it will download faster!

Development

If you want to run the source code locally, you can follow the steps below.

To run the client.

cd client/catai
npm install
npm run dev

To run the server.

cd server
npm install
npm run install-model Vicuna-7B
npm start

Troubleshooting

Error loading model OR executable error

Try change the config:

export const SELECTED_BINDING = 'alpaca-cpp';

It may be slower, but it has more chance to work with alpaca models.

Windows Subsystem for Linux has no installed distributions

Problem with the dependency zx, try to run inside git-bash.

License

This project use Alpaca.cpp to run Alpaca models on your computer. So any license applied to Alpaca.cpp is also applied to this project.

Credits

The GPT frontend is built on top of the chatGPT Frontend mimic


Star please

If you like this repo, star it ✨                                                    

cataigpt's People

Contributors

gaffner avatar ido-pluto avatar rikthepixel avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.