edward7770 / gpt4all

gpt4all: run open-source LLMs anywhere

GPT4All

Open-source large language models that run locally on your CPU and nearly any GPU

GPT4All Website and Models • GPT4All Documentation • Discord

🦜️🔗 Official Langchain Backend

GPT4All is made possible by our compute partner Paperspace.

Run on an M1 macOS Device (not sped up!)

GPT4All: An ecosystem of open-source on-edge large language models.

GPT4All is an ecosystem to run powerful and customized large language models that work locally on consumer grade CPUs and any GPU. Note that your CPU needs to support AVX or AVX2 instructions.

Learn more in the documentation.

A GPT4All model is a 3GB - 8GB file that you can download and plug into the GPT4All open-source ecosystem software. Nomic AI supports and maintains this software ecosystem to enforce quality and security alongside spearheading the effort to allow any person or enterprise to easily train and deploy their own on-edge large language models.

What's New (Issue Tracker)

• Latest Release
• October 19th, 2023: GGUF Support Launches with Support for:
  • Mistral 7b base model, an updated model gallery on gpt4all.io, several new local code models including Rift Coder v1.5
  • Nomic Vulkan support for Q4_0 and Q4_1 quantizations in GGUF.
  • Offline build support for running old versions of the GPT4All Local LLM Chat Client.
• September 18th, 2023: Nomic Vulkan launches supporting local LLM inference on AMD, Intel, Samsung, Qualcomm and NVIDIA GPUs.
• August 15th, 2023: GPT4All API launches allowing inference of local LLMs from docker containers.
• July 2023: Stable support for LocalDocs, a GPT4All Plugin that allows you to privately and locally chat with your data.

Chat Client

Run any GPT4All model natively on your home desktop with the auto-updating desktop chat client. See GPT4All Website for a full list of open-source models you can run with this powerful desktop application.

Direct Installer Links:

• macOS

• Windows

• Ubuntu

Find the most up-to-date information on the GPT4All Website

Chat Client building and running

• Follow the visual instructions on the chat client build_and_run page

Bindings

• 🐍 Official Python Bindings
• 💻 Official Typescript Bindings
• 💻 Official GoLang Bindings
• 💻 Official C# Bindings
• 💻 Official Java Bindings

Integrations

• 🗃️ Weaviate Vector Database - module docs

Contributing

GPT4All welcomes contributions, involvement, and discussion from the open source community! Please see CONTRIBUTING.md and follow the issues, bug reports, and PR markdown templates.

Check project discord, with project owners, or through existing issues/PRs to avoid duplicate work. Please make sure to tag all of the above with relevant project identifiers or your contribution could potentially get lost. Example tags: backend, bindings, python-bindings, documentation, etc.

GPT4All 2024 Roadmap

To contribute to the development of any of the below roadmap items, make or find the corresponding issue and cross-reference the in-progress task.

Each item should have an issue link below.

• Chat UI Language Localization (localize UI into the native languages of users)

  • Chinese
  • German
  • French
  • Portuguese
  • Your native language here.

• UI Redesign: an internal effort at Nomic to improve the UI/UX of gpt4all for all users.

  • Design new user interface and gather community feedback
  • Implement the new user interface and experience.

• Installer and Update Improvements

  • Seamless native installation and update process on OSX
  • Seamless native installation and update process on Windows
  • Seamless native installation and update process on Linux

• Model discoverability improvements:

  • Support huggingface model discoverability
  • Support Nomic hosted model discoverability

• LocalDocs (towards a local perplexity)

  • Multilingual LocalDocs Support
    • Create an multilingual experience
    • Incorporate a multilingual embedding model
    • Specify a preferred multilingual LLM for localdocs
  • Improved RAG techniques
    • Query augmentation and re-writing
    • Improved chunking and text extraction from arbitrary modalities
      • Custom PDF extractor past the QT default (charts, tables, text)
    • Faster indexing and local exact search with v1.5 hamming embeddings and reranking (skip ANN index construction!)
  • Support queries like 'summarize X document'
  • Multimodal LocalDocs support with Nomic Embed
  • Nomic Dataset Integration with real-time LocalDocs
    • Include an option to allow the export of private LocalDocs collections to Nomic Atlas for debugging data/chat quality
    • Allow optional sharing of LocalDocs collections between users.
    • Allow the import of a LocalDocs collection from an Atlas Datasets
      • Chat with live version of Wikipedia, Chat with Pubmed, chat with the latest snapshot of world news.

• First class Multilingual LLM Support

  • Recommend and set a default LLM for German
  • Recommend and set a default LLM for English
  • Recommend and set a default LLM for Chinese
  • Recommend and set a default LLM for Spanish

• Server Mode improvements

  • Improved UI and new requested features:
    • Fix outstanding bugs and feature requests around networking configurations.
    • Support Nomic Embed inferencing
    • First class documentation
    • Improving developer use and quality of server mode (e.g. support larger batches)

Technical Reports

📗 Technical Report 3: GPT4All Snoozy and Groovy

📗 Technical Report 2: GPT4All-J

📗 Technical Report 1: GPT4All

Citation

If you utilize this repository, models or data in a downstream project, please consider citing it with:

@misc{gpt4all,
  author = {Yuvanesh Anand and Zach Nussbaum and Brandon Duderstadt and Benjamin Schmidt and Andriy Mulyar},
  title = {GPT4All: Training an Assistant-style Chatbot with Large Scale Data Distillation from GPT-3.5-Turbo},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/nomic-ai/gpt4all}},
}