GithubHelp home page GithubHelp logo

bennyschmidt / ragdoll Goto Github PK

View Code? Open in Web Editor NEW
80.0 3.0 3.0 323 KB

The library for character-driven AI experiences.

License: MIT License

JavaScript 100.00%
ai bots chat chatbot-framework generative-ai llamaindex llm ollama rag rag-implementation

ragdoll's Issues

Bring your own models (BYOM)

Background

During a query, there are 2 lifecycle methods that invoke third-party models:

  • invokeChatAgent: The vendor-specific part being the chatAgent.chat({ message }); line that calls GPT-3.5 Turbo
  • invokeImageAgent: The vendor-specific part is where we call DALL-E 2
imageAgent.images.generate({
  model: IMAGE_MODEL,
  prompt: imageModelPrompt,
  size: `${IMAGE_SIZE}x${IMAGE_SIZE}`,
  quality: IMAGE_QUALITY,
  n: 1
});

I've already experimented with swapping this DALL-E 2 call out with one made to Leonardo.AI using the exact same prompt to get much better images from fine-tuned models - so it works as a proof of concept.

The feature request

Would be cool if - instead of just calling OpenAI during a query - we call a middleware function defined by the developer (which defaults to GPT-3.5/DALL-E 2 if none are provided) so they can use their own text or image models.

Let's say you wanted to use GPT-4 for text, or maybe you want to get outside OpenAI entirely by going with Claude 3. And for images, maybe you want to use a custom fine-tuned model that you trained using Leonardo.AI. The developer should be able to define their own middleware to send and augment prompts to whatever models they want.

Default models

I predict (hope?) that GPT-3.5 and DALL-E 2 will soon be free for general use, especially as more people see how varied the model outputs can be (how much better GPT-4 is than GPT-3.5, for example), and how advanced fine-tuned models seem compared to generic ones. I can at least imagine a pay-per-model future where some of them are free. There are already great model communities like Hugging Face and Nomic's GPT4All where developers are sharing models.

Until then, in terms of adoption Arthas might benefit from defaulting to free models that don't require an API key to get started. It would make it easier to install and adopt for developers working on LLM apps rather than having to create API keys, add credit cards, manage spending etc. before they're even able to try it.

Default data framework

The backbone of Arthas is the underlying data framework, giving it the ability to search and index knowledge in a conversational way. This framework - llamaindex (and there are some others out there, for example langchain) - requires model access too, and supports the following ones:

  • OpenAI GPT-3.5-turbo and GPT-4
  • Anthropic Claude Instant and Claude 2
  • Groq LLMs
  • Llama2 Chat LLMs (70B, 13B, and 7B parameters)
  • MistralAI Chat LLMs
  • Fireworks Chat LLMs

Arthas is tied to this vendor via the createIndex and createQuery functions, so in order to truly become model-agnostic the developer would need a middleware option surfaced to them to bypass these as well.

Ideally, a developer could just npm i arthasgpt and use the library without needing any accounts or API keys by default, and could just start creating personas and making stuff.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.