momegas / megabots

🤖 State-of-the-art, production ready LLM apps made mega-easy, so you don't have to build them from scratch 🤯 Create a bot, now 🫵

Hello 👋, I'm Megaklis Vasilakis the co-founder and CTO of Squaredev.

I work on machine learning engineering, DevOps and software in general. I'm mainly writing python for some years now. Before that, I was a JavaScript / Node.js guy. I also organise an AI meetup in my hometown, Patras.

I'm working on open source, mainly on Whitebox and Explainable AI repo of Squaredev (links below). Lately I also helped a bit with the Open Assistant project. In have done other OSS work here and there in the past as well.

💬 Ask me anything about DevOps, MLOps, AI/ML, APIs, Kubernetes.

🤝 Megas for friends, momegas (monkey megas) on the web.

Linkedin profile: https://www.linkedin.com/in/megaklis-vasilakis/

GitHub profile: https://github.com/momegas

Connect with me:

Languages and Tools:

megabots's Issues

support azure openai api

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

I have an azure openai api account, but I don't see where it can be used

Describe the solution you'd like
A clear and concise description of what you want to happen.

If not, can you access the azure openai api?

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

It may be possible to add a proxy, but I haven't tested it
https://github.com/stulzq/azure-openai-proxy

Additional context
Add any other context or screenshots about the feature request here.

Create a Dockerfile example

Exposing API - Multiprocessing error

I've created a main.app file as described in README file

from megabots import bot, create_api

app = create_api(bot("qna-over-docs"))

then I just run uvicorn to get this issue

Using model: gpt-3.5-turbo
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/moamen/.local/lib/python3.10/site-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
    target(sockets=sockets)
  File "/home/moamen/.local/lib/python3.10/site-packages/uvicorn/server.py", line 59, in run
    return asyncio.run(self.serve(sockets=sockets))
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/moamen/.local/lib/python3.10/site-packages/uvicorn/server.py", line 66, in serve
    config.load()
  File "/home/moamen/.local/lib/python3.10/site-packages/uvicorn/config.py", line 471, in load
    self.loaded_app = import_from_string(self.app)
  File "/home/moamen/.local/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string
    module = importlib.import_module(module_str)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/moamen/megabots/main.py", line 3, in <module>
    app = create_api(bot("qna-over-docs"))
    raise RuntimeError(
RuntimeError:
            Impossible to find a valid index.
            Either provide a valid path to a pickle file or a directory.

Environment:

Python: 3.10
Pip: 23.1
OS: WSL - Ubuntu

Import error

Hi, upon import:

from qnabot.QnABot import QnABot

I receive the error:

import os, sys
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/xxxxxx/miniforge3/lib/python3.9/site-packages/qnabot/__init__.py", line 1, in <module>
    from .QnABot import QnABot
  File "/Users/xxxxxx/miniforge3/lib/python3.9/site-packages/qnabot/QnABot.py", line 12, in <module>
    class QnABot:
  File "/Users/xxxxxx/miniforge3/lib/python3.9/site-packages/qnabot/QnABot.py", line 16, in QnABot
    index: str | None = None,
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

I am on an M1 Mac if that makes a difference.

Add vector database support

Create the release GH action

Error in langchain.embeddings load

Describe the bug
When I run in the notebook the "qnabot = bot("qna-over-docs", "./index")" I get an error

Retrying langchain.embeddings.openai.embed_with_retry.._embed_with_retry in 4.0 seconds as it raised RateLimitError: You exceeded your current quota, please check your plan and billing details..

Create docs site

Is your feature request related to a problem? Please describe.
Yup! The problem of not having good documentation. We need to create a nice site

Describe the solution you'd like
Well... Docs!

Describe alternatives you've considered
No alternatives when docs is on the table...

Additional context
NA

Alternative APIs

Title

Phind and Poe

Problem Description

Some countries have no direct access to OpenAI, and some of us have no paid accounts etc., but Poe provides access to them as well as Claude, and Phind is great for coding and has special prompting on the backend and up to 6000characters currently. It would be very helpful if we could use API keys for these services for our agents in an app like this. Thank you.

Proposed Solution

Python wrapper for poe-api and/or Phind

Alternatives Considered

No response

Additional Context

No response

Acknowledgements

• I have searched the existing issues to make sure this feature has not been requested yet
• I have provided enough information for the maintainers to understand and evaluate the feature request

Add memory to bot

Add option for models to provide sources in QnA

Is your feature request related to a problem? Please describe.
During bot instanciation the user should be able to specify if they need sources in their responses. This is especially true with the qna-over-docs bots.

Describe the solution you'd like

qnabot = bot("qna-over-docs", sources=True)

You can use the load_qa_with_sources_chain from LangChain but in my tests it failed when I tries to change the prompt. Maybe I was doing something wrong, but I also opened an issue

Describe alternatives you've considered
There is a chance that the ready-made chains provided by LangChain won't be enough after some time. But this is for later I think.

Additional context
NA

TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Hello,

I am following the code, but I am getting the below error in the first line of "from megabots import bot"

TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Add agents support

Add an autoGPT (or something similar) as a bot type

Do some research on HazyResearch/manifest

Have a look at this repo. Its an abstraction of different models. Can we use it?

An example:

from manifest import Manifest

# Start a manifest session to OpenAI - default `engine=text-davinci-002`
manifest = Manifest(
    client_name = "openai",
)
manifest.run("Why is the grass green?")

https://github.com/HazyResearch/manifest

Add Chroma vector database support

Add pinecone vector database support

Integrate with langchain-serve?

Is your feature request related to a problem? Please describe.
Great work with megabots. Looks like a nice & easy API to develop LLM-based bots. What plans do you have to deploy the bots without adding infrastructure headache?

Describe the solution you'd like
I'd recommend integrating with langchain-serve.

• Exposes APIs from function definitions locally as well as on the cloud.
• Very few lines of code changes, ease of development remains the same as local.
• Supports both REST & Websocket endpoints
• Serverless/autoscaling endpoints with automatic tls certs.
• Real-time streaming, human-in-the-loop support

Semanticly cache LLM Queries and reduce Costs

Support for Open Assistant

Is your feature request related to a problem? Please describe.
QnABot should also be able to choose OpenAssistant as a model for QnA because it's open source and rocks!

Describe the solution you'd like
Add a choice in QnABot instantiation to choose a model. Something like below:

bot = QnABot(directory="./mydata", model="OpenAssistant")

Describe alternatives you've considered
__

Additional context
At the moment Open Assistant is not part of LangChain, so maybe we will need to add it before implementing this in this repo.

Add GPT4 as a supported model

Is your feature request related to a problem? Please describe.
Make GPT4 available in the library with:

qnabot = bot("qna-over-docs", model="<GPT4 model>")

Describe the solution you'd like
Same as above

Describe alternatives you've considered
NA

Additional context
NA

Expose all bot functions in API with proper swagger docs

Accept voice as an input using whisper

Validate and correct the outputs of LLMs using guardrails