freedomfromfiat / embedchain

Data platform for LLMs - Load, index, retrieve and sync any unstructured data

embedchain

Embedchain is a Data Platform for LLMs - load, index, retrieve, and sync any unstructured data. Using embedchain, you can easily create LLM powered apps over any data. If you want a javascript version, check out embedchain-js

Community

• Join embedchain community on slack by accepting this invite

🤝 Schedule a 1-on-1 Session

Book a 1-on-1 Session with Taranjeet, the founder, to discuss any issues, provide feedback, or explore how we can improve Embedchain for you.

🔧 Quick install

pip install --upgrade embedchain

🔍 Demo

Try out embedchain in your browser:

📖 Documentation

The documentation for embedchain can be found at docs.embedchain.ai.

💻 Usage

Embedchain empowers you to create ChatGPT like apps, on your own dynamic dataset.

Data types supported

• Youtube video
• PDF file
• CSV file
• Web page
• MDX file
• XML file
• Sitemap
• Doc file
• Notion
• JSON file
• OpenAPI specs
• Code docs website
• Unstructured file loader and many more

You can find the full list of data types on our documentation.

Queries

For example, you can use Embedchain to create an Elon Musk bot using the following code:

import os
from embedchain import Pipeline as App

# Create a bot instance
os.environ["OPENAI_API_KEY"] = "YOUR API KEY"
elon_bot = App()

# Embed online resources
elon_bot.add("https://en.wikipedia.org/wiki/Elon_Musk")
elon_bot.add("https://www.forbes.com/profile/elon-musk")
elon_bot.add("https://www.youtube.com/watch?v=RcYjXbSJBN8")

# Query the bot
elon_bot.query("How many companies does Elon Musk run and name those?")
# Answer: Elon Musk currently runs several companies. As of my knowledge, he is the CEO and lead designer of SpaceX, the CEO and product architect of Tesla, Inc., the CEO and founder of Neuralink, and the CEO and founder of The Boring Company. However, please note that this information may change over time, so it's always good to verify the latest updates.

# (Optional): Deploy app to Embedchain Platform
app.deploy()
# 🔑 Enter your Embedchain API key. You can find the API key at https://app.embedchain.ai/settings/keys/
# ec-xxxxxx

# 🛠️ Creating pipeline on the platform...
# 🎉🎉🎉 Pipeline created successfully! View your pipeline: https://app.embedchain.ai/pipelines/xxxxx

# 🛠️ Adding data to your pipeline...
# ✅ Data of type: web_page, value: https://www.forbes.com/profile/elon-musk added successfully.

Examples

LLM	Google Colab	Replit
OpenAI
Anthropic
Azure OpenAI
VertexAI
Cohere
Hugging Face
JinaChat
GPT4All
Llama2

Embedding model	Google Colab	Replit
OpenAI
VertexAI
GPT4All
Hugging Face

Vector DB	Google Colab	Replit
ChromaDB
Elasticsearch
Opensearch
Pinecone

🤝 Contributing

Contributions are welcome! Please check out the issues on the repository, and feel free to open a pull request. For more information, please see the contributing guidelines.

For more reference, please go through Development Guide and Documentation Guide.

Telemetry

We collect anonymous usage metrics to enhance our package's quality and user experience. This includes data like feature usage frequency and system info, but never personal details. The data helps us prioritize improvements and ensure compatibility. If you wish to opt-out, set the app.config.collect_metrics = False in the code. We prioritize data security and don't share this data externally.

Citation

If you utilize this repository, please consider citing it with:

@misc{embedchain,
  author = {Taranjeet Singh, Deshraj Yadav},
  title = {Embedchain: Data platform for LLMs - load, index, retrieve, and sync any unstructured data},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/embedchain/embedchain}},
}