vidhyavarshanyjs / ensemblex

EnsembleX utilizes the Knapsack algorithm to optimize Large Language Model (LLM) ensembles for quality-cost trade-offs, offering tailored suggestions across various domains through a Streamlit dashboard visualization.

EnsembleX

A dynamic Language Model (LLM) ensemble selection application that leverages the Knapsack algorithm to optimize model selection across various domains. By integrating real-time web interaction with the Streamlit framework, users can receive cost optimization suggestions tailored to each selected model based on their dataset inputs.

📝 Project Description

EnsembleX collects diverse domain LLM performance datasets from the open-ll-leaderboard API provided by Hugging Face. It then processes and manipulates the data, applies the Knapsack algorithm for model selection optimization, and visualizes the results using Streamlit. The core objective of EnsembleX is to empower users with a data-driven approach to selecting LLM ensembles that balance performance quality and cost efficiency.

🚀 Demo

EnsembleX.Demo.Video.mp4

⬆️ Project Workflow

1. Data Collection: Diverse domain LLM performance datasets are collected from the open-llm-leaderboard API.
2. Data Manipulation: The collected data is processed and manipulated to prepare it for model selection optimization.
3. Knapsack Algorithm: The Knapsack algorithm is applied to optimize model selection within a constrained framework.
4. Real-time Web Application: Integration of a real-time web application using the Streamlit framework enables user interaction and visualization of the optimized model selection results.
5. Cost Optimization Suggestions: Individualized cost optimization suggestions for each selected model are provided based on user-provided dataset inputs.

🧐 Features

• Dynamic LLM ensemble selection
• Knapsack algorithm for quality-cost optimization
• Real-time cost optimization suggestions
• Data-driven model selection process

💻 Built with

• Python
• Streamlit
• Pandas
• NumPy
• Hugging Face API
• Plotly
• Gradio
• YData Profiling

🛠️ Installation Steps:

1. Clone the repository: git clone https://github.com/yourusername/yourproject.git
2. Install requirements: pip install -r requirements.txt
3. Run the app: streamlit run app.py

---

🛡️ License

This project is licensed under the MIT License - see the LICENSE file for details.

Reference

• Hugging Face Open LLM Leaderboard
• FelixZ Open LLM Leaderboard
• Which LLM is Better? Open LLM Leaderboard
• Managing LLM context is the knapsack problem