Topic: llm-serving Goto Github
Some thing interesting about llm-serving
Some thing interesting about llm-serving
llm-serving,RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Organization: alibaba
llm-serving,A collection of hand on notebook for LLMs practitioner
User: antoniogr7
llm-serving,A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
User: asprenger
llm-serving,It is a comprehensive resource hub compiling all LLM papers accepted at the International Conference on Learning Representations (ICLR) in 2024.
User: azminewasi
llm-serving,The easiest way to serve AI apps and models - Build reliable Inference APIs, LLM apps, Multi-model chains, RAG service, and much more!
Organization: bentoml
Home Page: https://bentoml.com
llm-serving,Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.
Organization: bentoml
Home Page: https://bentoml.com
llm-serving,Lightweight and extensible LLM Inference serving benchmark tool written in Rust.
Organization: centml
llm-serving,🪶 Lightweight OpenAI drop-in replacement for Kubernetes
User: chenhunghan
llm-serving,An unofficial Go port of the official Tavily API Python Wrapper.
User: diverged
llm-serving,A self-hosted personal chatbot API with FastAPI. It allows you to interact with the Llama2 LLM (and other open-source LLMs) to have natural language conversations, generate text, and perform various language-related tasks.
User: ehsanghaffar
llm-serving,EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU
Organization: embeddedllm
llm-serving,A Production-Ready, Scalable RAG-powered LLM-based Context-Aware QA App
User: fork123aniket
llm-serving,A library to benchmark LLMs via their API exposure
Organization: france-travail
llm-serving,A REST API for vLLM, production ready
Organization: france-travail
Home Page: https://france-travail.github.io/happy_vllm/
llm-serving,Friendli: the fastest serving engine for generative AI
Organization: friendliai
Home Page: https://friendli.ai
llm-serving,Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inference acceleration, and related works will be gradually added in the future. Welcome contributions!
User: galeselee
Home Page: https://galeselee.gitbook.io/awesome-papers/
llm-serving,LLM Security Platform Docs
Organization: genia-dev
Home Page: https://docs.vibraniumdome.com
llm-serving,Multi-node production AI stack. Run the best of open source AI easily on your own servers. Create your own AI by fine-tuning open source models. Integrate LLMs with APIs. Run gptscript securely on the server
Organization: helixml
Home Page: https://tryhelix.ai
llm-serving,Efficient AI Inference & Serving
Organization: hpcaitech
Home Page: https://hpc-ai.com/
llm-serving,A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of vLLM).
User: interestinglsy
llm-serving,Deno LLM API Service
User: ivanlulyf
Home Page: https://bunny-llm.deno.dev
llm-serving,internet llm - access your ollama (or any other local llm) instance from across the internet
User: ivynya
llm-serving,A Overview of Efficiently Serving Large Language Models across Edge Devices
User: jason-cs18
llm-serving,本项目旨在分享大模型相关技术原理以及实战经验。
User: liguodongiot
Home Page: https://www.zhihu.com/column/c_1456193767213043713
llm-serving,Hinglish Chatbot powered by Azure Cognitive Services, Google Translate and Open AI
User: loopglitch26
Home Page: https://hinglish-chatbot-loopglitch26.streamlit.app/
llm-serving,A collection of all available inference solutions for the LLMs
User: mani-kantap
llm-serving,This repository demonstrates LLM execution on CPUs using packages like llamafile, emphasizing low-latency, high-throughput, and cost-effective benefits for inference and serving.
User: mddunlap924
llm-serving,AICI: Prompts as (Wasm) Programs
Organization: microsoft
llm-serving,A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine
Organization: mosecorg
Home Page: https://mosec.readthedocs.io/
llm-serving,A Framework For Intelligence Farming
Organization: neural-dragon-ai
llm-serving,Awesome-LLM-Productization: a curated list of tools/tricks/news/regulations about AI and Large Language Model (LLM) productization
Organization: oscinis-com
llm-serving,Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Organization: predibase
Home Page: https://loraexchange.ai
llm-serving,Ray and Anyscale for UC Berkeley AI Hackathon!
Organization: ray-project
Home Page: https://ai.calhacks.io/
llm-serving,
Organization: ray-project
llm-serving,Deploy and Scale LLM-based applications
Organization: ray-project
Home Page: https://home.mlops.community/public/events/ray-workshop-2023-06-15
llm-serving,Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Organization: ray-project
Home Page: https://ray.io
llm-serving,This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.
Organization: ray-project
llm-serving,RayLLM - LLMs on Ray
Organization: ray-project
Home Page: https://aviary.anyscale.com
llm-serving,LLM (Large Language Model) FineTuning
User: rohan-paul
llm-serving,SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
Organization: skypilot-org
Home Page: https://skypilot.readthedocs.io
llm-serving,Run GPU inference and training jobs on serverless infrastructure that scales with you.
Organization: slai-labs
Home Page: https://beam.cloud
llm-serving,
User: stosan
Home Page: https://code-commentator.onrender.com/
llm-serving,Finetune LLMs on K8s by using Runbooks
Organization: substratusai
Home Page: https://www.substratus.ai
llm-serving,npm like package ecosystem for Prompts 🤖
Organization: sugarcane-ai
Home Page: https://sugarcaneai.dev
llm-serving,
Organization: sugarcane-ai
Home Page: https://sugarcaneai.dev
llm-serving,You can run any large language model on your local machine with this repository.
User: suleymansevimli
llm-serving,Superduper: Bring AI to your database! Integrate AI models and workflows with your database to implement custom AI applications, without moving your data. Including streaming inference, scalable model hosting, training and vector search.
Organization: superduper-io
Home Page: https://superduper.io
llm-serving,Serving Inside Pytorch
Organization: torchpipe
Home Page: https://torchpipe.github.io/
llm-serving,A high-throughput and memory-efficient inference and serving engine for LLMs
Organization: vllm-project
Home Page: https://docs.vllm.ai
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.