onuralpszr / awesome-osml-for-devs

List of resources, libraries and more for developers who would like to build with open-source machine learning off-the-shelf

Awesome Open-source Machine Learning for Developers

List of resources, libraries and more for developers who would like to build with open-source machine learning off-the-shelf.

Motivation: Developers are often building machine learning with closed-source models behind gated APIs. These models can change by time without developers knowing, companies are giving away their data during inference and have no control over the model nor data.

There are a ton of open-source models out there that can be deployed by developers, but reducing barrier of entry to use these models and making developers aware of them are necessary, so I created this repository to do so.

Using the resources here, you can learn to find the model you need and serve it on the platform of your choice using the tools given here.

Hint: Take a look at foundation models section for one-model-fits-all type of models.

Note: To contribute, send a pull request to this repository. Note that this repository is focused on open-source machine learning.

Table of Contents

• Resources
• Libraries, Platforms and Development Platform-specific Resources
  • Platforms
  • Development Platform
    • Framework
    • Web
    • Mobile
    • Edge
    • Cloud Deployment
    • Serving
    • Game Development
• Modalities and Tasks
  • Foundation Models
  • LLMs
    • Tools
  • Multimodal Models
    • Models and Demos
    • Understanding Image and Text
    • Document AI
  • Generative AI
    • Models and Demos
  • Computer Vision
    • Models and Demos
  • Natural Language Processing
  • Audio
• Advanced

Resources

• Tasks: A documentation project to let developers build their first machine learning based product using models off-the-shelf.
• Open-source AI Cookbook: Recipes and notebooks using open-source models and libraries.

Libraries, Platforms and Development Platform-specific Resources

Platforms

• Hugging Face Hub: Collaborative platform for machine learning. Discover hundreds of thousands of open-source models able to work off-the-shelf in /models.
• Kaggle Models: Discover and use thousands of machine learning models, including the most popular diffusion models and LLMs.
• Pytorch Hub: Discover and publish models to a pre-trained model repository designed for research exploration.

Development Platform

• ONNX Runtime: Platform agnostic model runtime to use ML models.

Framework

• TensorFlow: An end-to-end open source platform for machine learning.
• PyTorch: An open source machine learning framework that accelerates the path from research prototyping to production deployment.

Web

• Transformers.js: A library to run cutting edge models directly in-browser.
• huggingface.js: A library to play with models on Hugging Face Hub through javascript.
• TensorFlow.js: A library for machine learning in JavaScript.
• Mediapipe A framework that has prebuilt and customizable ML solutions, ready to deploy on Web

Mobile

• TensorFlow Lite: A library to deploy models on mobile and edge devices.
• Mediapipe: A framework that has prebuilt and customizable ML solutions, ready to deploy on Android, iOS.
• ExecuTorch: A library for enabling on-device ML in mobile/edge devices for PyTorch models.
• huggingface.dart: A Dart SDK to interact with the models on Hugging Face Hub.
• flutter-tflite: TensorFlow Lite Flutter plugin provides an easy, flexible, and fast Dart API to integrate TFLite models in flutter apps across mobile and desktop platforms.
• NCNN: A high-performance neural network inference framework optimized for the mobile platform.

Edge

• TensorFlow Lite: A library to deploy models on mobile and edge devices.
• ExecuTorch: A library for enabling on-device ML in everywhere from AR/VR wearables to mobile/edge devices for PyTorch models.

Cloud Deployment

Serving

• Text Generation Inference: Toolkit to serve large language models.
• Text Embedding Inference: Toolkit to serve text embeddings.
• TorchServe: Flexible, easy to use and scalable inference server.
• Tensorflow Serving: Flexible, high-performance serving system for machine learning models, designed for production environments.
• Inference: A fast, easy-to-use, production-ready inference server for computer vision supporting deployment of many popular model architectures and fine-tuned models.
• Nvidia Triton Inference Server: Open source inference serving software that streamlines AI inferencing. Triton enables teams to deploy any AI model from multiple deep learning and machine learning frameworks, including TensorRT, TensorFlow, PyTorch, ONNX, OpenVINO, Python, RAPIDS FIL, and more.
• PyTrion: PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments. The library allows serving Machine Learning models directly from Python through NVIDIA's Triton Inference Server.

Game Development

• Unity

  • MediaPipe Unity Plugin: Unity plugin to run MediaPipe. This approach may sacrifice performance when you need to call multiple APIs in a loop, but it gives you the flexibility to use MediaPipe instead.
  • Hugging Face API for Unity 🤗: This Unity package provides an easy-to-use integration for the Hugging Face Inference API, allowing developers to access and use Hugging Face AI models within their Unity projects.

Modalities and Tasks

This section contains powerful models that can generalize well and can be used out-of-the-box.

Foundation Models

The following resources are on zero-shot models: These models take in an image or text and possible classes in those images or texts.

• Zero-shot Object Detection
• Zero-shot Image Classification Resources
• Zero-shot Text Classification Resources

Note: The foundation model can be found under their associated task.

LLMs

Tools

• Text Generation Inference: Toolkit to serve large language models.
• Text Embedding Inference: Toolkit to serve text embeddings.
• TGI Benchmark: Understanding throughput of inference servers, with TGI example.
• Open LLM Leaderboard: A leaderboard to find the open-source LLM for your use and budget.
• Ollama: Get up and running with Llama 2, Mistral, Gemma, and other large language models.
• Open Webui: Open WebUI is an extensible, feature-rich, and user-friendly self-hosted WebUI designed to operate entirely offline. It supports various LLM runners, including Ollama and OpenAI-compatible APIs.

Multimodal Models

• Awesome Open-source Foundation and Multimodal Models: A curated list of models that have high zero-shot capabilities and the models that can take in two different modalities (e.g. a model that can take in image and text and output text).

Models and Demos

• Kosmos-2: Demo for Kosmos-2 model by Microsoft, that can understand image and text, answer questions about images, caption images, detect objects on images and gives answer without hallucinating.
• Fuyu-8B: Demo for Fuyu-8b by Adept, that can understand image and text and answer questions about images and caption images.

Understanding Image and Text

Document AI

• Document AI Collection: A collection on demos and models for document AI.

Generative AI

Models and Demos

• Stable Cascade: An app based on state-of-the-art image generation model Stable Cascade. (as of March '24)
• Stable Diffusion XL Inpainting: An application that can do inpainting when given a text prompt and an image.

Computer Vision

Models and Demos

• OWL: A curation about OWL model released by Google, the most powerful zero-shot object detection model. (as of March '24)
• Segment Anything: A curation about Segment Anything model released by Meta, the most powerful zero-shot image segmentation model. (as of March '24)
• Depth Anything: A highly practical solution for robust monocular depth estimation by training on a combination of 1.5M labeled images and 62M+ unlabeled images.

Natural Language Processing

Audio

• Insanely Fast Whisper: A CLI to shrink and serve Whisper on-device.
• Open TTS Tracker: An awesome repository to keep a track of open-source text-to-speech models.

Advanced

Other

• Raycast Automate commands on macOS apps with a local ollama LLM, with Raycast extensions.