GithubHelp home page GithubHelp logo

daveparr / starpilot Goto Github PK

View Code? Open in Web Editor NEW
48.0 2.0 4.0 1.88 MB

Use you a GitHub stars for great good!

Home Page: https://www.daveparr.info/blog/copilot-for-your-github-stars-1cep/

License: MIT License

Python 100.00%
ai artificial-intelligence cli command-line-tool github github-stars langchain langchain-python

starpilot's Introduction

Starpilot is like copilot, but for GitHub stars.

I've been starring repos for years thinking "This will definitely be useful later".

However I never really went back to them.

Starpilot helps this problem by allowing you to rediscover GitHub repos you had previously starred that are relevant to your current project.

Starpilot is a retrieval augmented generation CLI tool for rediscovering your GitHub stars.

Starpilot uses Large Language Models to query your GitHub stars and return the most relevant repos to your query.

❯ starpilot astrologer "How can I orchestrate llms?"
                                                   Source Documents                                                    
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Repo      ┃ Description             ┃ URL                      ┃ Topic                   ┃ Language                 ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Flowise   │ Drag & drop UI to build │ https://github.com/Flow… │ artificial-intelligence │ TypeScript JavaScript    │
│           │ your customized LLM     │                          │ chatgpt                 │ CSS SCSS HTML Dockerfile │
│           │ flow                    │                          │ large-language-models   │ Shell Batchfile          │
│           │                         │                          │ llm low-code no-code    │                          │
│           │                         │                          │ hacktoberfest           │                          │
│ langchain │ ⚡ Building             │ https://github.com/lang… │                         │ Python Jupyter Notebook  │
│           │ applications with LLMs  │                          │                         │ MDX Shell Makefile XSLT  │
│           │ through composability   │                          │                         │ HTML Dockerfile TeX      │
│           │ ⚡                      │                          │                         │ JavaScript               │
│ motorhead │ 🧠 Motorhead is a       │ https://github.com/getm… │ machine-learning ml     │ Rust Dockerfile          │
│           │ memory and information  │                          │ mlops rust llmops llms  │                          │
│           │ retrieval server for    │                          │                         │                          │
│           │ LLMs.                   │                          │                         │                          │
│ lancedb   │ Developer-friendly,     │ https://github.com/lanc… │ embeddings llms         │ Python Rust TypeScript   │
│           │ serverless vector       │                          │ vector-search           │ JavaScript Shell         │
│           │ database for AI         │                          │                         │ Dockerfile PowerShell    │
│           │ applications. Easily    │                          │                         │                          │
│           │ add long-term memory to │                          │                         │                          │
│           │ your LLM apps!          │                          │                         │                          │
└───────────┴─────────────────────────┴──────────────────────────┴─────────────────────────┴──────────────────────────┘

Here's some more details about the motivation for and state of the project.

Here is a talk I gave about the project

Installation

experimental

This project is in early development and is not yet available on PyPi

To install the latest version from GitHub run:

pip install git+https://github.com/DaveParr/starpilot --user

To install the latest tagged release (potentially the more stable approach) from GitHub run:

pip install git+https://github.com/DaveParr/[email protected] --user

Setup

You will need to have a .env file with

  • a GitHub personal access token saved to a .env file in the root of the project. This should have the user> read:user scope permission.
  • a OpenAI API key saved to a .env file in the root of the project. This is needed for Self Query in the astrologer command, not semantic similarity via the shoot command.
GITHUB_API_KEY="ghp_..."
OPENAI_API_KEY="sk-..."

The command starpilot setup will help you set this up.

Optional

This not required for the CLI to work, but is required if read is set to include the README.MD content for each repo.

You may potentially need Pandoc installed on your computer if your starred repos contain a rst formatted Readme that you want to load into the database which is used by some Python projects.

Usage

shoot for simple semantic search

asciicast

astrologer for more complex self querying

asciicast

Commands

❯ starpilot --help
                                                                                                                              
 Usage: starpilot [OPTIONS] COMMAND [ARGS]...                                                                                 
                                                                                                                              
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --install-completion          Install completion for the current shell.                                                    │
│ --show-completion             Show completion for the current shell, to copy it or customize the installation.             │
│ --help                        Show this message and exit.                                                                  │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ astrologer   A self-query of the vectorstore that allows the user to search for a repo while filtering by attributes       │
│ read         Read stars from GitHub                                                                                        │
│ setup        Setup the CLI with the required API keys                                                                      │
│ shoot        An embedding search of the vectorstore                                                                        │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

More examples

❯ starpilot astrologer "How do I use dataframes?"
                                                                               Source Documents                                                                                
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Repo       ┃ Description                ┃ URL                                      ┃ Topic                     ┃ Primary Language ┃ Languages                  ┃ Star Count ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ data.table │ R's data.table package     │ https://github.com/Rdatatable/data.table │                           │ R                │ R C Batchfile Shell        │ 3451       │
│            │ extends data.frame:        │                                          │                           │                  │ Makefile C++ CSS           │            │
│            │                            │                                          │                           │                  │ Dockerfile                 │            │
│ tibble     │ A modern re-imagining of   │ https://github.com/tidyverse/tibble      │ r tidy-data               │ R                │ R C Mermaid                │ 641        │
│            │ the data frame             │                                          │                           │                  │                            │            │
│ pandas     │ Flexible and powerful data │ https://github.com/pandas-dev/pandas     │ data-analysis pandas      │ Python           │ Python Shell HTML C Smarty │ 41377      │
│            │ analysis / manipulation    │                                          │ flexible alignment python │                  │ CSS Dockerfile XSLT Cython │            │
│            │ library for Python,        │                                          │ data-science              │                  │ Meson                      │            │
│            │ providing labeled data     │                                          │                           │                  │                            │            │
│            │ structures similar to R    │                                          │                           │                  │                            │            │
│            │ data.frame objects,        │                                          │                           │                  │                            │            │
│            │ statistical functions, and │                                          │                           │                  │                            │            │
│            │ much more                  │                                          │                           │                  │                            │            │
└────────────┴────────────────────────────┴──────────────────────────────────────────┴───────────────────────────┴──────────────────┴────────────────────────────┴────────────┘
❯ starpilot astrologer "How do I use dataframes in Rust?"
                                                                               Source Documents                                                                                
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Repo      ┃ Description                 ┃ URL                                    ┃ Topic                      ┃ Primary Language ┃ Languages                   ┃ Star Count ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ polars    │ Dataframes powered by a     │ https://github.com/pola-rs/polars      │ dataframe-library          │ Rust             │ Rust Python Makefile R CSS  │ 24592      │
│           │ multithreaded, vectorized   │                                        │ dataframe dataframes rust  │                  │                             │            │
│           │ query engine, written in    │                                        │ arrow python out-of-core   │                  │                             │            │
│           │ Rust                        │                                        │ polars                     │                  │                             │            │
│ weld      │ High-performance runtime    │ https://github.com/weld-project/weld   │ stanford data analytics    │ Rust             │ Rust Makefile C++ Python C  │ 2980       │
│           │ for data analytics          │                                        │ machine-learning           │                  │ Shell Batchfile             │            │
│           │ applications                │                                        │ code-generation            │                  │                             │            │
│           │                             │                                        │ performance rust llvm      │                  │                             │            │
│           │                             │                                        │ pandas                     │                  │                             │            │
│ polars-ai │ 💬 Chat with your Polars    │ https://github.com/wiseaidev/polars-ai │ cli dataframe evcxr        │ Rust             │ Rust                        │ 3          │
│           │ DataFrame from your CLI and │                                        │ jupyter openai polars rust │                  │                             │            │
│           │ your app! (WIP)             │                                        │ polars-ai                  │                  │                             │            │
└───────────┴─────────────────────────────┴────────────────────────────────────────┴────────────────────────────┴──────────────────┴─────────────────────────────┴────────────┘
❯ starpilot astrologer "Suggest dataframe packages on CRAN"
                                                                               Source Documents                                                                                
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Repo       ┃ Description                       ┃ URL                                      ┃ Topic       ┃ Primary Language ┃ Languages                         ┃ Star Count ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ tibble     │ A modern re-imagining of the data │ https://github.com/tidyverse/tibble      │ r tidy-data │ R                │ R C Mermaid                       │ 641        │
│            │ frame                             │                                          │             │                  │                                   │            │
│ data.table │ R's data.table package extends    │ https://github.com/Rdatatable/data.table │             │ R                │ R C Batchfile Shell Makefile C++  │ 3451       │
│            │ data.frame:                       │                                          │             │                  │ CSS Dockerfile                    │            │
│ broom      │ Convert statistical analysis      │ https://github.com/dgrtwo/broom          │             │ R                │ R                                 │ 19         │
│            │ objects from R into tidy format   │                                          │             │                  │                                   │            │
└────────────┴───────────────────────────────────┴──────────────────────────────────────────┴─────────────┴──────────────────┴───────────────────────────────────┴────────────┘
❯ starpilot astrologer "test api python"
                                                                               Source Documents                                                                                
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Repo      ┃ Description                 ┃ URL                                    ┃ Topic                      ┃ Primary Language ┃ Languages                   ┃ Star Count ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ responses │ A utility for mocking out   │ https://github.com/getsentry/responses │ tag-production             │ Python           │ Makefile Python Shell       │ 4018       │
│           │ the Python Requests         │                                        │                            │                  │                             │            │
│           │ library.                    │                                        │                            │                  │                             │            │
│ pytest    │ The pytest framework makes  │ https://github.com/pytest-dev/pytest   │ unit-testing test testing  │ Python           │ Python Gherkin              │ 11164      │
│           │ it easy to write small      │                                        │ python hacktoberfest       │                  │                             │            │
│           │ tests, yet scales to        │                                        │                            │                  │                             │            │
│           │ support complex functional  │                                        │                            │                  │                             │            │
│           │ testing                     │                                        │                            │                  │                             │            │
│ pokeapi   │ The Pokémon API             │ https://github.com/PokeAPI/pokeapi     │ beginner-friendly api      │ Python           │ Python Makefile Dockerfile  │ 3881       │
│           │                             │                                        │ pokemon pokeapi graphql    │                  │ Shell JavaScript Go         │            │
│           │                             │                                        │ hacktoberfest              │                  │                             │            │
└───────────┴─────────────────────────────┴────────────────────────────────────────┴────────────────────────────┴──────────────────┴─────────────────────────────┴────────────┘
❯ starpilot astrologer "How do I manipulate dates and times in R?"
                                                                               Source Documents                                                                                
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃                ┃                ┃                                                                         ┃                 ┃ Primary        ┃                 ┃            ┃
┃ Repo           ┃ Description    ┃ URL                                                                     ┃ Topic           ┃ Language       ┃ Languages       ┃ Star Count ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ lubridate      │ Make working   │ https://github.com/tidyverse/lubridate                                  │ r date-time     │ R              │ Emacs Lisp R    │ 712        │
│                │ with dates in  │                                                                         │ date            │                │ TeX C Shell CSS │            │
│                │ R just that    │                                                                         │                 │                │ Makefile        │            │
│                │ little bit     │                                                                         │                 │                │                 │            │
│                │ easier         │                                                                         │                 │                │                 │            │
│ Hands-On-Time… │ Hands-On-Time… │ https://github.com/PacktPublishing/Hands-On-Time-Series-Analysis-with-R │                 │ R              │ R               │ 102        │
│ fable          │ Tidy time      │ https://github.com/tidyverts/fable                                      │ forecasting     │ R              │ R C++ C         │ 539        │
│                │ series         │                                                                         │                 │                │                 │            │
│                │ forecasting    │                                                                         │                 │                │                 │            │
└────────────────┴────────────────┴─────────────────────────────────────────────────────────────────────────┴─────────────────┴────────────────┴─────────────────┴────────────┘

starpilot's People

Contributors

daveparr avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

starpilot's Issues

Perform HyDE at query time

Currently the query is tested for similarity on the topics, description and repo name.

An LLM might help the query process using a Hypotehtical document approach, using a prompt such as:
"Based on this query subject, write a short description and set of tags for GitHub project that would help with the intended goal"

Allow multiple users stars to be loaded into the vectorstore

The only reason multiple users star lists can not be used at the same time is because the read function drops the database each time.

It would be better if the database could add a new users data to the collection intelligently by understanding that the user us different ot the previous user.

Enhance document content intelligently

It might be valuable to call an llm during read time to enhance document data. 2 operations might be useful

  • At document creation, before vectorstor embedding, an llm could be tasked to catagorise the content based on user defined tags. e.g. if the repo is to do with data science, or if it is likely a repo from a hackathon and not a module to be used in other projects.
  • To use a form of Hypothetical document embedding at ETL time, e.g. supply the repo readme.md content to a llm with the prompt "based on this readme.md what is the intentended use case for this repository, and what problems does it solve/ tasks does it help with?"

Allow vectors to be updated intelligently

Currently the read function drops the database from the drive and rebuilds it fresh every time.

A better operation would be to check the 'new' information from the API against the json in the repo_content directory, and only run the update operation if the local data is different tto the return from the API.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.