uw-ssec / tutorials

SSEC tutorials for various topics

ℹ️ SciPy Tutorial Attendees: You can find the tutorial materials in the SciPy2024 directory or click here.

Tutorials

This is the repository for a Jupyter Book website with tutorial materials for the University of Washington Scientific Software Engineering Center (UW-SSEC). The tutorials are written in Jupyter notebooks and are organized by topic or workshop.

The website is hosted at https://uw-ssec-tutorials.readthedocs.io.

Github Codespaces

This tutorial is available to run within Github Codespaces - "a development environment that's hosted in the cloud".

A codespace is a development environment that's hosted in the cloud. GitHub currently gives every user 120 vCPU hours per month for free, beyond that you must pay. So be sure to explicitly stop or shut down your codespace when you are done by going to this page (https://github.com/codespaces).

Contributing

If you would like to contribute to the tutorials, please feel free to open up issues or pull requests, regarding current or future materials.

Thanks to our contributors so far!

tutorials's Issues

chore: Research on differences b/w using Qdrant(VectorDB) vs FAISS(Vector Library)

          I've implemented the vector DB part, could you try explore the vector library?

Originally posted by @lsetiawan in #16 (comment)

feat: Create Introduction to Language Model Notebook

chore: Convert slides to Jupyter nb format

Using https://rise.readthedocs.io/en/latest/ convert the tutorial slides to Jupyter notebook.

chore: Verify Python Environment Setup

https://uw-ssec-tutorials.readthedocs.io/en/latest/SciPy2024/module1/setup.html

feat: Tutorial Module 2: Introduction to RAG to Improve LLM’s Reliability

(VectorDB and Document Embedding-Retrieval Concepts)

feat: Tutorial Module 2: Prompt Testing to Invoke OLMo with Context

feat: Tutorial Module 2: Prompt Testing to Invoke OLMo and show its hallucinations

chore: Research on the best embedding models for the tutorial

Overview

There are so many embedding models as seen here: https://huggingface.co/sentence-transformers. This piece is critical on creating the best vector, which then gets feed into the VectorDB. Figuring out the most optimized embedding model is key here and probably having a good knowledge will be beneficial for participants, so let's look into it. @vanitech and @anantmittal do you have any insights on this?

Resources

• https://huggingface.co/blog/getting-started-with-embeddings
• https://github.com/reichenbch/RAG-examples/tree/main

feat: Tutorial Module 1: LLMs and Prompt Engineering

feat: Add data from full astro-ph arXiv to the RAG based model

Create a separate vectorDB with a larger astro dataset from https://arxiv.org/archive/astro-ph

(Will add test prompts to this issue shortly)

chore: Update document loaders to allow users to upload their own publications

Update arxiv loader to load multiple PDFs from a zip file
https://uw-ssec-tutorials.readthedocs.io/en/latest/SciPy2024/appendix/langchain-document-loaders.html

feat: Create Embedding Tutorial

Overview

After @anujsinha3 have done such a thorough investigation on embedding models on issue #6, we need to put together a comprehensive tutorial on this.

Outline

• What are embeddings?
• Importance of embeddings in RAG applications
• How to choose the right open-source embedding model for your RAG application.
• Evaluating embedding models: what to look out for? performance considerations?

feat: Tutorial Module 1: LLM-Powered Application Framework (LangChain)

feat: Create tutorial on parsing arXiv dataset

Overview

@vanitech presented a really great dataset that would be really beneficial to be inserted into the vector database for use as context for the RAG approach. This dataset is essentially all of the papers in arXiv: https://www.kaggle.com/datasets/Cornell-University/arxiv/data

After some investigation it looks like this data lives in Google Cloud Storage, so it can easily be retrieved with fsspec without any credentials!

The kaggle repository does contain an index of this data, we can either just grab the data from gcs or directly from arXiv using langchain arXiv loader: https://python.langchain.com/docs/integrations/document_loaders/arxiv

chore: Add word wrapping to text sections as necessary

Especially natural language outputs, needing to scroll to the side seems to (to me, at least) diminish readability.

In addition, restructuring code line lengths with a limit of 80 characters (where/when it makes sense) would likely let code snippets be displayed without needing to scroll to the side.

feat: Tutorial Module 2: Compare and Review LLM Results with and without Context

feat: Tutorial Module 2: Domain-Specific Question and Answer Examples

Beginners trying to build knowledge about specific scientific software (e.g., Astropy, Biopython, Scikit-Learn, GROMACS, icepyx, Echopype, etc.) have questions ranging from novice to expert level.

feat: Tutorial Module 3: Try with your data!

feat: Incorporate astro questions into the tutorial

Questions to ask

• What observations show evidence for dark matter
• What are the most popular theories for what dark matter might be made of
• Does dark matter need to be a particle
• What is the expected mass of dark matter
• What experiments are currently ongoing and searching for dark matter
• How do they work
• What type of dark matter would the LHC be able to detect
• Can you modify gravity to explain dark matter. If so how

• How do astronomers detect the presence of planets around nearby stars
• What is an eclipsing binary star
• Can you describe a taxonomy for different classes of variable stars
• What would the light curve of a tidal disruption event look like. How would it be different from a super nova
• How do supernova explode
• How is a Type II SN different from a Type 1a

TODO: define evaluation criteria

chore: Update RAG example to astro related

Need to update the RAG example in https://cfp.scipy.org/2024/talk/BZGQMC/ to be with the astro stuff that we're working on.

chore: Verify Astropy GitHub Fetch Method uses GitHub Personal Access token

chore,feat: Go through notebooks and create abstractions for repeated code

feat: Tutorial Module 2: Data Loading

(e.g., GitHub repositories, research manuscripts, software documentation, etc.), Chunking, and Embedding

feat: Add a way to symlink from shared folder structure

Overview

In nebari, the data will live in the shared folder, we need to symlink to the user cache so that the pathing all still works.

feat: Tutorial Module 1: Overview

feat: Tutorial Module 1: Introduction to Language Models

chore: Update assets

The current qdrant database contains the abstracts. Add Astropy's GitHub documentation and update the asset.

https://github.com/uw-ssec/tutorials-data

chore: Be consistent with embedding model

Ensure consistent usage of either sentence-transformers/all-MiniLM-L12-v2 or sentence-transformers/all-MiniLM-L6-v2.

feat: Tutorial Module 1: Demo Open-Source LLMs using OLMo

Ask it several questions to show its capacity (e.g., ability to answer fact-based questions, when and where OLMo hallucinates, etc.)

ci: Setup Jupyterbook

Overview

Need to setup jupyterbook infrastructure to build and host this repo to readthedocs!

feat: Tutorial Module 3: Publish your code on GitHub

feat: Create tutorial notebook from prototype

@lsetiawan has created a prototype for Module 2 of the scipy tutorial at https://gist.github.com/lsetiawan/ed5d45f037970515e75db0add987a1aa

Using this as a starting point, create the tutorial and write clear descriptions for each step.
Ensure coverage for the tasks outlined under the Module 2 description at https://github.com/uw-ssec/tutorials/tree/main/SciPy2024

feat: Tutorial Module 2: Setup LLM pipeline with LangChain

feat: Tutorial Module 2: Data Retriever Setup for Adding Context to LLM

feat: Tutorial Module 3: Expose your LLM as a Chat Application through CLI / Webapp

feat: Create tutorial on loading and trying out OLMo using transformers

Overview

The first step to demo a local LLM can be just downloading the model, loading it with transformers library, and then perform a small inference.

feat: Tutorial Module 1: Python Environment Setup

chore: Research on a simple, yet powerful way to put a RAG application together

Overview

Module 3 involves creating an application with all the various components of RAG. This is more of the frontend or api usage for the users that will be using this system. We need to figure out what would be the easiest way to get started. Some thoughts may be a CLI application with typer or web app with streamlit.

ci: Setup Github Codespaces to have all dependecies installed

Overview

To ease setup for users, maybe it's good to have the option of github codespaces, so even local users that have docker can quickly get up and running.

chore: Research on the best Vector DB for this tutorial

Overview

We need to figure out which of these vector db is optimized to be in use for the tutorials. Consideration would be:

• is it open source?
• can we deploy locally?
• does it integrate well with langchain?
• is it performant?

Note that the 3 databases below is the fairly popular one and easy to use, and seem to meet all of the criteria above.

Databases to consider

• https://qdrant.tech/
• https://faiss.ai/
• https://www.trychroma.com/

feat: Explore Stopping Criteria for OLMo Text Generation