nelson-liu/lost-in-the-middle
Code and data for "Lost in the Middle: How Language Models Use Long Contexts"
@stanfordnlp CS PhD student.
Repositories
nelson-liu/cython-crash-course
A quick intro to Cython for Python users who don't know C
nelson-liu/llama-recipes
Scripts for fine-tuning Llama2 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization & question answering. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment.Demo apps to showcase Llama2 for WhatsApp & Messenger
nelson-liu/website
nelson-liu/pytorch-manylinux-binaries
nelson-liu/phonesthemes
Code for the paper "Discovering Phonesthemes with Sparse Regularization", to be presented at the NAACL 2018 Workshop on Subword and Character Level Models in NLP.
nelson-liu/paraphrase-id-tensorflow
Various models and code (Manhattan LSTM, Siamese LSTM + Matching Layer, BiMPM) for the paraphrase identification task, specifically with the Quora Question Pairs dataset.
nelson-liu/evaluating-verifiability-in-generative-search-engines
Companion repo for "Evaluating Verifiability in Generative Search Engines".
nelson-liu/inoculation-by-finetuning
Code for the paper "Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets", to be presented at NAACL 2019.
nelson-liu/contextual-repr-analysis
A toolkit for evaluating the linguistic knowledge and transferability of contextual representations. Code for "Linguistic Knowledge and Transferability of Contextual Representations" (NAACL 2019).
nelson-liu/lexical-semantic-recognition
nelson-liu/rbloom-gcs
A fast, simple and lightweight Bloom filter library for Python, implemented in Rust.
nelson-liu/datatrove
Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.
nelson-liu/warcio
Streaming WARC/ARC library for fast web archive IO
nelson-liu/cloud-storage-rs
A crate for uploading files to Google cloud storage, and for generating download urls.
nelson-liu/flatten_gigaword
Dump the text of the Gigaword dataset into a single file, for use with language modeling (and other!) toolkits
nelson-liu/codalab-worksheets
A collaborative platform for reproducible research (web interface and CLI).
nelson-liu/downshift
🏎 A set of primitives to build simple, flexible, WAI-ARIA compliant React autocomplete, combobox or select dropdown components.
nelson-liu/transformers
🤗 Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch.
nelson-liu/ReadabiliPy
A simple HTML content extractor in Python. Can be run as a wrapper for Mozilla's Readability.js package or in pure-python mode.
nelson-liu/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
nelson-liu/chatnoir-resiliparse
A robust web archive analytics toolkit
nelson-liu/word2color
Given a description of a color, return its closest standard HTML4 color.
nelson-liu/alpaca_eval
An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.
nelson-liu/toma
Helps you write algorithms in PyTorch that adapt to the available (CUDA) memory
nelson-liu/HolisticTraceAnalysis
A library to analyze PyTorch traces.
nelson-liu/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
nelson-liu/pandas
Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
nelson-liu/simple-wikidata-db
A set of Python scripts for preprocessing the Wikidata JSON dump and running simple queries in an efficient manner.
nelson-liu/react-hash-link
Painless hash link routing for React applications.