shyhuai Goto Github PK

adahessian

ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning

attention-is-all-you-need-pytorch

A PyTorch implementation of the Transformer model in "Attention is All You Need".

b-caffe

MG-WFBP: Efficient Data Communication for Distributed Synchronous SGD Algorithms

byteps

A high performance and generic framework for distributed DNN training

caffe

Caffe: a fast open framework for deep learning.

dataprovider.torch

Data providers for Torch

ddl-benchmarks-1

ddl-benchmarks: Benchmarks for Distributed Deep Learning

deepspeechrecognition

A Chinese Deep Speech Recognition System 包括基于深度学习的声学模型和基于深度学习的语言模型

distnerf

A PyTorch implementation of NeRF (Neural Radiance Fields) that reproduces the results.

dl_scheduling

eva

[ICLR 2023] Eva: Practical Second-order Optimization with Kronecker-vectorized Approximation

fadnet

fiddler

Fast Inference of MoE Models with CPU-GPU Orchestration

gaussiank-sgd

Understanding Top-k Sparsification in Distributed Deep Learning

gblastn

G-BLASTN is a GPU-accelerated nucleotide alignment tool based on the widely used NCBI-BLAST.

gtopkssgd

gTop-k S-SGD: A Communication-Efficient Distributed Synchronous SGD Algorithm for Deep Learning

hkbu-benchmark

A git repository containing the scripts used in http://arxiv.org/abs/1608.07249

horovod

Distributed training framework for TensorFlow, Keras, and PyTorch.

kfac_pytorch

Distributed K-FAC Preconditioner for PyTorch

mpi4py

Python bindings for MPI that supports ULFM

nccl

Optimized primitives for collective multi-GPU communication

openai-gemm

Open single and half precision gemm implementations

optimus-cc

[ASPLOS'23] Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression

pipedream

pysyft

A library for encrypted, privacy preserving deep learning - based on PyTorch

pytorch-sso

PyTorch-SSO: Scalable Second-Order methods in PyTorch

tensorflow

Computation using data flow graphs for scalable machine learning

thundersvm

ThunderSVM: A Fast SVM Library on GPUs and CPUs

training

Reference implementations of MLPerf™ training benchmarks

tutel

Tutel MoE: An Optimized Mixture-of-Experts Implementation