GithubHelp home page GithubHelp logo

kssteven418 / biglittledecoder Goto Github PK

View Code? Open in Web Editor NEW
81.0 6.0 9.0 102.83 MB

[NeurIPS'23] Speculative Decoding with Big Little Decoder

Home Page: https://arxiv.org/pdf/2302.07863.pdf

License: Apache License 2.0

Python 99.26% Makefile 0.01% Dockerfile 0.06% Jsonnet 0.01% Shell 0.09% C++ 0.04% Cuda 0.51% C 0.02% MDX 0.02%
decoding efficient-inference fast-inference llm speculative-execution speculative-decoding

biglittledecoder's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

biglittledecoder's Issues

How to import T5_BiLD model in run_translation task.

System Info

I try from transformers.models.t5.modeling_t5 import T5_BiLDModel, but it doesn't work. I build the library from transformer repo.

Who can help?

No response

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

Just run the translation task. CUDA_VISIBLE_DEVICES=0 python run_bild_translation.py --model bild --small /nobackup/haozhang/BigLittleDecoder/models/smallmodel --large /nobackup/haozhang/BigLittleDecoder/models/bigmodel
--dataset_name iwslt2017 --dataset_config iwslt2017-de-en --source_lang de --target_lang en --bild_rollback 3 --bild_fallback 3

Expected behavior

It can not import the T5BiLDModel

[Solved] Missing 'T5BiLDModel' in the official transformers packages.

System Info

ImportError: cannot import name 'T5BiLDModel' from 'transformers.models.t5.modeling_t5' when run_bild_translation.py

Who can help?

No response

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

CUDA_VISIBLE_DEVICES=0 python run_bild_translation.py --model bild --small ../../models/mT5-small-iwslt2017-de-en.bin --large ../../models/mT5-large-iwslt2017-de-en.bin --dataset_name iwslt2017 --dataset_config iwslt2017-de-en --source_lang de --target_lang en --bild_rollback 3 --bild_fallback 2

Expected behavior

build successfully

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.