ppl.nn.llm's People
Forkers
alcanderian hijdk violetevergardenyyh alwaysssssss jonxia vincent-syr mayi140611 rayjue af-74413592 open-lm xiaoxiaowriter bjmsong jianfei-wangg xupinjie ouonline eddiewdy ai-mou leidanw jimurk alignment-lab-aippl.nn.llm's Issues
export模型的时候,报warning,有问题么?
使用命令: OMP_NUM_THREADS=1 torchrun --nproc_per_node
Export.py --ckpt_dir <llama_dir>
--tokenizer_path <llama_tokenizer_dir>/tokenizer.model
--export_path <path_to_store_exported_llama7b>
--fused_qkv 1 --fused_kvcache 1 --auto_causal 1
--quantized_cache 1 --dynamic_batching 1
export 模型的时候,报一些warning信息,这是什么原因?会有什么影响?需要解决么?谢谢
[W shape_type_inference.cpp:1920] Warning: The shape inference of pmx::SwiGLU type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
[W shape_type_inference.cpp:1920] Warning: The shape inference of pmx::RowParallelLinear type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)
benchmark script is only works for single token generation?
How to set the output length for the llama7B benchmark script? It seems that the script is only profiing for single token generation.
如何跑internlm2-20b模型的性能,也是使用pplnn-build/tools/benchmark_llama么?
如何用ppl 跑internlm2-20b模型的性能,也是使用pplnn-build/tools/benchmark_llama么?
怎么设置 kv cache int8 量化, 但 a 和 w 仍然是f16,测试 kvcache 量化的收益
是否支持moe模型?
看到代码里边有一些moe相关的算子。但是没看到moe模型的示例
Is there python binding support ?
There's no python binding related code in repository. Since ppl.nn.llm is ppl.nn, how can I integrate ppl.nn.llm with ppl.nn python binding ? Is it possible to use python to build a LLM chat interaction (or text completion) ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.