Comments (4)
为什么我安装的环境,编译时会有bug?你遇到过吗?
from lightseq.
编译我通过了,跑不了他给的example,这库不行, 别纠结了
from lightseq.
AssertionError: CUDA_HOME does not exist, unable to compile CUDA op(s)
有人遇到一样的问题吗?安装了pytorch==2.1.0
+ cuda11
,竟然说找不到CUDA……
跑的知乎上的demo
import torch
from lightseq.training.ops.pytorch.transformer_encoder_layer import LSTransformerEncoderLayer
def train(model, inputs, masks):
inputs = inputs.to(device="cuda:0")
masks = masks.to(device="cuda:0")
model.to(device="cuda:0")
model.train()
opt = torch.optim.Adam(model.parameters(), lr=1e-3)
for epoch in range(1000):
opt.zero_grad()
outputs = model(inputs, masks)
loss = torch.square(outputs).mean()
loss.backward()
opt.step()
if epoch % 200 == 0:
print("epoch {:>3d}: loss = {:>5.3f}".format(epoch, loss))
if __name__ == "__main__":
# 定义LightSeq配置
config = LSTransformerEncoderLayer.get_config(
max_batch_tokens=4096,
max_seq_len=256,
hidden_size=1024,
intermediate_size=4096,
nhead=16,
attn_prob_dropout_ratio=0.1,
activation_dropout_ratio=0.1,
hidden_dropout_ratio=0.1,
pre_layer_norm=True,
fp16=False,
local_rank=0
)
# 随机生成输入
bsz, sl = 10, 80
inputs = torch.randn(bsz, sl, config.hidden_size)
masks = torch.zeros(bsz, sl)
# 定义LightSeq编码层并进行训练
model = LSTransformerEncoderLayer(config)
train(model, inputs, masks)
from lightseq.
from lightseq.
Related Issues (20)
- ls_torch_hf_quant_gpt2_export.py的使用问题
- lightseq' Transformer expects an extra layer_norm on both encoder and decoder level
- LLaMA example 结果验证
- 请问lightseq在推理流程中有gemm调参这一步吗?
- Is llama inference available now? HOT 1
- Do you have plans to support token_type_ids?
- llama inference test HOT 3
- 请问lightseq可以支持segmentAnyting的推理优化吗 HOT 1
- [Question] gptj, mpt support.
- question about environment
- how to resolve xlm-roberta convert fail
- Can int8 in pre-training large model ???
- lightseq是否支持clip模型的int8量化?
- Is it normal that A10 inference speed is lower than 2080ti? HOT 1
- identifier "__hisnan" is undefined HOT 4
- 要求C++ 17 HOT 1
- Exception: Installed CUDA version 12.3 does not match the version torch was compiled with 12.1, unable to compile cuda/cpp extensions without a matching cuda version.
- 参数格式不对args format wrong
- is Baichuan2 model inference available now?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lightseq.