Comments (5)
I tried the command from the triton's open issue and it worked:
!echo /usr/lib64-nvidia/ >/etc/ld.so.conf.d/libcuda.conf; ldconfig
thank you
from mamba.
I'm not familiar with Google Colab, and I'm not sure how well Triton works with V100. Seems like there's also an issue in the triton repo tracking this.
You can try !ldconfig
.
from mamba.
I am using Google Colab Pro+ with V100 GPU. I have followed your example but couldn't get the output because of the error: AssertionError: libcuda.so cannot found! It seems that triton backend is causing the problem:
/usr/local/lib/python3.10/dist-packages/mamba_ssm/ops/triton/layernorm.py in _layer_norm_fwd(x, weight, bias, eps, residual, out_dtype, residual_dtype, is_rms_norm) 153 # heuristics for number of warps 154 with torch.cuda.device(x.device.index): --> 155 _layer_norm_fwd_1pass_kernel[(M,)]( 156 x, 157 y,
/usr/local/lib/python3.10/dist-packages/triton/runtime/autotuner.py in run(self, *args, **kwargs) 98 pruned_configs = self.prune_configs(kwargs) 99 bench_start = time.time() --> 100 timings = {config: self._bench(*args, config=config, **kwargs) 101 for config in pruned_configs} 102 bench_end = time.time()
/usr/local/lib/python3.10/dist-packages/triton/runtime/autotuner.py in (.0) 98 pruned_configs = self.prune_configs(kwargs) 99 bench_start = time.time() --> 100 timings = {config: self._bench(*args, config=config, **kwargs) 101 for config in pruned_configs} 102 bench_end = time.time()
/usr/local/lib/python3.10/dist-packages/triton/runtime/autotuner.py in _bench(self, config, *args, **meta) 81 self.fn.run(*args, num_warps=config.num_warps, num_stages=config.num_stages, **current) 82 try: ---> 83 return do_bench(kernel_call, warmup=self.warmup, rep=self.rep, quantiles=(0.5, 0.2, 0.8)) 84 except OutOfResources: 85 return [float('inf'), float('inf'), float('inf')]
/usr/local/lib/python3.10/dist-packages/triton/testing.py in do_bench(fn, warmup, rep, grad_to_none, quantiles, fast_flush, return_mode) 102 """ 103 --> 104 fn() 105 torch.cuda.synchronize() 106
/usr/local/lib/python3.10/dist-packages/triton/runtime/autotuner.py in kernel_call() 79 config.pre_hook(full_nargs) 80 self.hook(args) ---> 81 self.fn.run(*args, num_warps=config.num_warps, num_stages=config.num_stages, **current) 82 try: 83 return do_bench(kernel_call, warmup=self.warmup, rep=self.rep, quantiles=(0.5, 0.2, 0.8))
in _layer_norm_fwd_1pass_kernel(X, Y, W, B, RESIDUAL, RESIDUAL_OUT, Mean, Rstd, stride_x_row, stride_y_row, stride_res_row, stride_res_out_row, N, eps, IS_RMS_NORM, BLOCK_N, HAS_RESIDUAL, STORE_RESIDUAL_OUT, HAS_BIAS, grid, num_warps, num_stages, extern_libs, stream, warmup, device, device_type)
/usr/local/lib/python3.10/dist-packages/triton/compiler/compiler.py in compile(fn, **kwargs) 423 # cache manager 424 if is_cuda or is_hip: --> 425 so_path = make_stub(name, signature, constants) 426 else: 427 so_path = _device_backend.make_launcher_stub(name, signature, constants)
/usr/local/lib/python3.10/dist-packages/triton/compiler/make_launcher.py in make_stub(name, signature, constants) 37 with open(src_path, "w") as f: 38 f.write(src) ---> 39 so = _build(name, src_path, tmpdir) 40 with open(so, "rb") as f: 41 return so_cache_manager.put(f.read(), so_name, binary=True)
/usr/local/lib/python3.10/dist-packages/triton/common/build.py in _build(name, src, srcdir) 59 hip_include_dir = os.path.join(rocm_path_dir(), "include") 60 else: ---> 61 cuda_lib_dirs = libcuda_dirs() 62 cu_include_dir = cuda_include_dir() 63 suffix = sysconfig.get_config_var('EXT_SUFFIX')
/usr/local/lib/python3.10/dist-packages/triton/common/build.py in libcuda_dirs() 28 msg += 'Possible files are located at %s.' % str(locs) 29 msg += 'Please create a symlink of libcuda.so to any of the file.' ---> 30 assert any(os.path.exists(os.path.join(path, 'libcuda.so')) for path in dirs), msg 31 return dirs 32
AssertionError: libcuda.so cannot found!
How can I solve this on Google Colab environment?
Hello, I have encountered the same problem, have you solved it?
from mamba.
I tried the command from the triton's open issue and it worked: !echo /usr/lib64-nvidia/ >/etc/ld.so.conf.d/libcuda.conf; ldconfig
thank you
Hello, I have encountered the same problem, have you solved it?
from mamba.
For this problem, I found !echo /usr/lib64-nvidia/ >/etc/ld.so.conf.d/libcuda.conf; ldconfig
this not work for me, and I go in triton package and change it directly at here:
/data/App/anaconda3/envs/mamba/lib/python3.10/site-packages/triton/common/build.py:line 25
add this code: dirs.append("/usr/local/cuda/cuda/lib64/stubs")
, this path depands on your own stubs path, and I fixed this problem.
from mamba.
Related Issues (20)
- Question for 'self.use_mem_eff_path and inference_params' HOT 4
- triton.runtime.autotuner.OutOfResources: out of resource: shared memory, Required: 254208, Hardware limit: 101376. HOT 5
- I want to ask does anyone know how to solve this problem
- /anaconda3/lib/python3.11/site-packages/causal_conv1d_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c107WarningC1ENS_7variantIJNS0_11UserWarningENS0_18DeprecationWarningEEEERKNS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEb HOT 1
- Mamba-2 Error: `'NoneType' object has no attribute 'causal_conv1d_fwd'` HOT 8
- Used selective_scan_cuda and causal_conv1d_cuda, but still very slow to train HOT 1
- mamba / self-attention hybrid generation
- Inference multiple tokens HOT 2
- Error when using FP16 or Mixed precision HOT 3
- How to use Mamba2?
- How to extract whole sentence embeddings HOT 1
- Does mamba support data packing?
- Slow Mamba 2 training speeds with higher d_state values HOT 1
- Where is ‘Block’ class in the new version mamba? HOT 1
- mamba_ssm Install Failure HOT 9
- Sequence parallelism in the mixer (Context Parallelism)
- Support Mamba-codestral
- Why does it take so long to build HOT 1
- Is mamba suitable for time-series classification task? HOT 1
- Question on Comparison between Mamba and S4 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mamba.