Comments (5)
I came into the same issue with gtx1080ti which compute capability is compute_60. In the setup.py file, I see both the mamba_ssm and causal_conv1d only compile the code with compute_70, compute_80 and compute_90 except compute_60 which is corresponding to the sm_60:
Lines 108 to 114 in 86a3a90
So I add two line to both setup.py file of mamba_ssm and causal_conv1d, then reinstall both packages and it works for me.
#add
cc_flag.append("-gencode")
cc_flag.append("arch=compute_60,code=sm_60")
#
cc_flag.append("-gencode")
cc_flag.append("arch=compute_70,code=sm_70")
cc_flag.append("-gencode")
cc_flag.append("arch=compute_80,code=sm_80")
if bare_metal_version >= Version("11.8"):
cc_flag.append("-gencode")
cc_flag.append("arch=compute_90,code=sm_90")
And I think this may be because the Pascal architecture GPU(compute_60) does not support the bf16 data type. And this may cause the functions using bf16 to be unusable.
from mamba.
I built a wheel under the Python 3.10
, PyTorch 2.1
, and CUDA 11.8
environment to support the compute_60. For details, please refer to the link.
@SamsongB @DewEfresh
from mamba.
Have you tried running it on a single gpu? (Maybe a smaller one) If that's the case, I think there are some issues with the custom operators and DDP.
Or, it could be the fact that P40 is missing some kernels that the mamba optimized version uses. On my ampere-series card no probs whatsoever.
from mamba.
I also see the same issue with GM200 [GeForce GTX TITAN X], tried the above solution with compute_60, but still cannot seem to run. is there an update regarding this?
from mamba.
@DewEfresh Hello, my dear. Have you solved your problem? The single card p40 I use also has this problem. Please help me with my cuda version 11.8
from mamba.
Related Issues (20)
- Reducing _chunk_scan_bwd_kernel computation HOT 4
- Exploding gradients if ngroups is higher than 1. HOT 3
- clarification on how to interpret kernel size for conv1d HOT 1
- Some questions about the shape of A,B,C,D HOT 5
- How can I avoid using causal-conv1d? HOT 11
- Is it okay to put S6(MAMBA) and gated MLP blocks like a transformer? (Also please open the discussion tab)
- Vanishing gradient problem with more layer HOT 1
- Results vary greatly across experiments
- Gradient explosion in Mamba2 training, norm and loss divergence HOT 3
- Optimizing the bwd pass of Mamba 2 HOT 3
- Question about d_state. HOT 1
- Understanding about the selective scan HOT 2
- ModuleNotFoundError: No module named 'mamba_ssm.ops.triton.ssd_combined
- ERROR: Failed building wheel for mamba-ssm
- Issue about the FLOPs of selective scan
- Chunked inference
- Datasets error
- OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like state-spaces is not the path to a directory containing a file named config.json. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
- Mamba.__init__() got an unexpected keyword argument 'layer_idx'
- How to get all hidden_states of selective_scan_cuda?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mamba.