Comments (3)
hawkinsp
commented on June 14, 2024
I can't reproduce this. Can you please
a) try with a fresh virtualenv, and
b) share the output of pip freeze and nvidia-smi?
from jax.
adam-hartshorne
commented on June 14, 2024
I think I have found the problem (and have definitely fixed the issue). The only obvious difference between a fresh installation of JAX followed by PyTorch in a new environment vs the problematic environment was the following package.
nvidia-nccl-cu12-2.20.5
After uninstalling this and reinstalling jax / pytorch, I am now left with
nvidia-nccl-cu12-2.19.3
and everything works fine again.
from jax.
hawkinsp
commented on June 14, 2024
Hmm. Even if I update my NCCL package to that version, things still work for me.
Maybe it was some kind of broken package environment. If you see it again, let me know!
from jax.
Related Issues (20)
- ndarray.at.set(mode="drop") gives incorrect value for the last element in the array HOT 4
- partial eval silently skips effects HOT 3
- NaN when computing gradient of squared norm evaluated at 0. HOT 2
- TracerBoolConversionError when jitting jax.numpy.linalg.norm HOT 4
- Allow for `is_leaf` in tree_all
- jax-metal: non-deterministic behavior of `jnp.take_along_axis` HOT 2
- jax.scipy.special.expi extremely slow when applied to a jnp.array HOT 3
- TPU Initialization Failed
- jax-metal on m1 couldn't compile random.PRNGKey properly HOT 1
- jax.numpy.linalg.inv returns spurious results when called with array of matrices HOT 2
- jax.debug.print printing empyt output HOT 1
- PJRT CUDA Plugin and Custom Call extensions HOT 15
- [ROCM] Error: wheel file is invalid HOT 8
- pmax bug on negative numbers on CPU HOT 3
- jax-metal: minimum/maximum/min/max/argmin/argmax invalid output with NaNs
- Tracers are now hashable. HOT 12
- `jit-eval_shape-<callable PyTree with numpy __eq__ semantics>` crashes on JAX 0.4.29
- Passed `stacklevel=2` for the new `tree_flatten` deprecation warning. HOT 6
- jnp fails after enable x64 with Apple M1 chip HOT 2
- Unexpected AllReduce in backward pass with shard_map, custom_vjp, and pallas
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jax.