AttributeError: 'DeepSpeedCPUAdam' object has no attribute 'ds_opt_adam' about finetune-gpt2xl

[CharanSG]

In my case installing "cudatoolkit-dev" solved the issue

[yochrisbolton]

For me I noticed it was exciting on a ['which', 'c++'] eval before AttributeError: 'DeepSpeedCPUAdam' object has no attribute 'ds_opt_adam'

In my case, installing / updating g++ successfully resolved the issue for me

[remotejob]

without :

---

"optimizer": {
"type": "AdamW",
"params": {
"lr": "auto",
"betas": "auto",
"eps": "auto",
"weight_decay": "auto"
}

---

in ds_config.json all work it takes 17 min

[Elfsong]

Same problem

[ziweiji]

I also occur that. before AttributeError: 'DeepSpeedCPUAdam' object has no attribute 'ds_opt_adam'
The error also show: cannot make a dir in /tmp/torch_extensions/build for cpu_adam.
So I change the DEFAULT_TORCH_EXTENSION_PATH in the file /anaconda3/envs/XXXXX/lib/python3.6/site-packages/deepspeed/ops/op_builder/builder.py
from "/tmp/torch_extensions/" to any path where I have permission to create folders.
then it works.

[aniruddhakal]

torch offers different versions for cpu and cuda devices.
I removed cpu version and install cuda version as per guidelines here:
https://pytorch.org/get-started/locally/

This is what I installed for pip:
pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113

Additionally there was this intermediate error:
"cannot find libcurand.so.o something"
which was solved by installing:
apt-get install -y libopenblas-base

But make sure, you on Ubuntu 20.04 or higher before installing libopenblas-base.

And that's how my problem was solved!

[uahmad235]

I think it is the problem with that specific deepspeed version (i.e., 0.4.0) in requirements. In my case, it was solved by upgrading deepspeed. You can upgrade it by using this command pip install -U deepspeed and it should be fixed.

[KelleyYin]

For me I noticed it was exciting on a ['which', 'c++'] eval before AttributeError: 'DeepSpeedCPUAdam' object has no attribute 'ds_opt_adam'

In my case, installing / updating g++ successfully resolved the issue for me

Thanks.
This is effective for me.