MPS Alternatives to Natten about all-in-one

[tae-jun]

Hi, the most of computing time is from source separation (Demucs), so Demucs should support mps.

However, I did a bit search and I guess it's not possible for now sadly 😢
facebookresearch/demucs#432

[filtercodes]

Hi @tae-jun, thanks for clarification. The issue seems to be that mps doesn't support complex number operations or any other float than float32... and then it's a matter of finding a right spot and using .to("cpu") function to drag back processing of that particular math operation back to cpu.

https://github.com/facebookresearch/demucs/blob/main/demucs/htdemucs.py#L628C1-L634C24

# to cpu as mps doesnt support complex numbers
        # demucs issue #435 ##432
        # NOTE: in this case z already is on cpu
        # TODO: remove this when mps supports complex numbers
        x_is_mps = x.device.type == "mps"
        if x_is_mps:
        x = x.cpu()

and then

https://github.com/facebookresearch/demucs/blob/main/demucs/htdemucs.py#L645

# back to mps device
    if x_is_mps:
    x = x.to("mps")

But then we still have cpu doing most of the work. The real solution would be to not use complex numbers at all... if the algorithm can be adopted to use only real number like for example FFT, can be done with or without complex numbers.

Are there any other source separation alternatives that we could use instead of Demucs?

[tae-jun]

There are many publicly available source separation tools nowadays, such as Spleeter.

However, I have not tested all-in-one on other source separation models, and since all-in-one is trained on outputs of Demucs, I can't guarantee its performance.

But I think it's worth a try!

[filtercodes]

I found this one with MPS support

https://github.com/karaokenerds/python-audio-separator

It seems like with the latest os Sonoma+ it is possible to get complex numbers working

pytorch/pytorch#78044 (comment)