dragen1860 / darts-pytorch Goto Github PK

When I run the current version of the code on my machine I am getting an run time error saying that one of the differentiated Tensors is unused.
RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.
It is showing the error on this line: dtheta = concat(autograd.grad(loss, self.model.parameters())).data (this is from arch.py)
Can you please help me to resolve the issue? Thank you in advance.

During training cpu usage reaches upto 1000%(can go upto 3200) in a 32 thread cpu and during validation all 32 threads reach 100% making it 3200% usage even though validation uses gpu. How do I reduce cpu usage?

nn.Parameters() will make the alpha and beta registered to model.parameters(), so your optimizer will update the alpha and beta when optimize the weight of operations. So i think the nn.parameters() should not be used in here, which will be not consistent with the paper or original code.

Thanks again for nice migration!
I've tried running your code in my environment, but it seems like I get an OOM even when I run train_search with smaller batch size. It looks like the memory consumption sees a big spike at the beginning and starts to settle into smaller usage later on...

e.g.)

python train_search.py --batchsz=16

then

...
(abbreviated)
...
RuntimeError: CUDA out of memory. Tried to allocate 1024.00 KiB (GPU 0; 15.90 GiB total capacity; 858.54 MiB already alloc│
ated; 1.88 MiB free; 14.39 GiB cached) 

I'm not sure what could be causing this issue

It takes one hour for a epoch to search architecture. However, the paper use "a small network of 8 cells is trained using DARTS for 50 epochs. The search takes one day on a single GPU". If I train 50 epochs. It will take more than two days.

I want to know if pytorch 1.0 is faster than pytorch 0.3 in this project？