about "Pruning - Scratchpad.ipynb" about wavernn

[fatchord]

@hdmjdp I haven't trained with pruning yet. I'm still writing the code/reading papers.

[hdmjdp]

@fatchord Thanks for replying. Another question: in the paper, S=200k, total training step is 500k. After 200k step, weight is not pruning. So in the end of training, how to implement the weight sparsity?

[fatchord]

I reckon just keep applying the mask calculated at 200k steps for the rest of training - at least that's what I'm getting from the paper.

[hdmjdp]

@fatchord thanks, I think as same as you. I think in the "model" class of Pruning - Scratchpad.ipynb: you do not need to add self.prune() in "forward " function.

[bjtommychen]

@fatchord @hdmjdp FYI. I did pruning on well-trained original model weeks ago. with loss increased 1% from the smallest value, the prune ratio stop at 93% for the top5 matrix, ~ 92% of total parameters.

[hdmjdp]

@bjtommychen how about the generate wav? and which layer you pruning? if rnn, dou you only prune "weight_hh_l0"? what about "weight_ih_l0"?

[bjtommychen]

@hdmjdp I only pruned the original model. the original model's output wave is not good quality one, but almost same as the pruned one. I think the Alternative model's output is better.
('R.weight', 0.93),
('O1.weight', 0.93),
('O3.weight', 0.93),
('I_coarse.weight', 0.93),
('I_fine.weight', 0.93),