scarfmonster / hifipln

Multispeaker Community Vocoder Model for DiffSinger

HiFiPLN

Multispeaker Community Vocoder model for DiffSinger

This is the code used to train the "HiFiPLN" vocoder.

A trained model for use with OpenUtau is available for download on the official release page.

Why HiFiPLN?

Because a lot of PLN was spent training this thing.

Training

Python

Python 3.10 or greater is required.

Data preparation

python dataset-utils/split.py --length 1 -sr 44100 -o "dataset/train" PATH_TO_DATASET

You will also need to provide some validation audio files and save them to dataset/valid and then run:

python preproc.py --path dataset --config "configs/hifipln.yaml"

Train model

python train.py --config "configs/hifipln.yaml"

• If you see an error saying "Total length of `Data Loader` across ranks is zero" then you do not have enough validation files.
• You may want to edit configs/hifipln.yaml and change train: batch_size: 12 to a value that better fits your available VRAM.

Resume

python train.py --config "configs/hifipln.yaml" --resume CKPT_PATH

You may set CKPT_PATH to a log directory (eg. logs/HiFiPLN), and it will find the last checkpoint of the last run.

Finetuning

Download a checkpoint from https://utau.pl/hifipln/#checkpoints-for-finetuning
Save the checkpoint as ckpt/HiFiPLN.ckpt then run:

python train.py --config "configs/hifipln-finetune.yaml"

• Finetuning shouldn't be run for too long, especially for small datasets. Just 2-3 epochs or ~20000 steps should be fine.

Exporting for use in OpenUtau

python export.py --config configs/hifipln.yaml --output out/hifipln --model CKPT_PATH

You may set CKPT_PATH to a log directory (eg. logs/HiFiPLN), and it will find the last checkpoint of the last run.

Credits

• DiffSinger
• Fish Diffusion
• HiFi-GAN (Paper)
• OpenVPI
• PC-DDSP
• RefineGAN
• UnivNet (Paper)

hifipln's Issues

Dataset

Hello Author ,
This is an amazing open source project especially for vocoders.
I was wondering if I can get the 78 hours dataset ? and if it is used in any training I would definitely cite you/the repo.

Thanks in advance!

Error when exporting DDSP vocoder to ONNX

Hi. After finishing training my DDSP vocoder, I want to export it to ONNX, but I am getting the following error.

G:\HifiPLN\HiFiPLN>python export.py --config configs/ddsp.yaml --output out/ddsp --model logs\DDSP
Output path out\ddsp already exists, deleting
Exporting logs\DDSP\version_3\checkpoints\epoch=1091-step=1125000-loss=1.469.ckpt to out\ddsp
Exporting ONNX
Removed 1 parametrizations.
Model loaded
Traceback (most recent call last):
File "G:\HifiPLN\HiFiPLN\export.py", line 196, in
main(args.model, args.output, config, best=args.best, dynamo=args.dynamo)
File "G:\HifiPLN\HiFiPLN\export.py", line 159, in main
torch.onnx.export(
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\onnx\utils.py", line 516, in export
_export(
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\onnx\utils.py", line 1613, in _export
graph, params_dict, torch_out = _model_to_graph(
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\onnx\utils.py", line 1135, in _model_to_graph
graph, params, torch_out, module = _create_jit_graph(model, args)
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\onnx\utils.py", line 1011, in _create_jit_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\onnx\utils.py", line 915, in _trace_and_get_graph_from_model
trace_graph, torch_out, inputs_states = torch.jit._get_trace_graph(
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\jit_trace.py", line 1296, in _get_trace_graph
outs = ONNXTracedModule(
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\jit_trace.py", line 138, in forward
graph, out = torch._C._create_graph_by_tracing(
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\jit_trace.py", line 129, in wrapper
outs.append(self.inner(*trace_inputs))
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\jaimo\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1501, in _slow_forward
result = self.forward(*input, **kwargs)
File "G:\HifiPLN\HiFiPLN\export.py", line 95, in forward
wav = wav.squeeze(1)
AttributeError: 'tuple' object has no attribute 'squeeze'

What can I do to solve this? Thanks in advance.