so-vits-svc's Introduction

SoftVC VITS Singing Voice Conversion

Update

针对sovits3.0 48khz模型推理显存占用大的问题，可以切换到32khz的分支版本训练32khz的模型
目前发现一个较大问题，3.0推理时显存占用巨大，6G显存基本只能推理30s左右长度音频
断音问题已解决，音质提升了不少
2.0版本已经移至 sovits_2.0分支
3.0版本使用FreeVC的代码结构，与旧版本不通用
目前音质上依然与DiffSVC 有较大的差距

模型简介

歌声音色转换模型，通过SoftVC内容编码器提取源音频语音特征，与F0同时输入VITS替换原本的文本输入达到歌声转换的效果。同时，更换声码器为 NSF HiFiGAN 解决断音问题

注意

当前分支是48khz的版本，推理时显存占用较大，经常会出现爆显存的问题，如果爆显存需要手动将音频切片逐片段转换，推荐切换到32khz的分支训练32khz版本的模型

预先下载的模型文件

soft vc hubert：hubert-soft-0d54a1f4.pt
- 放在hubert目录下
预训练底模文件 G_0.pth 与 D_0.pth
- 放在logs/48k 目录下
- 预训练底模为必选项，因为据测试从零开始训练有概率不收敛，同时底模也能加快训练速度
- 预训练底模训练数据集包含云灏即霜辉宇·星AI 派蒙绫地宁宁，覆盖男女生常见音域，可以认为是相对通用的底模
- 底模删除了optimizer speaker_embedding 等无关权重, 只可以用于初始化训练，无法用于推理

# 一键下载
# hubert
wget -P hubert/ https://github.com/bshall/hubert/releases/download/v0.1/hubert-soft-0d54a1f4.pt
# G与D预训练模型
wget -P logs/48k/ https://huggingface.co/innnky/sovits_pretrained/resolve/main/G_0.pth
wget -P logs/48k/ https://huggingface.co/innnky/sovits_pretrained/resolve/main/D_0.pth

数据集准备

仅需要以以下文件结构将数据集放入dataset_raw目录即可

dataset_raw
├───speaker0
│   ├───xxx1-xxx1.wav
│   ├───...
│   └───Lxx-0xx8.wav
└───speaker1
    ├───xx2-0xxx2.wav
    ├───...
    └───xxx7-xxx007.wav

数据预处理

重采样至 48khz

python resample.py

自动划分训练集验证集测试集以及自动生成配置文件

python preprocess_flist_config.py
# 注意
# 自动生成的配置文件中，说话人数量n_speakers会自动按照数据集中的人数而定
# 为了给之后添加说话人留下一定空间，n_speakers自动设置为 当前数据集人数乘2
# 如果想多留一些空位可以在此步骤后 自行修改生成的config.json中n_speakers数量
# 一旦模型开始训练后此项不可再更改

生成hubert与f0

python preprocess_hubert_f0.py

执行完以上步骤后 dataset 目录便是预处理完成的数据，可以删除dataset_raw文件夹了

训练

python train.py -c configs/config.json -m 48k

训练中追加说话人数据

基本类似预处理过程

将新追加的说话人数据按之前的结构放入dataset_raw目录下，并重采样至48khz

python resample.py

使用add_speaker.py重新生成训练集、验证集，重新生成配置文件

python add_speaker.py

重新生成hubert与f0

python preprocess_hubert_f0.py

之后便可以删除dataset_raw文件夹了

推理

使用inference_main.py

更改model_path为你自己训练的最新模型记录点
将待转换的音频放在raw文件夹下
clean_names 写待转换的音频名称
trans 填写变调半音数量
spk_list 填写合成的说话人名称

so-vits-svc's People

Contributors

Stargazers

Watchers

so-vits-svc's Issues

我用的是Nvidia GTX 660M的显卡，最高只能安装到CUDA 10.1和Torch 1.8.1的版本。请问我还能运行你的这个程序吗？

I am a music instructor and I would love to introduce this lovely AI software to our students to try out.

Here in my school we have several Windows 7 Pro 64-bit computers in our classrooms, running on Nvidia GeForce GTX 660M GPU. According to Nvidia, the highest version of graphic driver we can install is 425.31, and the highest CUDA Toolkit we can install would be 10.1.

According to pytorch dot org, with CUDA version 10.1, the highest torch we can install would be:
“torch-1.8.1+cu101-cp39-cp39-win_amd64.whl”.

Here, “cu101” in the file name, is referring to CUDA 10.1.

Any torch version higher than 1.8.1, will have a higher “cu” number attached in the whl file name, such as:
“torch-1.10.0+cu102-cp36-cp36m-win_amd64.whl”, or
“torch-1.13.0+cu116-cp310-cp310-win_amd64.whl”, etc.

In other words, our school can not install any torch higher than version 1.8.1.

In the non-fork so-vits-svc-4.0 program folder, there is a file called “requirements.txt”. We opened that file, and can see it says “torch==1.13.1”. Can we assume torch version 1.13.1 is the lowest minimum requirement for so-vits-svc program to run?

Does it mean we can not install your amazing software on our school’s computers, because our Nvidia GPU are too old, and can’t reach your required pytorch 1.13.1 version? Or maybe it doesn’t matter, a lower pytorch 1.8.1 version can still run?

Too bad! My colleagues have already trained several G_43200.pth models on their home computers, and they can just simply copy these models to our school’s computers and start the voice inference right away. We don’t need to train on the classroom’s computers, we just need to infer on existing models, to demonstrate to our students. Inferring takes an awful lot less of GPU powers to do.

Has anyone tested this program on CUDA 10.1?

Please let me know. So, should I give up? Is it a death penalty for our students to see this?

Recommend Projects

dlseed / so-vits-svc Goto Github PK