waityousea / xuniren Goto Github PK

老实说我问过几个同事了，都看不懂在讲什么， 又是 Ubuntu 又是 mac 又是 win 来回说明 根本不知道哪个需要哪个不需要， 建议把win/mac/Ubuntu 对应的步骤分开说。 该跳过的步骤， 清晰地说出两个字：跳过。

pip install ./freqencoder
pip install ./shencoder
pip install ./gridencoder

这几个缺失， 是哪个步骤出了问题？

安装时出现如下错误

bash scripts/install_ext.sh
Processing ./freqencoder
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/Volumes/KINGSTON/projects/xuniren/freqencoder/setup.py", line 36, in <module>
          CUDAExtension(
        File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1048, in CUDAExtension
          library_dirs += library_paths(cuda=True)
        File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1179, in library_paths
          if (not os.path.exists(_join_cuda_home(lib_dir)) and
        File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2223, in _join_cuda_home
          raise EnvironmentError('CUDA_HOME environment variable is not set. '
      OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.0.1 -> 23.1.2
[notice] To update, run: pip install --upgrade pip
Processing ./shencoder
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/Volumes/KINGSTON/projects/xuniren/shencoder/setup.py", line 35, in <module>
          CUDAExtension(
        File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1048, in CUDAExtension
          library_dirs += library_paths(cuda=True)
        File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1179, in library_paths
          if (not os.path.exists(_join_cuda_home(lib_dir)) and
        File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2223, in _join_cuda_home
          raise EnvironmentError('CUDA_HOME environment variable is not set. '
      OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.0.1 -> 23.1.2
[notice] To update, run: pip install --upgrade pip
Processing ./gridencoder
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/Volumes/KINGSTON/projects/xuniren/gridencoder/setup.py", line 35, in <module>
          CUDAExtension(
        File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1048, in CUDAExtension
          library_dirs += library_paths(cuda=True)
        File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1179, in library_paths
          if (not os.path.exists(_join_cuda_home(lib_dir)) and
        File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2223, in _join_cuda_home
          raise EnvironmentError('CUDA_HOME environment variable is not set. '
      OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

跳过这步之后，启动服务 python app.py, 也出现如下错误

python app.py
Traceback (most recent call last):
  File "/Volumes/KINGSTON/projects/xuniren/raymarching/raymarching.py", line 10, in <module>
    import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Volumes/KINGSTON/projects/xuniren/app.py", line 10, in <module>
    from tools import audio_pre_process, video_pre_process, generate_video,audio_process
  File "/Volumes/KINGSTON/projects/xuniren/tools.py", line 20, in <module>
    from nerf.network import NeRFNetwork
  File "/Volumes/KINGSTON/projects/xuniren/nerf/network.py", line 7, in <module>
    from .renderer import NeRFRenderer
  File "/Volumes/KINGSTON/projects/xuniren/nerf/renderer.py", line 10, in <module>
    import raymarching
  File "/Volumes/KINGSTON/projects/xuniren/raymarching/__init__.py", line 1, in <module>
    from .raymarching import *
  File "/Volumes/KINGSTON/projects/xuniren/raymarching/raymarching.py", line 12, in <module>
    from .backend import _backend
  File "/Volumes/KINGSTON/projects/xuniren/raymarching/backend.py", line 31, in <module>
    _backend = load(name='_raymarching_face',
  File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1284, in load
    return _jit_compile(
  File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1509, in _jit_compile
    _write_ninja_file_and_build_library(
  File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1601, in _write_ninja_file_and_build_library
    extra_ldflags = _prepare_ldflags(
  File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1699, in _prepare_ldflags
    extra_ldflags.append(f'-L{_join_cuda_home("lib64")}')
  File "/Volumes/KINGSTON/projects/xuniren/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 2223, in _join_cuda_home
    raise EnvironmentError('CUDA_HOME environment variable is not set. '
OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root.

pytorch 3d windows cuda11.8安装失败

采用指令
conda create --name xuniren python=3.10
conda activate xuniren
git clone https://github.com/waityousea/xuniren.git
cd xuniren
pip install -r requirements.txt
conda install -c bottler nvidiacub
pip install "git+https://github.com/facebookresearch/pytorch3d.git"
但一直报下面错误，但实际上执行nvcc -V完全没问题的，后面降低了pytorch、torchvision的版本，改了pytorch3d的tag进行pip安装，还是报下面的错误，env里面也检查过CUDA_HOME=:/usr/local/cuda，没找到问题解决办法
running build_ext
error: [Errno 2] No such file or directory: ':/usr/local/cuda/bin/nvcc'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pytorch3d
Running setup.py clean for pytorch3d
Failed to build pytorch3d
ERROR: Could not build wheels for pytorch3d, which is required to install pyproject.toml-based projects

我看geneface++训练也是用的rednerf，但是生成的是ckpt，
我转成pth后用在用，加载模型的时候会提示数组位对不上‵‵‵

这个需要连接fay才有的吗？还是代码有bug

按照要求安装，执行python fay_connect.py,可以运行成功.但是在fay数字人助理版用麦克风进行说话。
控制台显示如下错误，什么原因导致？
......
Fay Error: The expanded size of the tensor (50) must match the existing size (56) at non-singleton dimension 0. Target sizes: [50, 44]. Tensor sizes: [56, 44]
Fay Connection opened

test.html:64 WebSocket is already in CLOSING or CLOSED state.

按照环境要求配置好环境，尝试运行时语音正常合成，但是合成后fay_connect.py提示Fay Error: [WinError 2] 系统找不到指定的文件，检查绝对路径发现并没有问题，恳请各位大佬帮忙看看

在尝试加载优化器时失败。
运行停止在加载优化器。。。

VS++编译freqencoder时，python setup.py install提示，error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin\nvcc.exe' failed with exit code 1,请教一下，这个应该怎么解决呢，cuda安装成功环境变量也配置，nvcc -V也提示成功，为啥会有这个错误呢

建立连接！
[WARN] audio sample rate is 24000, resampling into 16000.
[INFO] loaded audio stream data/audio/aud_0.wav: (55680,)
[START]
haj ajam dablo es klajent
[END]
[INFO] save all feats for training purpose...
[INFO] saved logits to data/audio/aud_0_eo.npy
[INFO] load 3508 frames.
[INFO] load data/audio/aud_0_eo.npy aud_features: torch.Size([87, 44, 16])
Loading data: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3508/3508 [00:00<00:00, 80353.99it/s]
[INFO] eye_area: 0.25 - 0.25
==> Start Test, save results to data/video/results
0% 0/87 [00:00<?, ?it/s]Traceback (most recent call last):
File "/home/cv/.conda/envs/torch3d_ay/lib/python3.9/site-packages/gevent/pywsgi.py", line 1107, in handle_one_response
self.run_application()
File "/home/cv/.conda/envs/torch3d_ay/lib/python3.9/site-packages/geventwebsocket/handler.py", line 75, in run_application
self.run_websocket()
File "/home/cv/.conda/envs/torch3d_ay/lib/python3.9/site-packages/geventwebsocket/handler.py", line 52, in run_websocket
list(self.application(self.environ, lambda s, h, e=None: []))
File "/home/cv/.conda/envs/torch3d_ay/lib/python3.9/site-packages/flask/app.py", line 2464, in call
return self.wsgi_app(environ, start_response)
File "/home/cv/.conda/envs/torch3d_ay/lib/python3.9/site-packages/flask_sockets.py", line 45, in call
handler(environment, **values)
File "/data/cv/aoyang/xuniren-main/app.py", line 87, in echo_socket
generate_video(audio_path, audio_path_eo, video_path, output_path)
File "/data/cv/aoyang/xuniren-main/tools.py", line 606, in generate_video
path = video_process(opt_vid, trainer_vid, model_vid, dir_path)
File "/data/cv/aoyang/xuniren-main/tools.py", line 586, in video_process
test = trainer.test(test_loader, name=dir_path['input'].split("/")[-1].split(".")[0])
File "/data/cv/aoyang/xuniren-main/nerf/utils.py", line 967, in test
video_stream.write([pred])
File "/data/cv/aoyang/xuniren-main/video_stream.py", line 8, in write
video_stream_cache.write(frame)
AttributeError: 'NoneType' object has no attribute 'write'
2024-01-25T06:56:37Z {'REMOTE_ADDR': '127.0.0.1', 'REMOTE_PORT': '53192', 'HTTP_HOST': '0.0.0.0:38800', (hidden keys: 25)} failed with AttributeError

请问是否仅针对RAD-NeRF做了实时化，效果相比RAD-NeRF有无提升？
这样的Nerf效果肯定是不能接受的

ubuntu下：

pip install -r requirements
pip install "git+https://github.com/facebookresearch/pytorch3d.git"
pip install flask==1.1.4 markupsafe==2.0.1 werkzeug==1.0.1
python app.py

websocket: ws://127.0.0.1:8800/dighuman

PS D:\project\individual\digital-human\xuniren> python app.py
Traceback (most recent call last):
File "D:\project\individual\digital-human\xuniren\raymarching\raymarching.py", line 10, in
import _raymarching_face as _backend
ModuleNotFoundError: No module named '_raymarching_face'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\project\individual\digital-human\xuniren\app.py", line 10, in
from tools import audio_pre_process, video_pre_process, generate_video,audio_process
File "D:\project\individual\digital-human\xuniren\tools.py", line 20, in
from nerf.network import NeRFNetwork
File "D:\project\individual\digital-human\xuniren\nerf\network.py", line 7, in
from .renderer import NeRFRenderer
File "D:\project\individual\digital-human\xuniren\nerf\renderer.py", line 10, in
import raymarching
File "D:\project\individual\digital-human\xuniren\raymarching_init_.py", line 1, in
from .raymarching import *
File "D:\project\individual\digital-human\xuniren\raymarching\raymarching.py", line 12, in
from .backend import _backend
File "D:\project\individual\digital-human\xuniren\raymarching\backend.py", line 31, in
_backend = load(name='_raymarching_face',
File "D:\Users\Hyacinth\anaconda3\envs\xuniren\lib\site-packages\torch\utils\cpp_extension.py", line 1284, in load
return _jit_compile(
File "D:\Users\Hyacinth\anaconda3\envs\xuniren\lib\site-packages\torch\utils\cpp_extension.py", line 1534, in _jit_compile
return _import_module_from_library(name, build_directory, is_python_module)
File "D:\Users\Hyacinth\anaconda3\envs\xuniren\lib\site-packages\torch\utils\cpp_extension.py", line 1936, in _import_module_from_library
module = importlib.util.module_from_spec(spec)
ImportError: DLL load failed while importing _raymarching_face: 找不到指定的模块。

提示 No such file or directory: './data/audio/aud_1698005355.wav'

Traceback (most recent call last):
File "C:\Users\XXXXXXX.conda\envs\torch3d\lib\site-packages\gevent\pywsgi.py", line 988, in handle_one_response
self.run_application()
File "C:\Users\XXXXXXX.conda\envs\torch3d\lib\site-packages\geventwebsocket\handler.py", line 75, in run_application
self.run_websocket()
File "C:\Users\XXXXXXX.conda\envs\torch3d\lib\site-packages\geventwebsocket\handler.py", line 52, in run_websocket
list(self.application(self.environ, lambda s, h, e=None: []))
File "C:\Users\XXXXXXX.conda\envs\torch3d\lib\site-packages\flask\app.py", line 2213, in call
return self.wsgi_app(environ, start_response)
File "C:\Users\XXXXXXX.conda\envs\torch3d\lib\site-packages\flask_sockets.py", line 40, in call
handler, values = adapter.match()
File "C:\Users\XXXXXXX.conda\envs\torch3d\lib\site-packages\werkzeug\routing\map.py", line 653, in match
raise WebsocketMismatch() from None
werkzeug.routing.exceptions.WebsocketMismatch: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.
2023-08-17T10:43:19Z {'REMOTE_ADDR': '127.0.0.1', 'REMOTE_PORT': '52284', 'HTTP_HOST': '127.0.0.1:8800', (hidden keys: 30)} failed with WebsocketMismatch

数字人可以更换形象吗

大哥 传东西能出完拿证吗

怎么在服务器上运行fay_connect.py,服务器上没有图形显示，130行左右第一段代码： cv2.imshow('Fay-2d', frame)，怎么进行打开图片
deepin系统，服务器是linux的
fay已经在本地配置完成

C:\Users\Administrator\Desktop\xuniren> pip install .\freqencoder
Processing c:\users\administrator\desktop\xuniren\freqencoder
Preparing metadata (setup.py) ... done
Building wheels for collected packages: freqencoder
Building wheel for freqencoder (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [37 lines of output]
running bdist_wheel
running build
running build_ext
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "C:\Users\Administrator\Desktop\xuniren\freqencoder\setup.py", line 33, in
setup(
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools_init_.py", line 153, in setup
return distutils.core.setup(**attrs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\distutils\dist.py", line 966, in run_commands
self.run_command(cmd)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\wheel\bdist_wheel.py", line 343, in run
self.run_command("build")
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\distutils\command\build.py", line 135, in run
self.run_command(cmd_name)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\build_ext.py", line 79, in run
_build_ext.run(self)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\distutils\command\build_ext.py", line 340, in run
self.build_extensions()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils\cpp_extension.py", line 485, in build_extensions
compiler_name, compiler_version = self._check_abi()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils\cpp_extension.py", line 875, in _check_abi
raise UserWarning(msg)
UserWarning: It seems that the VC environment is activated but DISTUTILS_USE_SDK is not set.This may lead to multiple activations of the VC env.Please set DISTUTILS_USE_SDK=1 and try again.
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for freqencoder
Running setup.py clean for freqencoder
Failed to build freqencoder
ERROR: Could not build wheels for freqencoder, which is required to install pyproject.toml-based projects

ResolvePackageNotFound:

• libdrm==2.4.114=h166bdaf_0
• xorg-xproto==7.0.31=h7f98852_1007
• openjpeg==2.5.0=hfec8fc6_2
• svt-av1==1.4.1=hcb278e6_0
• libsqlite==3.40.0=h753d276_0
• libdeflate==1.17=h0b41bf4_0
• aom==3.5.0=h27087fc_0
• libiconv==1.17=h166bdaf_0
• portalocker==2.7.0=py38h578d9bd_0
• xz==5.2.6=h166bdaf_0
• lame==3.100=h166bdaf_1003
• libstdcxx-ng==12.2.0=h46fd767_19
• ca-certificates==2022.12.7=ha878542_0
• p11-kit==0.24.1=hc5aa10d_0
• numpy==1.23.5=py38h14f4228_0
• fontconfig==2.14.2=h14ed4e7_0
• lcms2==2.15=hfd0df8a_0
• bzip2==1.0.8=h7f98852_4
• pytorch3d==0.7.2=py38_cu113_pyt1121
• mkl==2021.4.0=h8d4b97c_729
• xorg-kbproto==1.0.7=h7f98852_1002
• x265==3.5=h924138e_3
• pyyaml==6.0=py38h0a891b7_5
• xorg-libx11==1.8.4=h0b41bf4_0
• _libgcc_mutex==0.1=conda_forge
• libuuid==2.32.1=h7f98852_1000
• mkl_random==1.2.2=py38h1abd341_0
• gnutls==3.7.8=hf3e180e_0
• readline==8.1.2=h0f457ee_0
• xorg-libxdmcp==1.1.3=h7f98852_0
• libpng==1.6.39=h753d276_0
• yaml==0.2.5=h7f98852_2
• libffi==3.4.2=h7f98852_5
• lerc==4.0.0=h27087fc_0
• libpciaccess==0.17=h166bdaf_0
• libvpx==1.11.0=h9c3ff4c_3
• nvidiacub==1.10.0=0
• libunistring==0.9.10=h7f98852_0
• libxml2==2.10.3=hfdac1af_6
• cudatoolkit==11.3.1=h9edb442_11
• libzlib==1.2.13=h166bdaf_4
• gmp==6.2.1=h58526e2_0
• mkl-service==2.4.0=py38h95df7f1_0
• pthread-stubs==0.4=h36c2ea0_1001
• xorg-xextproto==7.3.0=h0b41bf4_1003
• _openmp_mutex==4.5=2_kmp_llvm
• numpy-base==1.23.5=py38h31eccc5_0
• zstd==1.5.2=h3eb15da_6
• libnsl==2.0.0=h7f98852_0
• llvm-openmp==16.0.0=h417c0b6_0
• expat==2.5.0=h27087fc_0
• xorg-libxext==1.3.4=h0b41bf4_2
• libva==2.18.0=h0b41bf4_0
• mkl_fft==1.3.1=py38h8666266_1
• tbb==2021.8.0=hf52228f_0
• libidn2==2.3.4=h166bdaf_0
• tk==8.6.12=h27826a3_0
• nettle==3.8.1=hc379101_1
• cryptography==39.0.2=py38h3d167d9_0
• xorg-fixesproto==5.0=h7f98852_1002
• libtasn1==4.19.0=h166bdaf_0
• libwebp-base==1.3.0=h0b41bf4_0
• pillow==9.4.0=py38hde6dc18_1
• libxcb==1.13=h7f98852_1004
• libhwloc==2.9.0=hd6dc26d_0
• libgcc-ng==12.2.0=h65d4601_19
• python==3.8.16=he550d4f_1_cpython
• freetype==2.12.1=hca18f0e_1
• ffmpeg==5.1.2=gpl_h8dda1f0_106
• libtiff==4.5.0=h6adf6a1_2
• openh264==2.3.1=hcb278e6_2
• ncurses==6.3=h27087fc_1
• xorg-libxfixes==5.0.3=h7f98852_1004
• libopus==1.3.1=h7f98852_1
• xorg-libxau==1.0.9=h7f98852_0
• x264==1!164.3095=h166bdaf_2
• openssl==3.1.0=h0b41bf4_0
• pytorch==1.12.1=py3.8_cuda11.3_cudnn8.3.2_0
• gettext==0.21.1=h27087fc_0
• icu==72.1=hcb278e6_0
• ld_impl_linux-64==2.40=h41732ed_0
• brotlipy==0.7.0=py38h0a891b7_1005
• cffi==1.15.1=py38h4a40e3a_3
• jpeg==9e=h0b41bf4_3

(pytorch3d) PS E:\xuniren>

(xuniren) E:\mywork\xuniren>python app.py
[INFO] loading ASR model ./models/wav2vec2-large-xlsr-53-esperanto...
E:\software\anaconda3\envs\xuniren\lib\site-packages\transformers\configuration_utils.py:380: UserWarning: Passing gradient_checkpointing to a config initialization is deprecated and will be removed in v5 Transformers. Using model.gradient_checkpointing_enable() instead, or if you are using the Trainer API, pass gradient_checkpointing=True in your TrainingArguments.
warnings.warn(
Ignored unknown kwarg option normalize
Ignored unknown kwarg option normalize
Ignored unknown kwarg option normalize
Ignored unknown kwarg option normalize
[INFO] Trainer: ngp | 2023-11-02_14-12-43 | cuda | fp16 | data/video
[INFO] #parameters: 4231701
[INFO] Loading data/pretrained/ngp_kf.pth ...
[INFO] loaded model.
[INFO] load at epoch 58, global step 203464
[WARN] Failed to load optimizer.
[INFO] loaded scheduler.
[INFO] loaded scaler.

执行到这里一直没反应了

我在Windows 11 下面， pytorch 1.12.1， cuda 11.3， pytorch3d 0.7.1 环境下，所有的组件也已经成功安装。执行命令的时候， 出现下面的结果。

能否告知，哪里有问题了？

我在云端服务器上启动了脚本，python app.py，但是没有任何的显示页面等反应，应该怎么处理呢？

raise WebsocketMismatch() from None

werkzeug.routing.exceptions.WebsocketMismatch: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.
2023-06-02T13:34:31Z {'REMOTE_ADDR': '127.0.0.1', 'REMOTE_PORT': '55949', 'HTTP_HOST': '127.0.0.1:8800', (hidden keys: 24)} failed with WebsocketMismatch

这是什么问题呢请教一下

x64 Native Tools Command Prompt for vs2022 下面编译安装freqencoder 模块报错
CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6'

Processing d:\fayproject\xuniren\freqencoder
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: freqencoder
  Building wheel for freqencoder (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [50 lines of output]
      No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6'
      running bdist_wheel
      running build
      running build_ext
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "D:\FayProject\xuniren\freqencoder\setup.py", line 33, in <module>
          setup(
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
          return run_commands(dist)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
          dist.run_commands()
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\dist.py", line 1208, in run_command
          super().run_command(command)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\wheel\bdist_wheel.py", line 325, in run
          self.run_command("build")
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\dist.py", line 1208, in run_command
          super().run_command(command)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\command\build.py", line 132, in run
          self.run_command(cmd_name)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\dist.py", line 1208, in run_command
          super().run_command(command)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\command\build_ext.py", line 84, in run
          _build_ext.run(self)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\setuptools\_distutils\command\build_ext.py", line 346, in run
          self.build_extensions()
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\torch\utils\cpp_extension.py", line 499, in build_extensions
          _check_cuda_version(compiler_name, compiler_version)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\torch\utils\cpp_extension.py", line 382, in _check_cuda_version
          torch_cuda_version = packaging.version.parse(torch.version.cuda)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\pkg_resources\_vendor\packaging\version.py", line 49, in parse
          return Version(version)
        File "C:\ProgramData\Anaconda3\envs\pytorch3d\lib\site-packages\pkg_resources\_vendor\packaging\version.py", line 264, in __init__
          match = self._regex.search(version)
      TypeError: expected string or bytes-like object
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for freqencoder
  Running setup.py clean for freqencoder

good job!

运行 fay_connect.py 在生成视频步骤的时候报错：Fay Error: weakly-referenced object no longer exists

message: D:/project/github/Fay/samples/sample-1692692582857.mp3 <class 'str'>
[WARN] audio sample rate is 24000, resampling into 16000.
[INFO] loaded audio stream data/audio/aud_1692692584.wav: (178176,)
[START]
mi ha bo ŝi ĉafiĝi al en pa liu pi li reci i jemu sin ka faregon kun ŝi non ĉuŝa odeg aŭnu ŝi lejun hutikon ŝitan de dafu re ĉiĝi
[END]
[INFO] save all feats for training purpose...
[INFO] saved logits to data/audio/aud_1692692584_eo.npy
[INFO] load 3508 frames.
[INFO] load data/audio/aud_1692692584_eo.npy aud_features: torch.Size([279, 44, 16])
Loading data: 100%|██████████████████████████████████████████████████████████████| 3508/3508 [00:00<00:00, 92562.93it/s]
[INFO] eye_area: 0.25 - 0.25
==> Start Test, save results to data/video\results
***** 1
0% 0/279 [00:00<?, ?it/s]***** 2
***** 3
<torch.utils.data.dataloader.DataLoader object at 0x00000257575F1630>
Fay Error: weakly-referenced object no longer exists�

Traceback (most recent call last):
File "/home/house365ai/.conda/envs/xuniren/lib/python3.9/site-packages/gevent/pywsgi.py", line 999, in handle_one_response
self.run_application()
File "/home/house365ai/.conda/envs/xuniren/lib/python3.9/site-packages/geventwebsocket/handler.py", line 75, in run_application
self.run_websocket()
File "/home/house365ai/.conda/envs/xuniren/lib/python3.9/site-packages/geventwebsocket/handler.py", line 52, in run_websocket
list(self.application(self.environ, lambda s, h, e=None: []))
File "/home/house365ai/.conda/envs/xuniren/lib/python3.9/site-packages/flask/app.py", line 2464, in call
return self.wsgi_app(environ, start_response)
File "/home/house365ai/.conda/envs/xuniren/lib/python3.9/site-packages/flask_sockets.py", line 45, in call
handler(environment, **values)
File "/home/house365ai/xxm/vrh/app.py", line 83, in echo_socket
generate_video(audio_path, audio_path_eo, video_path, output_path)
File "/home/house365ai/xxm/vrh/tools.py", line 603, in generate_video
save_path = video_process(opt_vid, trainer_vid, model_vid, dir,name,audio_path)
File "/home/house365ai/xxm/vrh/tools.py", line 582, in video_process
test_loader = NeRFDataset_Test(opt, device='cuda').dataloader()
File "/home/house365ai/xxm/vrh/nerf/provider.py", line 126, in init
aud_features = np.load(self.opt.aud)
File "/home/house365ai/.conda/envs/xuniren/lib/python3.9/site-packages/numpy/lib/npyio.py", line 438, in load
raise ValueError("Cannot load file containing pickled data "
ValueError: Cannot load file containing pickled data when allow_pickle=False

两个版本都试过好几次了，从安装各种依赖开始，没成功过

用的云服务器
/root/miniconda3/lib/python3.8/site-packages/pydub/utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work
warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning)
ALSA lib confmisc.c:767:(parse_card) cannot find card '0'
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5220:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM default
ALSA lib confmisc.c:767:(parse_card) cannot find card '0'
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1246:(snd_func_refer) error evaluating name
ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5220:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM default
[INFO] loading ASR model cpierse/wav2vec2-large-xlsr-53-esperanto...
Downloading (…)lve/main/config.json: 100%|████████████████████████████████████████████| 1.56k/1.56k [00:00<00:00, 197kB/s]
Downloading (…)olve/main/vocab.json: 100%|███████████████████████████████████████████████| 420/420 [00:00<00:00, 59.7kB/s]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Traceback (most recent call last):
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1010, in validate_conn
conn.connect()
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 411, in connect
self.sock = ssl_wrap_socket(
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/util/ssl.py", line 449, in ssl_wrap_socket
ssl_sock = ssl_wrap_socket_impl(
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/util/ssl.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/root/miniconda3/lib/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/root/miniconda3/lib/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
File "/root/miniconda3/lib/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/root/miniconda3/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/util/retry.py", line 532, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/packages/six.py", line 769, in reraise
raise value.with_traceback(tb)
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 1010, in validate_conn
conn.connect()
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 411, in connect
self.sock = ssl_wrap_socket(
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/util/ssl.py", line 449, in ssl_wrap_socket
ssl_sock = ssl_wrap_socket_impl(
File "/root/miniconda3/lib/python3.8/site-packages/urllib3/util/ssl.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/root/miniconda3/lib/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/root/miniconda3/lib/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
File "/root/miniconda3/lib/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "fay_connect.py", line 119, in
audio_pre_process()
File "/root/autodl-tmp/xuniren/tools.py", line 428, in audio_pre_process
model_au = AutoModelForCTC.from_pretrained(asr_model).to('cuda')
File "/root/miniconda3/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 467, in from_pretrained
return model_class.from_pretrained(
File "/root/miniconda3/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2480, in from_pretrained
if has_file(pretrained_model_name_or_path, TF2_WEIGHTS_NAME, **has_file_kwargs):
File "/root/miniconda3/lib/python3.8/site-packages/transformers/utils/hub.py", line 610, in has_file
r = requests.head(url, headers=headers, allow_redirects=False, proxies=proxies, timeout=10)
File "/root/miniconda3/lib/python3.8/site-packages/requests/api.py", line 104, in head
return request('head', url, **kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/root/miniconda3/lib/python3.8/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))