Higher Batchsize VAE Decode/Encode? about ait

[hlky]

Refer to the documentation for compilation instructions.

Expected VRAM usage will be a multiple of batch size 1 usage listed on the huggingface page.

It may be better for AITemplate be integrated into batched VAE Decode/Encode and used with lower resolution higher batch size modules.

[IdiotSandwichTheThird]

Hmm, I'm following the documentation on windows, in a default comfyUI venv with diffusers + click added.
But after running the example command
python vae.py --hf-hub-or-path "runwayml/stable-diffusion-v1-5" --width 64 1024 --height 64 1024 --batch-size 1 1 --model-name "v1_vae_64_1024" --work-dir "./tmp/"
It ends up crashing with:

2023-06-26 23:26:26,775 INFO <aitemplate.backend.builder> linking 7 profiler executables
2023-06-26 23:26:26,790 WARNING <aitemplate.backend.builder> CACHE: Could not determine version of host compiler.
2023-06-26 23:26:26,832 INFO <aitemplate.backend.builder> make stdout:


2023-06-26 23:26:26,833 INFO <aitemplate.backend.builder> make stderr:

The system cannot find the path specified.

Traceback (most recent call last):
  File "F:\ComfyUI\ComfyUI\custom_nodes\AIT\AITemplate\vae.py", line 143, in <module>
    compile_diffusers()
  File "F:\ComfyUI\ComfyUI\venv\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "F:\ComfyUI\ComfyUI\venv\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "F:\ComfyUI\ComfyUI\venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "F:\ComfyUI\ComfyUI\venv\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "F:\ComfyUI\ComfyUI\custom_nodes\AIT\AITemplate\vae.py", line 117, in compile_diffusers
    compile_vae(
  File "F:\ComfyUI\ComfyUI\custom_nodes\AIT\AITemplate\ait\compile\vae.py", line 119, in compile_vae
    compile_model(
  File "F:\ComfyUI\ComfyUI\venv\lib\site-packages\aitemplate\utils\misc.py", line 93, in inner_function
    return f(*args, **kwargs)
  File "F:\ComfyUI\ComfyUI\venv\lib\site-packages\aitemplate\compiler\compiler.py", line 276, in compile_model
    compiler.transform.profile(
  File "F:\ComfyUI\ComfyUI\venv\lib\site-packages\aitemplate\compiler\transform\profile.py", line 90, in profile
    compile_engine.make_profilers(generated_profilers, profiler_dir)
  File "F:\ComfyUI\ComfyUI\venv\lib\site-packages\aitemplate\backend\builder.py", line 835, in make_profilers
    _run_make_cmds(
  File "F:\ComfyUI\ComfyUI\venv\lib\site-packages\aitemplate\backend\builder.py", line 183, in _run_make_cmds
    raise RuntimeError("Build has failed.")
RuntimeError: Build has failed.

Do you have any idea whats going wrong?

[hlky]

You need Visual Studio 2022, Community edition will do, only build tools will not be detected by CMake.

Run from x64 Native Tools Command Prompt for VS 2022 and run set AIT_USE_CMAKE_COMPILATION=1.

You also need my Cutlass 2.x/Windows branch of AITemplate but it is not published at the moment, and I should revert the logging changes before I publish it.

I'm compiling more VAE batch sizes now, they'll be available on huggingface soon.

[IdiotSandwichTheThird]

You need Visual Studio 2022, Community edition will do, only build tools will not be detected by CMake.

Run from x64 Native Tools Command Prompt for VS 2022 and run set AIT_USE_CMAKE_COMPILATION=1.

You also need my Cutlass 2.x/Windows branch of AITemplate but it is not published at the moment, and I should revert the logging changes before I publish it.

I'm compiling more VAE batch sizes now, they'll be available on huggingface soon.

Thank you for your hard work once again! I will patiently wait and be happy with my current rebatch option.