ADD FEATURE [REQUEST] about ait

[hlky]

Yes, I implemented ESRGAN for AITemplate.

Checklist for integration:

• bring compilation code to this repo
• improve inference code (alpha channel, tiled, etc)
• compile modules
• document memory planning (AITemplate pre-plans required vram usage according to profile)
• mapping for ESRGAN finetunes

I've tested RealESRGAN_x4plus and RealESRGAN_x4plus_anime_6B, if you use others list them here and I'll test/create mapping for the weights

[Shaistrong]

Yes, I implemented ESRGAN for AITemplate.

Checklist for integration:

• bring compilation code to this repo
• improve inference code (alpha channel, tiled, etc)
• compile modules
• document memory planning (AITemplate pre-plans required vram usage according to profile)
• mapping for ESRGAN finetunes

I've tested RealESRGAN_x4plus and RealESRGAN_x4plus_anime_6B, if you use others list them here and I'll test/create mapping for the weights

The AIT workflow I made is getting quite popular, so yeah.. I use an upscaler called "4x_Ultrasharp" it's an ESRGAN upscaler, a really good one. 4x_Ultrasharp: https://openmodeldb.info/models/4x-UltraSharp

[Shaistrong]

also, @hlky , many users that have 3000 series GPUs report getting barely any speed boost when using this node. could this be AIT being more efficient for 4000 series GPUs?

[hlky]

In my experience as resolution increases past around 2048x2048 the performance increase drops off, the same applies to other optimization libraries also, large generation gain less performance. Still, I generally see 1.5x-2x or more, for example earlier XL 1032x1032 took 10s with AIT and 30s with Comfy.
Let me know some more details so I can look into it further and provide a better answer; gpus, resolutions, models.

Regarding 4000 series, it would likely benefit more with modules profiled on 4000 series, I couldn't say whether it's more efficient, I'd need more data.

[hlky]

Also @Shaistrong I find that with AIT my speeds are lower than expected unless I lock my gpu clocks.
From admin prompt on Windows:
nvidia-smi -lgc 1920

Without locking, I observe that gpu clocks boost too slowly, speed increases as generation progresses, and while performance is faster than standard the generation is complete before reaching full speed

[Shaistrong]

Also @Shaistrong I find that with AIT my speeds are lower than expected unless I lock my gpu clocks. From admin prompt on Windows: nvidia-smi -lgc 1920

Without locking, I observe that gpu clocks boost too slowly, speed increases as generation progresses, and while performance is faster than standard the generation is complete before reaching full speed

Huh, does this need to be specifically 1920, or is that value dependent on something? Also by 30 second with Comfy you mean without the AIT node, right? @hlky EDIT: changed it to 2805, it got insanely fast, is this bad?

[hlky]

The maximum depends on gpu, that is the maximum for my gpu

without the AIT node

Yes

it got insanely fast, is this bad?

Fast is good, no? :)
Maybe it will increase your temperatures though.

[Shaistrong]

The maximum depends on gpu, that is the maximum for my gpu

without the AIT node

Yes

it got insanely fast, is this bad?

Fast is good, no? :) Maybe it will increase your temperatures though.

will it get even faster if I compile my own modules and replace them with the shipped ones?

[hlky]

will it get even faster if I compile my own modules and replace them with the shipped ones?

It is possible that profiling could result in different kernel selection on other 30xx series gpus. Though I wouldn't say there's a need for people to compile their own unless they're wanting to experiment whether it is the case, and it's more likely to make a difference for 40xx series.

[Shaistrong]

will it get even faster if I compile my own modules and replace them with the shipped ones?

It is possible that profiling could result in different kernel selection on other 30xx series gpus. Though I wouldn't say there's a need for people to compile their own unless they're wanting to experiment whether it is the case, and it's more likely to make a difference for 40xx series.

yeah, I figured. so, as a user of a 4000 series card- should I try to compile modules myself?