vram requirements about if

[Anatoly03]

I bought PC components two days ago (with the plan of going for SD, now that this is out...) and now that the minimum requirement grew to 16 i regret not going for intel arc, but sticking to the rtx 3060 😂

You are a life saviour. I will surely try this out when the components arrive!

[neonsecret]

see #61

[tildebyte]

@kanttouchthis; What is inference speed like when running it this way (and what are the hardware specs)?

[Trimad]

This didn't work on an RTX 4080 with 16GB of VRAM.

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 GiB. GPU 0 has a total capacty of 15.99 GiB of which 10.82 GiB is free. Of the allocated memory 2.11 GiB is allocated by PyTorch, and 729.21 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[tildebyte]

I know that this will probably sound a certain way, but: is this even English? Personally, I'm sick of Torch's horrible technical writing...

torch.cuda.set_per_process_memory_fraction

Set memory fraction for a process. The fraction is used to limit an caching allocator to allocated memory on a CUDA device. The allowed value equals the total visible memory multiplied fraction. If trying to allocate more than the allowed value in a process, will raise an out of memory error in allocator.

[Gitterman69]

id love to see a full script and not some random snippets.....