Alternate Subtraction Method, Faster about aphantasia

[eps696]

1. good point, will move subtraction out of the training loop.
2. your method of "increasing the difference" in fact just decreasing the effect of subtraction (like adding weight < 1): here 2x-y ~ x-0.5y. and the examples did show that - some kind of "faces" appeared with such weighing down.
3. sure; on my understanding, any continuous embedding is a latent vector by definition. we just don't have decoder for that, like from proper dall-e (not the stripped down published version, but the photorealistic one from the article), so have to move around with optimization techniques instead.

[torridgristle]

Ha! Whoops, I was so focused on trying to do something involving the tendency for CLIP to label an image with a face as "a photo of a human face" with a higher score than "a photo of a human face" that I done went and did 2*enc1-enc2, shit. Back to the drawing board.

[eps696]

regarding preliminary text subtraction txt_enc - text_enc0: after second thinking, it's not the same. when we compare the losses after cossimilarity, we check how far or close we're to those prompts/concepts (that's what we probably want). if we subtract it at once, we will check instead how close we are to the difference between the two, essentially losing the position of "center of mass" of the pair (in the embedding space). so the resulting vector may have nothing in common with either of prompts, and most likely we'd get smth rather different.

[eps696]

just to ensure - i've tried direct subtraction method on a few meaningful sentences, and it predictably went totally aside of main topic. and just to make it clear - encoded embeddings are NOT losses, their summation/subtraction have different impact.
finally, cossim comparison is just an op, it's probably few orders of magnitude faster than encoding (and even slicing), so "time savings" should not be measurable