Returning only the gradients/"multipliers" about captum

[AvantiShri]

Thanks @orionr! One thing to note about how I implemented support for this feature in my fork of the shap repository (which is linked to in the notebook above) is that I allow the user to optionally provide a custom function that takes as input the (multipliers, original input, references) and returns the final scores - I think the equivalent in Captum would be allowing a user to provide a custom function that varies the logic in this line:

captum/captum/attr/_core/deep_lift.py

Line 207 in 2fea0d7

(input - baseline) * gradient

The main reason to provide some flexibility here (in terms of how the user might like to combine the multipliers with the inputs/references to give the final contribution scores) is that for one-hot encoded sequences, sometimes getting the contributions is not as simple as doing multiplier*difference-from-baseline. Specifically, imagine you have a one-hot encoded sequence where only one character at each position can be a 1 (as is often done in character encodings in NLP, or sequence encodings in genomics). Say there are four characters, and the encoding at some position in the input sequence is [1,0,0,0] while the reference at that position is [0,1,0,0]. DeepLIFT will assign a contribution to the first character having a difference-from-reference of +1 and the second character having a difference-from-reference of -1. However, because of the one-hot encoding constraint that only one character at each position can have a 1, the difference-from-reference of -1 in the second character is in some sense "due to" the difference-from-reference of +1 in the first character. Thus, for ease of visualization, we add the contribution of the second character (the one that is absent from the input) to the contribution of the character that is actually present in the input, and then only visualize the total contribution on the character that is actually present in the input. If we did not do this, then at visualization time we would we get scores on more than one character at each position, which is extremely confusing given the one-hot encoding constraint!

Just something I thought I would highlight since @NarineK referenced this feature and I wasn't sure if you had decided to start working on it yet. Please let me know if you would like me to clarify more, or if anything I have said does not make sense!

Thank you again for the hard work!

Avanti

[NarineK]

Hi @AvantiShri, sure, that makes sense to me. We can take an additional custom function as an argument and in case it is not provided we can use our default logic. This shouldn't be a big change. I'm currently adding layer/neuron based support and will add this functionality as next.

[NarineK]

Hi Avanti, Thank you very much for the feedback! Happy to hear that you liked it.
I think that it shouldn't be an issue to return the multipliers.

Thank you very much for jupyter notebook. I'll check it out :)

[orionr]

Not going to close this, since we can consider it as a feature request that we aren't working on yet. Thanks, @AvantiShri!

[AvantiShri]

Awesome, thank you!

[AidenSb]

Hi @AvantiShri
Thanks for suggesting this. I have a question regarding the custom function and would really appreciate it if you could give some advice.
so I have used captum deeplift to calculate my attributions and followed the best recipe ( I basically run captum deep lift on my sequence with 10 different dinucleotides shuffled sequences and averaged over them).
Now I'm trying to get the hypothetical contributions and planning to use the same approach but this time input a custom function. I couldn't understand the example on the deepShap repo, and inputting it to the captum function is not working, would you be able to kindly suggest a simple formula to use the input, base and multiplier in a custom function? or if you already have a function that works for this matter, I would really appreciate it if you could share it.

[AvantiShri]

Hi Aiden, sorry for the late reply - your message came during a challenging time for me and it slipped off my radar. I prepared these slides to explain deepshap to my colleagues - perhaps they are useful? I start discussing the issue of how to combine the multipliers with the difference from reference for genomic data on slide 22: https://docs.google.com/presentation/d/1JCLMTW7ppA3Oaz9YA2ldDgx8ItW9XHASXM1B3regxPw/edit?usp=drivesdk

…

On Tue, 19 Jul 2022, 03:20 Aiden M Sababi, ***@***.***> wrote: Hi @AvantiShri <https://github.com/AvantiShri> Thanks for suggesting this. I have a question regarding the custom function and would really appreciate if you can give some advice. so I have used captum deeplift to calculate my attributions and followed the best recipe ( I basically run captum deep lift on my sequence with 10 different dinucleotides shuffled sequences and averaged over them). Now I'm trying to get the hypothetical contributions and planning to use the same approach but this time input a custom function. I couldn't make sense of the example on deepShap repo and inputting it to the captum function is not working, would you be able to kindly suggest a simple formula to use the input, base and multiplier in a custom function? or if you have already a function that works for this matter I would really appreciate if you can share. — Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARSFBV55R2ODRKYZXGPFT3VUXGRVANCNFSM4IY47J3A> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AidenSb]

@AvantiShri Thanks Avanti, I hope all is well now. The slides are really helpful I appreciate it.