Comments (1)
Thank you for the in depth and thoughtful review. We're glad that you found our work exciting! We've responded to points inline below.
One comment on this: while using visual examples definitely gives a finer degree of granularity than lumping several abstractions together under one description -- e.g. “floppy ear detectors” -- it seems likely that any visual concept that the network reliably learns that is unfamiliar to humans is likely to still be overlooked, even using canonical examples. (This would be an example of the human scale issue). One way to study this further (probably in future work) might be to extract semantic concepts that are repeatedly learned, and thus identify these important concepts that aren’t immediately human recognizable.
We agree! The factorization approach we used is only one from a larger family of techniques, and we’re excited about future research that explores the impact of using these other techniques. We’ve made this clearer with 790f4c9.
I would have really liked to see a simple equation or two (maybe even as an aside), so that the mathematically minded readers can have a (simplified) mathematical description.
We want to keep the focus of the article on how the various interpretability building blocks come together to form rich user interfaces. The challenge is that digging into the details of each of the individual building blocks involves many moving parts (e.g., with feature visualization, we would need to unpack initialization, parameterization, optimization objective, procedure, etc.; with matrix factorization, it’s unclear to us how more formal mathematic description would not essentially reduce to redefining NMF).
With that said, we’ve improved our prose description of our approaches in 3c4312b, and added a link to our previous feature visualization article (which discusses our specific approach at length) in 90979c4.
Is this kind of influence visualization robust to other attribution methods?
Our interfaces reify attribution methods, so we believe that they will be as robust as the attribution methods they rely on. We can use the same interface but swap out the underlying attribution method used, and we think doing so (and evaluating the resultant interfaces) is a promising direction for future work.
It is good to see the human scale problem being addressed but the results here appear to be preliminary, and there is likely much more work to be done in the future.
We agree, and have clarified this as of 790f4c9.
Have the authors considered methods to automatically cluster together similar concepts? (E.g. first collecting a set of canonical examples corresponding to a certain class, and then applying clustering to see what natural groups they fall into?)
Yes! This is one of the things we are actively researching, but unfortunately don’t have conclusive results yet to include in the article :)
Without having read other distill articles, it’s not immediately obvious that the little orange box selecting a certain part of the image can actually be moved around, and it might be worthwhile noting this somewhere.
We’ve improved our annotations and controls for all diagrams (e.g., with 5f9cebb, we’ve added new zooming controls for this particular interface).
from post--building-blocks.
Related Issues (20)
- Review #1 HOT 1
- Review #2 HOT 1
- spelling mistake HOT 1
- In teaser image, hovering over spatial locations doesn't change the feature visualization HOT 2
- Typo HOT 1
- Some errors in "Spatial Attribution with Saliency Maps" visualization HOT 3
- spelling: trusthworthy -> trustworthy HOT 1
- Spritemap notebooks HOT 3
- MobileNet Visualization HOT 1
- Misspelling in notebook HOT 4
- events.js:141 throw er; // Unhandled 'error' event HOT 1
- Module build failed: TypeError: Cannot read property 'watchRun' of undefined in AllActivationGrids.html HOT 3
- Question regarding "we use dimensionality reduction to produce a multi-directional saliency map" HOT 8
- typo HOT 1
- Some <d-figure> tags failing to render with Firefox
- Error while using a different model than inception_v1 in your notebook, unable to evaluate gradients (inception_v4) HOT 1
- Colab notebook does not work as is
- Tough to reproduce flickering bug HOT 1
- When example changes, default to interesting position HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from post--building-blocks.