Comments (6)
Hi @ltroin, can you be more specific about the model or component you're trying to use? LITs model wrappers generally assume the data will be sent in as represented in the dataset, so typically text, numbers, or image bytes, not pre-computed tensors. Components (interpreters, projectors, etc) are more flexible but also tend toward assuming the incoming data will be the same as the dataset or whatever the model outputs. That said, we should be able to help you adapt/customize the LIT code you want to use to fit your needs.
from lit.
You can use tensor data for Embeddings
, TokenEmbeddings
, or similar types - you will just want to be sure to convert it to numpy arrays as LIT expects plain-old-data that can be serialized between Python and Javascript.
from lit.
Hi @ltroin, can you be more specific about the model or component you're trying to use? LITs model wrappers generally assume the data will be sent in as represented in the dataset, so typically text, numbers, or image bytes, not pre-computed tensors. Components (interpreters, projectors, etc) are more flexible but also tend toward assuming the incoming data will be the same as the dataset or whatever the model outputs. That said, we should be able to help you adapt/customize the LIT code you want to use to fit your needs.
Thank you for the quick response. I'm currently working with the LLama model, which accepts two types of inputs: tokenized text or input embeddings. I'm interested in using the input embeddings to analyze and visualize the patterns of attention weights across different layers and create a salience map based on "tokens", where each "token" corresponds to a row of input embeddings.
from lit.
You can use tensor data for
Embeddings
,TokenEmbeddings
, or similar types - you will just want to be sure to convert it to numpy arrays as LIT expects plain-old-data that can be serialized between Python and Javascript.
Does this imply that the LIT framework will interpret this numpy array as input embeddings rather than raw text?
from lit.
Does this imply that the LIT framework will interpret this numpy array as input embeddings rather than raw text?
tl;dr -- Probably not. NumPy arrays are not raw text and LIT expects to operate over language-native string
values when dealing with strings, so if you passed a NumPy array in a place where it was expecting a Python str
you would most likely get a ValueError
.
Longer explanation -- LIT operates over a JSON Object representation of examples. Since JSON has very limited support for types, LIT provides its own type system that our TypeScript and Python codebases use to decide how to handle different values in the JSON Objects we pass around. Model and Dataset classes declare the shape (i.e., field names and types) of the JSON Objects that provide/accept as Specs, and components (interpreters, generators, metrics, and UI modules) look for specific LIT types in these Specs to determine compatibility and decide how to handle them. As above, you should expect a ValueError
, etc. if you pass around a non-conforming value for the expected type in LIT.
I'm currently working with the LLama model
We don't have a wrapper for Llama in an official release yet, but we're working on adding one in #1421. It's designed to take raw text as the input and then the HF implementation handles tokenization, embedding, generation, etc. It would be possible for you to subclass this (or write your own) so that the wrapper class takes embeddings as input instead of raw text.
I'm interested in using ... [analyzing and visualizing] patterns of attention weights across different layers and create a salience map based on "tokens", where each "token" corresponds to a row of input embeddings.
LIT provides a Sequence Salience module that renders a salience map over tokenized text. Is that the kind of thing you're looking for, or do you want to display a salience map over a matrix of shape <float>(num_tokens, hidden_dims)
, or maybe even something else?
from lit.
Hi, sorry for the late reply.
Thank you so much for the detailed explanation.
We don't have a wrapper for Llama in an official release yet, but we're working on adding one in #1421.
This feature will be awesome! And yes, I also want to display a salience map over a matrix of shape (num_tokens, hidden_dims), where certain num_tokens are highlighted.
from lit.
Related Issues (20)
- Would LiT support Object Detection Computer Vision models? HOT 1
- Defective pip dependencies: numba, numpy and protobuf HOT 3
- Most important words for false positives / false negatives
- Integration with TransformerLens ?
- LIT demo notebook not working HOT 8
- error when using the pytorch demo HOT 2
- How to deploy this on a webserver HOT 2
- Potential performance issue: .to_dict method slow in pandas below 2.2 HOT 1
- Running LIT for a fine-tuned LLM HOT 4
- Default docker install instructions shows scary errors and seems to fail... HOT 6
- Default source install instructions shows scary errors and seems to fail... HOT 2
- Excessive Duplicated Sentences in LIME Text Output HOT 3
- Widget on Google Cloud Vertex AI Workbench Notebook HOT 12
- TCAV with a DeBERTa model HOT 2
- Unable to Render LIT Widget in Jupyter Notebook on Server HOT 3
- Relax shap version HOT 4
- Errors while trying to run the examples HOT 7
- How to analyze remote models on a server? HOT 1
- In the demo, only single-tower structures are shown. Are there examples of twin-tower structures, similar to Sentence-BERT? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lit.