Comments (6)
I'd also like to do this for analysing text as a sequence. It says in Module google/nnlm-en-dim50-with-normalization/1 that it "preprocesses its input by removing punctuation and splitting on spaces". But the result is as said above. Are the results of embedding lookup for each word then combined back into a single vector output for the sequence?
How can we get the sequence of tokens with word embedding back rather than just (batch_size, embedding_size)
result?
from hub.
Yes, the results of embedding lookup for each word is combined back into a single vector output for the sequence (phrase, sentence, etc.).
You can pass individual words to the module instead of phrases or sentences, and then you'll get the embedding vectors for the individual words that you can use/combine in any way you want. Some embeddings use n-grams in their vocabularies, so it's usually preferable to let the module do tokenization internally, but for the application you mention you might want to tokenize/combine yourself.
from hub.
@svsgoogle I agree that we can preprocess ourselves, but the input dimension expected is TensorShape([Dimension(None)])
which doesn't work if we are passing a batch of data to the module. Is there another way of passing a batched dataset?
Example
import tensorflow_hub as hub
import numpy as np
embed = hub.Module("https://tfhub.dev/google/nnlm-en-dim50-with-normalization/1")
data = np.asarray(["cat is on the mat".split(), "dog is in the fog".split()])
print(data.shape) # (2, 5) i.e. batch_size = 2, sequence_length = 5
embed(data)
Output
TypeError: Can't convert 'default': Shape TensorShape([Dimension(2), Dimension(5)]) is incompatible with TensorShape([Dimension(None)])
from hub.
Assuming that after preprocessing, your string tensor is a dense tensor (this will be needed to feed into LSTM anyway), you can reshape to [None] before passing to the module, then reshape back:
words = tf.constant(["cat is on the mat".split(), "dog is in the fog".split()])
words = tf.reshape(words, [-1])
result = embed(words)
result = tf.reshape(result, [2, 5, 128]) # the second array can be constructed with tf.concat, tf.shape(words) and [-1].
from hub.
For now, I used nnlm-en-dim128 embedding https://www.tensorflow.org/hub/modules/google/nnlm-en-dim128/1 which gives me acceptable accuracy.
Thanks,
Sachin B. Ichake
from hub.
A small example of using this from @vbardiovskyg's code snippet:
SEQ_LENGTH = 5
EMBEDDING_DIM = 50
with tf.Graph().as_default() as g:
embed_layer = hub.Module(
f"https://tfhub.dev/google/nnlm-en-dim{EMBEDDING_DIM}-with-normalization/1",
trainable=False, name='text_embedding')
sentences = tf.placeholder(dtype=tf.string, shape=(None, SEQ_LENGTH))
batch_size = tf.shape(sentences)[0]
flat_sentences = tf.reshape(sentences, [-1])
embeddings = embed_layer(flat_sentences)
sentence_embedding = tf.reshape(embeddings,
[batch_size, SEQ_LENGTH, EMBEDDING_DIM])
with tf.Session(graph=g) as sess:
sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())
output = sess.run(sentence_embedding, feed_dict={
sentences: [
"cat is on the mat".split(),
"dog is in the fog".split(),
"padded sentence UNK UNK UNK".split()]
})
print(output.shape)
# (3, 5, 50)
You'd want better handling of padding/trimming sequences, but I normally do that with the tf.data
API
from hub.
Related Issues (20)
- Bug: AttributeError: module 'tensorflow_hub.tf_v1' has no attribute 'estimator' HOT 1
- Feature request: TensorFlow Hub and HuggingFace integration HOT 3
- Bug: Missing function register_module_for_export HOT 4
- Bug: Getting CORS issue in getting facemesh model.json HOT 3
- Bug: TF Hub is not compatible with Keras 3 and TensorFlow 2.16+ HOT 3
- Bug: 403 CORS error -tfhub/imagenet/mobilenet. Issue was last night, resolved this morning. Occurred again 3/26 1:47pm est HOT 12
- Model import issue HOT 3
- Tensorflow hub upgrades Tensorflow?
- Bug: Building tensorflow 2.15/2.16 from source is not possible : Missing tensorrt HOT 2
- Bug: CORS fetching model face-detection HOT 14
- Bug: Getting CORS error while loading tensorflow-models/pose-detection, tensorflow-models/face-detection from today HOT 6
- Bug: AttributeError: module 'numpy' has no attribute 'object'. HOT 2
- Update object detection link in web site HOT 3
- Bug: Intermittent CORS issues on qna model fetch
- Trouble Running TensorFlow v2.16.1 with NVIDIA GeForce 940MX GPU HOT 2
- Bug: tfds.load split error
- <Android> Tensorflow crashs on Android15 ( with 16kb page size support)
- Not compatible with actual TF 2.17
- Bug: Unable to create a sequential model using a TensorFlow Hub model
- Bug: Why is the TensorFlow audio classification tutorial no longer providing accurate predictions? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hub.