Comments (10)
For now one has to do it manually, example:
import tensorflow as tf
import tensorflow_hub as hub
with tf.Graph().as_default():
module = hub.Module("http://tfhub.dev/google/universal-sentence-encoder/2")
text = tf.placeholder(tf.string, [None])
embedding = module(text)
init_op = tf.group([tf.global_variables_initializer(), tf.tables_initializer()])
with tf.Session() as session:
session.run(init_op)
tf.saved_model.simple_save(
session,
"/tmp/serving_saved_model",
inputs = {"text": text},
outputs = {"embedding": embedding}
)
Each modules differs slightly from others in input/output names, additionally each serving use case might have different requirements (e.g. raw features in vs serialized tf.Example protos). By having users creating the graph they want to serve (e.g. as done above) seems more flexible than require users to guess what Servo config to use and/or modify the client side each time they change the module being served.
from hub.
+1
Making hub modules consumable by Tensorflow Serving would be very helpful.
from hub.
Any workarounds for this currently? I want to serve a hub module using Tensorflow Serving.
from hub.
I have not tested this, but I that suspect you could load the module into an empty graph and then export with savedmodelbuilder
. We could build an automated tool.
from hub.
Closing as this is now obsolete.
In TF-2 users should create reusable saved models with tf.saved_model.save().
from hub.
@andresusanopinto quick question, does it mean the pre-trained model on tensorflow hub won't have the serving_default
signature by default and the user need to re-export it? Thanks
from hub.
A TF2 SavedModel can both have signatures for deployment to TensorFlow Serving and tf.functions for reuse in a Python TensorFlow program.
See https://www.tensorflow.org/hub/tf2_saved_model#advanced_topic_what_to_expect_from_the_savedmodel_after_loading and https://www.tensorflow.org/guide/saved_model
from hub.
@arnoegw thanks for the pointer. I noticed some inconsistency of some TF2 models that some have serving_default
signature while some does not. For example:
- Only
default
signature: - No signature: "https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/feature_vector/4" (from the example notebook)
- Only
serving_default
https://tfhub.dev/tensorflow/resnet_50/classification/1
My understanding is if serving_default
is not present, the model cannot be served as-is in TF serving. Should all TF2 models have this signature? Let me know if I should open a new issue to track this.
from hub.
@chenliu0831, there's nothing wrong with those examples:
- The following are hub.Module assets for TF1 that use SavedModel's SignatureDef to define the signatures that hub.Module can invoke. They cannot be served as-is, because the hub.Module format and SavedModels for serving had some small but crucial differences (which were removed for TF2).
https://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1
https://tfhub.dev/google/openimages_v4/ssd/mobilenet_v2/1 - The following (and many more with similar names) is a TF2 SavedModel meant only for reuse in a Python TensorFlow program (put a to-be-learned layer on top of the features), so it has a
@tf.function __call__
but not SignatureDefs:
https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/feature_vector/4 - The following is a TF2 SavedModel meant both for reuse and stand-alone serving, so it has both a
@tf.function __call__
and a serving signature:
https://tfhub.dev/tensorflow/resnet_50/classification/1
from hub.
@arnoegw Ah thanks for clarifying. I give a bad example for the second case since that's a feature vector.
I spot check another few on TF hub with the TF2 filter and with classification
variants and looks like their signature map is empty as well:
- https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/classification/4
- https://tfhub.dev/google/imagenet/nasnet_large/classification/4
- https://tfhub.dev/google/imagenet/mobilenet_v2_035_160/classification/4
In the detail page of above models I think they all show up as "TF2.0 Saved Model" format.
from hub.
Related Issues (20)
- Bug: AttributeError: module 'tensorflow_hub.tf_v1' has no attribute 'estimator' HOT 1
- Feature request: TensorFlow Hub and HuggingFace integration HOT 3
- Bug: Missing function register_module_for_export HOT 4
- Bug: Getting CORS issue in getting facemesh model.json HOT 3
- Bug: TF Hub is not compatible with Keras 3 and TensorFlow 2.16+ HOT 3
- Bug: 403 CORS error -tfhub/imagenet/mobilenet. Issue was last night, resolved this morning. Occurred again 3/26 1:47pm est HOT 12
- Model import issue HOT 3
- Tensorflow hub upgrades Tensorflow?
- Bug: Building tensorflow 2.15/2.16 from source is not possible : Missing tensorrt HOT 2
- Bug: CORS fetching model face-detection HOT 14
- Bug: Getting CORS error while loading tensorflow-models/pose-detection, tensorflow-models/face-detection from today HOT 6
- Bug: AttributeError: module 'numpy' has no attribute 'object'. HOT 2
- Update object detection link in web site HOT 3
- Bug: Intermittent CORS issues on qna model fetch
- Trouble Running TensorFlow v2.16.1 with NVIDIA GeForce 940MX GPU HOT 2
- Bug: tfds.load split error
- <Android> Tensorflow crashs on Android15 ( with 16kb page size support)
- Not compatible with actual TF 2.17
- Bug: Unable to create a sequential model using a TensorFlow Hub model
- Bug: Why is the TensorFlow audio classification tutorial no longer providing accurate predictions? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hub.