zkmkarlsruhe / ofxtensorflow2 Goto Github PK
View Code? Open in Web Editor NEWTensorFlow 2 AI/ML library wrapper for openFrameworks
License: Other
TensorFlow 2 AI/ML library wrapper for openFrameworks
License: Other
Hello
Thanks for that addon, i want to move some python code i had to c++ in oF but i need to get a new machine.
Seems like now is very hard to find a 2080 so i must go for 3080 but this last one seems to work only with cuda 11 and tf > 2.4
Have you tested this addon with that config?
Thank you!!
i'm using 060_hair_segmentation model which is download from https://github.com/PINTO0309/PINTO_model_zoo/tree/main/060_hair_segmentation/10_tftrt
got those errors during the loading
SavedModel load for tags { serve }; Status: fail: Not found: Op type not registered 'CreateTRTResourceHandle' in binary running on SLDEV. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) tf.contrib.resampler
should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.. Took 175474 microseconds.
any ideas? since i'm not familiar with those model conversion stuff,any hints would be appreciated!
This was not easy for me (to find and understand a pretrained colorization model).
With this pretrained model I got it working (had to convert it to a saved model):
https://github.com/EnigmAI/Artistia/blob/main/ImageColorization/trained_models_v2/U-Net-epoch-100-loss-0.006095.hdf5
It tried it with this model before, but it seems they use two models together (do not really understand it yet): https://github.com/pvitoria/ChromaGAN
I converted the model like this (with Python):
from tensorflow.keras.models import Model, load_model
model = load_model(MODEL_FULLPATH)
model.save(MODEL_FULLPATH_MINUS_EXTENSION)
Anyway, here is the example: https://github.com/Jonathhhan/ofxTensorFlow2/tree/example_video_and_image_colorization/example_video_and_image_colorization
The ofxTF2::ThreadedModel inherits the load()
function from ofxTF2::Model which is a simple, blocking load. On slower systems, this can be relatively expensive as it will generally happen on the main thread.
We could look into adding asynchronous model loading on the background thread, possibly by an additional loadAsync()
function.
This addon was initially developed on Linux and macOS, but has not been tested on Windows. The layout, scripting, and project support files are set up, more or less, but we haven't been able to verify yet.
If you are a Windows developer, please let us know if this addon is working as expected in:
Hi.
Thanks for make this addon available to us.
I am on macOS 10.15.7, OF 0.11.2m Xcode 12.4
I followed the macOS install steps and get a bunch of compile errors related to arm64
Would you know why?
Should I rather try OF 0.11.0 ?
Should I be on a more recent macOS?
Undefined symbols for architecture arm64:
"_TF_NewSessionOptions", referenced from:
cppflow::model::model(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in ofxTF2Model.o
"_TF_NewBufferFromString", referenced from:
cppflow::model::model(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in ofxTF2Model.o
"_TF_NewGraph", referenced from:
cppflow::model::model(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in ofxTF2Model.o
"_TF_DeleteSessionOptions", referenced from:
cppflow::model::model(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in ofxTF2Model.o
"_TF_GraphNextOperation", referenced from:
cppflow::model::get_operations() const in ofxTF2Model.o
"_TF_GraphGetTensorNumDims", referenced from:
cppflow::model::get_operation_shape(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const in ofxTF2Model.o
"_TF_OperationName", referenced from:
cppflow::model::get_operations() const in ofxTF2Model.o
"_TF_GraphOperationByName", referenced from:
cppflow::model::operator()(std::__1::vector<std::__1::tuple<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, cppflow::tensor>, std::__1::allocator<std::__1::tuple<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, cppflow::tensor> > >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >) in ofxTF2Model.o
cppflow::model::get_operation_shape(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const in ofxTF2Model.o
"_TF_SessionRun", referenced from:
The script for downloading the models does not work:
mv: cannot stat 'download?path=%2F&files=model_basics.zip': No such file or directory
It is still possible with the direct link: https://cloud.zkm.de/index.php/s/gfWEjyEr9X4gyY6
I just came across this post, 3D Pose Detection with MediaPipe BlazePose GHUM and TensorFlow.js.
Do you have plans to make an example on how to use this library. I think it's TF.js pose-detection API ?
As always thank you for your efforts to make tensor flow accessible to the OF community.
I had this addon working successfully with OF 11.2 on a previous macOS system.
now I am on a M1 with macOS 12.6.8 and Xcode 14.1.0 and get these 2 errors:
CodeSign /Applications/of_v0.12.0_osx_release/addons/ofxTensorFlow2/example_video_matting/bin/example_video_mattingDebug.app/Contents/MacOS/libtensorflow_framework.dylib (in target 'example_video_matting' from project 'example_video_matting')
cd /Applications/of_v0.12.0_osx_release/addons/ofxTensorFlow2/example_video_matting
Signing Identity: "-"
/usr/bin/codesign --force --sign - --deep --timestamp\=none --preserve-metadata\=identifier,entitlements,flags --generate-entitlement-der /Applications/of_v0.12.0_osx_release/addons/ofxTensorFlow2/example_video_matting/bin/example_video_mattingDebug.app/Contents/MacOS/libtensorflow_framework.dylib
/Applications/of_v0.12.0_osx_release/addons/ofxTensorFlow2/example_video_matting/bin/example_video_mattingDebug.app/Contents/MacOS/libtensorflow_framework.dylib: main executable failed strict validation
Command CodeSign failed with a nonzero exit code
The .dylib is in the right place
and I think it has the right permissions
and then the main issue I guess is the Signing Identity: "-"
thing?
the same issue is mentioned but for a different project.
openframeworks/openFrameworks#6705
I am a bit lost.
other default OF examples compile and run just fine.
I am on macOS 12.6.3 with OF of_v20220530_osx_release on an M1.
I cloned the repo. Had trouble with the submodules so downloaded the cpp stuff manually.
Then ran these scripts
./scripts/download_tensorflow.sh
./scripts/download_example_models.sh
which placed things where they belong.
Then used PG on example_yolo_v4
Closed Xcode project.
Ran scripts/configure_xcode.sh example_yolo_v4
And then get the below error.
Would anyone know why?
Thx a bunch.
dyld[66435]: Library not loaded: '@rpath/libtensorflow_framework.2.dylib'
Referenced from: '/Applications/of_v20220530_osx_release/addons/ofxTensorFlow2/example_yolo_v4/bin/example_yolo_v4.app/Contents/MacOS/example_yolo_v4'
Reason: tried: '/Applications/of_v20220530_osx_release/addons/ofxTensorFlow2/example_yolo_v4/bin/libtensorflow_framework.2.dylib' (no such file),
'/usr/lib/system/introspection/libtensorflow_framework.2.dylib' (no such file),
'/usr/local/lib/libtensorflow_framework.2.dylib' (no such file),
'/usr/lib/libtensorflow_framework.2.dylib' (no such file)
i tried to use this model for music generation: https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/audio/music_generation.ipynb#scrollTo=1mil8ZyJNe1w
its very rough yet, and not sure if everything works as expected.
maybe you can find something wrong...
especially my replacement for tf.random.categorical(pitch_logits, num_samples=1)
could be wrong.
the model is very small, so it is included in the example...
https://github.com/Jonathhhan/ofxTensorFlow2/tree/music_generation_example/example_music_generation
Trying to get the code from here (as prompted by the README of the frozen_graph_char_rnn example) to work with TensorFlow2, but the API is based on TensorFlow1, which is deprecated. Is there any more recent Python example for training char RNNs for the current TensorFlow library?
I wonder if it is possible to implement stable diffusion or any other text-to-image network (dall-e for example).
https://github.com/CompVis/stable-diffusion
I guess, the complicated part is the pre-/postprocessing and converting the model to a saved_model...
Any hints are very welcome...
I'm trying to compile in MacOsX example_basic, but I'm having this error:
No member named 'exchange' in namespace 'std', in context.h line 78.
inline context::context(context&& ctx) noexcept : tfe_context(std::exchange(ctx.tfe_context, nullptr)) { }
inline context& context::operator=(context&& ctx) noexcept { tfe_context = std::exchange(ctx.tfe_context, tfe_context); return *this; }
I might be missing a step of the installation?
Add support for optimized libtensorflow builds for:
References:
I get the following access rights issue when following the Quick Start guide. Do I need to fork the add-on first?
stephanschulz@Stephans-Komputer ofxTensorFlow2 % git submodule update --init --recursive
Cloning into '/Applications/of_v20220530_osx_release/addons/ofxTensorFlow2/libs/cppflow'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:serizba/cppflow.git' into submodule path '/Applications/of_v20220530_osx_release/addons/ofxTensorFlow2/libs/cppflow' failed
Failed to clone 'libs/cppflow'. Retry scheduled
Cloning into '/Applications/of_v20220530_osx_release/addons/ofxTensorFlow2/libs/cppflow'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:serizba/cppflow.git' into submodule path '/Applications/of_v20220530_osx_release/addons/ofxTensorFlow2/libs/cppflow' failed
Failed to clone 'libs/cppflow' a second time, aborting
I trained a new model for neural style using the python files in your repo, that create a model is pb format instead of checkpoints as original repo.
But when i try to load the model in the of code i get a black screen and a msg on console
[warning] ofxTensorFlow2: shape mismatch at dimension 0 for lhs [480, 640, 3] and rhs [640, 480, 3]
I opened the model with Netron and i see a input called "serving_default_input_1" with size float32[?,640,480,3] so i don't understand whats happening
Can you give me some clue?
I am on macOS 12.6.8 with Xcode 141.0. and OF 12
I had this issue. I did add the needed line "$OF_PATH"/addons/ofxTensorFlow2/scripts/macos_install_libs.sh "$TARGET_BUILD_DIR/$PRODUCT_NAME.app";
to the 2nd run script but it did not work.
I had to add a new run script field by clicking on the +
and then add the line as required.
roymacdonald also suggested this option:
in the addons addon_config.mk file
add the following to the osx section
ADDON_LIBS = libs/tensorflow/lib/osx/libtensorflow_framework.2.8.0.dylib
ADDON_LIBS += libs/tensorflow/lib/osx/libtensorflow_framework.2.dylib
ADDON_LIBS += libs/tensorflow/lib/osx/libtensorflow_framework.dylib
ADDON_LIBS += libs/tensorflow/lib/osx/libtensorflow.2.8.0.dylib
ADDON_LIBS += libs/tensorflow/lib/osx/libtensorflow.2.dylib
ADDON_LIBS += libs/tensorflow/lib/osx/libtensorflow.dylib
ADDON_LDFLAGS = -Xlinker -rpath -Xlinker @executable_path
make sure there is a tab at the begining of each line
Hello!!
Im playing with this to help me to understand how tensorflow c pi works.
I have a centernet model that has one input and 4 outputs
The given SavedModel SignatureDef contains the following input(s):
inputs['input_tensor'] tensor_info:
dtype: DT_UINT8
shape: (1, -1, -1, 3)
name: serving_default_input_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['detection_boxes'] tensor_info:
dtype: DT_FLOAT
shape: (1, 100, 4)
name: StatefulPartitionedCall:0
outputs['detection_classes'] tensor_info:
dtype: DT_FLOAT
shape: (1, 100)
name: StatefulPartitionedCall:1
outputs['detection_scores'] tensor_info:
dtype: DT_FLOAT
shape: (1, 100)
name: StatefulPartitionedCall:2
outputs['num_detections'] tensor_info:
dtype: DT_FLOAT
shape: (1)
name: StatefulPartitionedCall:3
Method name is: tensorflow/serving/predict
I load the model and set input and output names
model.load("model");
vector<string> inputs{"serving_default_input_tensor:0"};
vector<string> outputs{"StatefulPartitionedCall:0",
"StatefulPartitionedCall:1",
"StatefulPartitionedCall:2",
"StatefulPartitionedCall:3"};
model.setup(inputs,outputs);
Add one dimension to input to follow model requeriments
input = cppflow::expand_dims(input, 0);
auto output = model.runModel(input);
And now how i acces to the output data? I tried this but dont work, i expected that the return of runModel was a vector of vectors.... i tried reading the cppflow doc but no luck
auto detection_boxes = output[0];
auto detection_classes = output[1];
auto detection_scores = output[2];
auto num_detections = output[3];
Thanks!
GPU training using main.py
with pix2pix is not working on my setup. The CPU training and the oF examples are working.
First I was missing .dll such as cudart64_101.dll or cusolver64_10.dll , i added the .dll from other folder or changed the name as suggested here https://stackoverflow.com/questions/65608713/tensorflow-gpu-could-not-load-dynamic-library-cusolver64-10-dll-dlerror-cuso
Ultimately it started training but blocking on Epoch 1 then, epoch 1 stays at 0% and the training continue. After training the resulting predicated image output is a full black images.
my setup:
Windows 11
RTX 3090 driver 516.01
conda environement with python 3.7 and the packages version from requirments.txt
CUDA 11.7 and CUDNN 8.4.1
Hello
I moved/ported/coded/play or whatever is called correctly this repo https://github.com/PeterL1n/RobustVideoMatting to use it with you addon.
Is supercool, is interesting to add to examples? It has multiple inputs/outputs and use initial recurrent states.
I wonder if somebody use a word2vec model with ofxTensorflow2?
I found a model and tried to use it, but without success: https://tfhub.dev/google/Wiki-words-250-with-normalization/2
For now I use ofxWord2vec https://github.com/perevalovds/ofxWord2Vec for finding subtitle similarities (with a pretrained model) and I would like to do that with ofxTensorflow2 too...
https://github.com/Jonathhhan/ofEmscriptenExamples/tree/main/montageautomat_threaded_2
Thank you so so much for making this.
It hopefully removes the need for use JS to run a lot of the models and send their info to OF over OSC. As the awesome Lingdong had done here: https://github.com/LingDong-/PoseOSC
Do you have advice on how to make other models like PoseNet or the mediapipe handpose from tensorflow.js work with your addon ?
Again, thanks so much for making this.
Stephan.
I posted it already in the OF forum. Here is a Yolo v4 example: https://forum.openframeworks.cc/t/yolo-v4-example-with-ofxtensorflow2/40040
I am on macOS 10.15.7 with OF 0.11.0 and Xcode 12.4
I successfully compiled and ran all provided examples except example_keyword_spotting
Not sure why or how to fix it.
I don't have an immediate need for it but thought you might like to know.
/Applications/of_v0.11.0_osx_release/addons/ofxTensorFlow2/src/ofxTensorFlow2Utils.h:119:14: Non-constant-expression cannot be narrowed from type 'size_t' (aka 'unsigned long') to 'long long' in initializer list
Thanks,
Stephan.
Hello!!
What do you think to add support to use frozen graph? That way you can use a pb created in tf1, so is more compatible.... i'm doing that and works ok. Old cppflow had that option
Hello
Im trying to convert some neural style models im using on python. I have those model in checkpoints
If i understand correctly (sure is not the case) i need to run the checkpoint2SavedModel.py to set the input layer size to expected size and get the model in savedModel format
Problem is that i get a very extrange result and sure you have more experience than me on those so maybe you can give me some clues on what im doing wrong
Thanks!
I try to run the arbitrary-image-stylization-v1-256 model: https://tfhub.dev/google/magenta/arbitrary-image-stylization-v1-256/2
I can load the model without any errors, but it chrashes if I try to run the model (without an error message).
This is how I try to run it:
if(!model.load("magenta")) {
std::exit(EXIT_FAILURE);
}
model.setup({ {"serving_default_placeholder:0"} ,{"serving_default_placeholder_1:0"} }, { {"StatefulPartitionedCall:0"} });
auto input_1 = cppflow::decode_jpeg(cppflow::read_file(std::string(ofToDataPath("example_1.jpg"))));
input_1 = cppflow::cast(input_1, TF_UINT8, TF_FLOAT);
input_1 = cppflow::expand_dims(input_1, 0);
auto input_2 = cppflow::decode_jpeg(cppflow::read_file(std::string(ofToDataPath("example_2.jpg"))));
input_2 = cppflow::cast(input_2, TF_UINT8, TF_FLOAT);
input_2 = cppflow::expand_dims(input_2, 0);
std::vector<cppflow::tensor> vectorOfInputTensors = {
input_1, input_2
};
auto output = model.runModel(vectorOfInputTensors);
Any idea what I am doing wrong (or if the model runs at all with ofxTensorflow2)? Thank you.
Hi friends,
I'm having M1-architecture-related linking problems ("Undefined Symbol" etc.) similar to those described in Issues #15 and #3. I have tried the posted fixes without luck, and also (possibly) discovered a problem with one of the scripts. My configuration is:
I have followed the posted instructions as follows:
git submodule update --init --recursive
); and I downloaded the models using ./scripts/download_example_models.sh.CLANG_CXX_LANGUAGE_STANDARD = c++14
suchly in CoreOF.xcconfig. I also updated PLATFORM_CXXFLAGS += -std=c++14
suchly at line 142 of config.osx.default.mk.$OF_PATH/addons/ofxTensorFlow2/scripts/macos_install_libs.sh "$TARGET_BUILD_DIR/$PRODUCT_NAME.app";
Unfortunately, I'm still getting Undefined Symbols for all of the Tensorflow stuff:
I feel like I'm close, but I'm out of ideas for how to get further. Thanks so much for any help.
Side Note 1: The ofxTensorFlow2 script, configure_xcode.sh
, appears to be corrupting XCode 14 projects. After I run this script, .xcodeproj files no longer open, and XCode reports the error: "The project ‘example_video_matting’ is damaged and cannot be opened due to a parse error. Examine the project file for invalid edits or unresolved source control conflicts."
Side Note 2: FWIW, TensorFlow.org is now offering version 2.9.1 for download (though, there is no indication that it works for arm64).
This was the most difficult for me to understand. But now it works quite well, I guess:
https://github.com/Jonathhhan/ofxTensorFlow2/tree/example_chatbot/example_chatbot
https://forum.openframeworks.cc/t/ofxtensorflow2-chatbot-almost-solved/40090/9
As noted by @golanlevin, the configure_xcode.sh configure script appears to break Xcode projects. Whoops.
It comes down to how the build script is stored as an escaped string internally in the .pbxproj file... it does not match how the script contents are displayed in the Xcode editor, so bad assumption on my part.
I managed to train images in format 256x256 and tried with format 512x512
in config.py
IMG_WIDTH = 512
IMG_HEIGHT = 512
got errors the following errors
Epoch 1 going on....
0%| | 0/40 [00:00<?, ?it/s]2022-06-26 00:18:30.959669: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
0%| | 0/40 [00:00<?, ?it/s]
Traceback (most recent call last):
File "main.py", line 96, in <module>
checkpoint, checkpoint_prefix
File "C:\Users\plesk\Documents\coding\of_v0.11.2_vs2017_release\addons\ofxTensorFlow2\example_pix2pix\python\src\training.py", line 143, in train
fit(train_dataset, test_dataset, EPOCHS)
File "C:\Users\plesk\Documents\coding\of_v0.11.2_vs2017_release\addons\ofxTensorFlow2\example_pix2pix\python\src\training.py", line 130, in fit
for input_image, target in tqdm(train_ds):
File "C:\Users\plesk\.conda\envs\inpaintEnv37\lib\site-packages\tqdm\std.py", line 1167, in __iter__
for obj in iterable:
File "C:\Users\plesk\.conda\envs\inpaintEnv37\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 761, in __next__
return self._next_internal()
File "C:\Users\plesk\.conda\envs\inpaintEnv37\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 747, in _next_internal
output_shapes=self._flat_output_shapes)
File "C:\Users\plesk\.conda\envs\inpaintEnv37\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 2727, in iterator_get_next
_ops.raise_from_not_ok_status(e, name)
File "C:\Users\plesk\.conda\envs\inpaintEnv37\lib\site-packages\tensorflow\python\framework\ops.py", line 6941, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [Need value.shape >= size, got ] [2 286 286 3] [2 512 512 3]
[[{{node StatefulPartitionedCall/random_crop/Assert/Assert}}]] [Op:IteratorGetNext]
Is it possible to train larger sizes?
Is it possible training rectangular format images?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.