akosiorek / stacked_capsule_autoencoders Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
stacked_capsule_autoencoders's People
Stargazers
Watchers
Forkers
ll752003446 fabio-deep shadowkun cinjon czgdp1807 biprateep damehou loongtop dancinglinks cxiang26 ishitamed19 sjoerdteunisse raul1968 xianxuhua wwy99stacked_capsule_autoencoders's Issues
Error occurs when adding "--plot=True" arguement for CCAE
Hello, Akosiorek! Thanks for sharing the code.
I notice that you implemented the visualization methods for CCAE, so I tried to add --plot=True argument for CCAE, but the following error shows when training, do you and anybody else have the same problem?
My arguments for running CCAE:
--name=constellation
--model=constellation
--dataset=constellation
--prior_within_example_sparsity_weight=1.
--prior_between_example_sparsity_weight=1.
--posterior_within_example_sparsity_weight=0.
--posterior_between_example_sparsity_weight=0.
--overwrite
error:
INFO:tensorflow:Running local_init_op.
I1115 21:15:43.785577 140489565673280 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I1115 21:15:44.053265 140489565673280 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into stacked_capsule_autoencoders/checkpoints/constellation_with_plot/model.ckpt.
I1115 21:16:50.631423 140489565673280 basic_session_run_hooks.py:606] Saving checkpoints for 0 into stacked_capsule_autoencoders/checkpoints/constellation_with_plot/model.ckpt.
2020-11-15 21:17:06.914638: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at scatter_nd_op.cc:133 : Invalid argument: indices[0,6] = [0, 64, 55] does not index into shape [32,64,64,3]
File "/home/qiangzibro/stacked_capsule_autoencoders/stacked_capsule_autoencoders/train.py", line 230, in <module>
tf.app.run()
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "/home/qiangzibro/stacked_capsule_autoencoders/stacked_capsule_autoencoders/train.py", line 195, in main
train_itr, _ = sess.run([global_step, update_ops])
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 754, in run
run_metadata=run_metadata)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 1360, in run
raise six.reraise(*original_exc_info)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/six.py", line 703, in reraise
raise value
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 1345, in run
return self._sess.run(*args, **kwargs)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 1418, in run
run_metadata=run_metadata)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/training/monitored_session.py", line 1176, in run
return self._sess.run(*args, **kwargs)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
indices[0,6] = [0, 64, 55] does not index into shape [32,64,64,3]
[[node ScatterNd (defined at home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1748) ]]
Original stack trace for 'ScatterNd':
File "snap/pycharm-professional/218/plugins/python/helpers/pydev/pydevd.py", line 2141, in <module>
main()
File "snap/pycharm-professional/218/plugins/python/helpers/pydev/pydevd.py", line 2132, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "snap/pycharm-professional/218/plugins/python/helpers/pydev/pydevd.py", line 1441, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "snap/pycharm-professional/218/plugins/python/helpers/pydev/pydevd.py", line 1448, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "snap/pycharm-professional/218/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "home/qiangzibro/stacked_capsule_autoencoders/stacked_capsule_autoencoders/train.py", line 230, in <module>
tf.app.run()
File "home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "home/qiangzibro/stacked_capsule_autoencoders/stacked_capsule_autoencoders/train.py", line 143, in main
plot_dict, plot_params = model.make_plot(trainset, 'train')
File "home/qiangzibro/stacked_capsule_autoencoders/stacked_capsule_autoencoders/capsules/models/model.py", line 43, in wrapper
self.memo[hashed_args] = func(*args, **kwargs)
File "home/qiangzibro/stacked_capsule_autoencoders/stacked_capsule_autoencoders/capsules/models/model.py", line 126, in make_plot
plots = self._plot(data, res)
File "home/qiangzibro/stacked_capsule_autoencoders/stacked_capsule_autoencoders/capsules/models/constellation.py", line 258, in _plot
caps_presence_prob=res.caps_presence_prob,
File "home/qiangzibro/stacked_capsule_autoencoders/stacked_capsule_autoencoders/capsules/plot.py", line 169, in render_constellations
colored = render_by_scatter(canvas_size, pred_points, color, pred_presence)
File "home/qiangzibro/stacked_capsule_autoencoders/stacked_capsule_autoencoders/capsules/plot.py", line 125, in render_by_scatter
return tf.scatter_nd(idx, colors, shape)
File "home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_array_ops.py", line 8782, in scatter_nd
"ScatterNd", indices=indices, updates=updates, shape=shape, name=name)
File "home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 1748, in __init__
self._traceback = tf_stack.extract_stack()
> /home/qiangzibro/stacked_capsule_autoencoders/lib/python3.7/site-packages/tensorflow_core/python/client/session.py(1384)_do_call()
-> raise type(e)(node_def, op, message)visualization
Hi!
Nice work! I noticed that in your paper there is a visualization of templates. However, I did not found it in this repo. Also I was wondering that if you can give a visualization of OCAE feature and reconstruction result.
What does CPR and CCR stand for?
They are in capsule.py.
Pre trained model for MNIST
Hi,
Is there a way to reproduce results mentioned in the paper via pre trained model? Can you share weights file? Thanks!
Explaination of pose calculation from logits of PCAE encoder
Hi,
Could you please explain how you came with the affine transform matrix from pose logits (1x6) from the output of PCAE encoder (function here). It would be really helpful if you could point to relevant source ?
Also, I was wondering what could be way of working around of calculating 3D pose if we had pose logits for objects in three dimension space (say 3x3 rotation matrix or quarternions) ?
Thanks
Saksham
No module named 'stacked_capsule_autoencoders'
I have installed all the dependencies as mentioned in the README.md and all requirements mentioned in requirement.txt also satisfies, but it is still giving following error :
/home/Abhishek/.local/lib/python3.6/site-packages/pandas/compat/init.py:117: UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
warnings.warn(msg)
Traceback (most recent call last):
File "train.py", line 33, in
from stacked_capsule_autoencoders.capsules.configs import data_config
ModuleNotFoundError: No module named 'stacked_capsule_autoencoders'
I also tried running from the google-research-master folder but the error is persistent.
routing -= (1. - presence) * 1e32
Hi, it is a great model and thank you for sharing the code. I have one question about how the attention is calculated in "QKVAttention". I'm having difficulty to understand how multiplying such a large number, 1e32, to the "presence" of part capsules would help with the "routing". I think the variants among the presence would overwhelm the difference in routing. Could you explain in detail please? Thank you!
Config issue
What was the config you used for CIFAR? I am asking even though it's written in the paper because it seems like the MNIST config in the repo is different from what is listed in the paper.
The paper says:
We used an PCAE with 24 single-channel 11 × 11 templates for MNIST and 24 and 32 three-channel14 × 14 templates for SVHN and CIFAR10, respectively. We used sobel-filtered images as the reconstruction target for SVHN and CIFAR10, as in Jaiswal et al., 2018, while using the raw pixel intensities as the input to PCAE. The OCAE used 24, 32 and 64 object capsules, respectively.
However, there seems to be 40 single-channel 11x11 templates and 32 object capsules in the MNIST config. The hyperparameter settings are also all different from what's in the paper.
Thoughts? The config in the repo clearly works (we reproduced it), but we'd like to have the correct one for other data going forward, especially others with larger sizes as we're finding that the MNIST config doesn't work as well when the size of the data is increased.
no "normalized_sobel_edges"
stacked_capsule_autoencoders\capsules\models\scae.py
line 158
if prep == 'sobel':
img = preprocess.normalized_sobel_edges(img)
but there is no "normalized_sobel_edges" in
stacked_capsule_autoencoders\capsules\data\preprocess.py
Load image dataset from directory in place of MNIST dataset
I am trying to load my dataset in place of MNIST, changes I made in image.py are as follows :
dataset = tf.data.Dataset.from_generator(lambda: func(subset, batch_size, **kwargs),{'label': tf.int64, 'image' : tf.uint8}, {'label': (None,),'image':(None,1200,2000,1)})
def _create_mydataset(subset, batch_size, **kwargs):
img_list = []
images_path = sorted(glob.glob('update_207/*.png'))
print(images_path)
for img in images_path:
image_string = tf.read_file(img)
img_1 = tf.io.read_file(img)
n= tf.image.decode_png(img_1)
n = tf.image.rgb_to_grayscale(n, name=None)
n = tf.image.resize(n,size =[1200,2000])
n = tf.cast(n,tf.uint8)
img_list.append(n)
index_1 = int(0.8 * len(img_list))
print(subset)
if subset == 'train':
img_list_train = img_list[:index_1]
for i in range(0,index_1,batch_size):
img_dict = {}
if (i+batch_size<=index_1):
img_list_train_batch = img_list_train[i:i+batch_size]
else:
img_list_train_batch = img_list_train[i:i+index_1]
img_list_train_batch = np.array(img_list_train_batch)
img_dict = {'label':0,'image':img_list_train_batch}
yield img_dict
elif subset == 'test':
img_list_test = img_list[index_1+1:len(img_list)]
for i in range(index_1+1,len(img_list),batch_size):
img_dict = {}
if (i+batch_size <= len(img_list)):
img_list_test_batch = img_list_test[i:i+batch_size]
else :
img_list_test_batch = img_list_test[i:i+len(img_list)]
img_list_test_batch = np.array(img_list_test_batch)
img_dict = {'label':0 ,'image':img_list_test_batch}
yield img_dict
But I am getting the following error :
ValueError: generator yielded an element of shape (0,) where an element of shape (?, 1200, 2000, 1) was expected.
Intermediate values are as follows:
<DatasetV1Adapter shapes: {label: (?,), image: (?, 1200, 2000, 1)}, types: {label: tf.int64, image: tf.uint8}>
{'label': <tf.Tensor 'args_1:0' shape=(?,) dtype=int64>, 'image': <tf.Tensor 'args_0:0' shape=(?, 1200, 2000, 1) dtype=uint8>}
{'validset': {'label': <tf.Tensor 'IteratorGetNext_1:1' shape=(40,) dtype=int64>, 'image': <tf.Tensor 'IteratorGetNext_1:0' shape=(40, 40, 40, 1) dtype=float32>}, 'trainset': {'label': <tf.Tensor 'IteratorGetNext:1' shape=(40,) dtype=int64>, 'image': <tf.Tensor 'IteratorGetNext:0' shape=(40, 40, 40, 1) dtype=float32>}}
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.