yscacaca / deepsense Goto Github PK
View Code? Open in Web Editor NEWDeepsense: a unified deep learning framework for time-series mobile sensing data processing.
Deepsense: a unified deep learning framework for time-series mobile sensing data processing.
Why is CONV_MERGE_LEN = 8?
The shape of sensor_conv_in
is (64, 20, 2, 4, 64). So shouldn't CONV_MERGE_LEN be 4 so that it matches matches the features from each sensor?
In this line for the function layers.batch_norm you have used
updates_collections=None
So what if we use updates_collections=tf.GraphKeys.UPDATE_OPS
In order to work this we need to explicitly update the GraphKeys class rel event to moving averages .
update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
with tf.control_dependencies(update_ops):
train_op = optimizer.minimize(loss)
Also the TF document says ,
One can set updates_collections=None to force the updates in place, but that can have a speed penalty, especially in distributed settings.
In your paper you write that you first apply a 2d convolution to X^(k) and then two 1d convolution. Finally you write that you flatten the matrix to stack the vectors of each sensor.
When I look at your code on Github, I see that you don't flatten the matrix after the individual convolution and use a 3d convolution layer .
Question:
Is there an important reason why you didn't flatten the matrix and use a 3d convolution for the matrix X^(3)?
Here you have put WIDE = 20 as the length of a sequence .
You also use tf.nn.dynamic in order to work with different length sequences .
In order to make that happen you have a tensor - length that calculate the length of the sequence in each batch.
But as my understanding all the input examples have fixed size . My problem is in the real world scenario we can get time steps with only zeros . Then the number of time steps can be changed. That means we can't say always this number is fixed WIDE*FEATURE_DIM .
What should I do ?
I download the data, unzip it in ./ and run deepSense_HHAR_tf.py, and got this error:
Traceback (most recent call last):
File "deepSense_HHAR_tf.py", line 257, in <module>
_, lossV, _trainY, _predict = sess.run([discOptimizer, loss, batch_label, predict])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 10)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
[[Node: shuffle_batch/_19477 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_2574_shuffle_batch", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
Caused by op u'shuffle_batch', defined at:
File "deepSense_HHAR_tf.py", line 209, in <module>
batch_feature, batch_label = input_pipeline(csvFileList, BATCH_SIZE)
File "deepSense_HHAR_tf.py", line 59, in input_pipeline
min_after_dequeue=min_after_dequeue)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 1217, in shuffle_batch
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/input.py", line 788, in _shuffle_batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/data_flow_ops.py", line 457, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 946, in _queue_dequeue_many_v2
timeout_ms=timeout_ms, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
self._traceback = _extract_stack()
OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 10)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
[[Node: shuffle_batch/_19477 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_2574_shuffle_batch", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
The "current size" changes randomly for different run. If set shuffle_sample as False, it also gives an error but with RandomShuffleQueue changed to FIFOQueue.
Any recommendation to solve it? Thanks!
I noticed that the data you shared is different from the original data, can you share your code that for data processing? Thanks for your sharing!!!
In this This line.
These things mainly control how to extract a batch from the que. So assigning them incorrectly can cause errors. So can you please explain ?
In this line what I have understood is you represent the label as an one hot encoding . Any way in the csv file where can I find the label?
I would like to test the neural network in an Android app, by importing the frozen and optimized model with TF's TensorFlowInferenceInterface
. I have edited the deepSense_HHAR_tf.py
source to export the evaluation model in PB and PBTXT formats, and exported a checkpoint using tf.train.Saver
. Then, I applied TF's freeze_graph
tool.
In the Android application I can successfully load the model and feed the input parsed from the CSV files, but trying to run it yields the following exception:
No OpKernel was registered to support Op 'FIFOQueueV2' with these attrs. Registered devices: [CPU], Registered kernels: no registered kernels
This is probably due to Android's TensorFlow library not including the specified Op.
I have then applied TF's optimize_for_inference
tool to the frozen graph, and imported the output in the Android app, but with this approach an exception is immediately thrown upon loading the model:
NodeDef expected inputs '' do not match 1 inputs specified; Op<name=Const; signature= -> output:dtype; attr=value:tensor; attr=dtype:type>; NodeDef: deepSense_1/rnn/while/add/y = Const[dtype=DT_INT32, value=Tensor<type: int32 shape: [] values: 1>](deepSense_1/rnn/while/Switch:1)
Do you have some advice on exporting the model for Android, based on your experience with the Nexus 5?
anyone can run the code without problem? I am using tensorflow 1.8.0 with python 3.5 but keep getting problem to run the code, currently I am stuck with plot.py for y_vals.
The error occur mention about unhashable type : 'dict_keys'
I hope someone can help me if you have successfully run the code
In this line you do the convolution2D operation with acceleration inputs(acc_inputs) . The size has mentioned as follows ,
(BATCH_SIZE, WIDE, FEATURE_DIM, CHANNEL=1)
So I have this problem . Batch_Size is for number of examples in a mini batch . But here you have to do the convolution operation for acceleration input of each time step . In the acc_inputs do we have each time step inputs . And can we do the convolution in parallel way ?
Hi,
We are trying to understand the preprocessing and we ran into a couple of questions.
In the preprocessed dataset, for each file (let's say eval_1.csv), we have 2406 values in one row. We know that the last 6 elements compose the one hot representation of ground truth. Can you elaborate on how the remaining 2400 points are generated.
Thanks for your time,
Hi I have modified this code to run on multiple GPUs referencing Cifar 10 multi GPU example . But then I am not sure how to do batch normalization in distributed training . Specially
Calculating moving averages for use in the inference . How to do this ?
I tried train this model with data provided in readme, I find the training procedure is slow. It already has taken more than 2 hours to just iterate about 700 times.
I watch the usage of gpu and cpu, gpu usage is about 10%~20%, sometimes even low to 0%~3%, cpu usage is about 50%, sometimes drop to near 0.
CPU usage diagram is as below
My laptop is with intel i7 6700k and gtx 1050ti. I run the code in pycharm with python 2.7 and tensorflow 1.3.
Nice work!
Would you release the training/eval codes for the CarTrack task?
Thanks!
Why this is bit slow . What is the reason ? Do you think it's because of the #5 issue the input/output stream ?
Hello,
Thanks for your work. Recently I have read the DeepSense and SADeepSense and want to run the code on my computer, but I haven't found the SADeepSense code. I'd appreciate it if you could share the SADeepSense code.
Best wishes!
In the keras there's TimeDistributed wrapper where we can use to do convolution for each time step. So I think by feeding the tensor with Batch Size , Sequence Length , .. etc we can have the same property. Am I right ?
Hi,
I've read your code for sep_HHAR_data and compared with the paper. I found that you set the fftSpan=5 (correspond to 0.5 s) while the paper take 0.25 s. So, can you confirm your code?
Thanks Yao!
Hi, I'm working on the cat tracking problem with DeepSense, you said that
I think the number of time intervals(i.e "T" in your paper) is different in different samples, and my question is how to fix this problem, to pad 0 in pre-processing or slove it in the model?
When computing loss with the optimizer object does tensorflow add regularization loss automatically for all the parameters ?
Hi,thank you for your release.I run your code in computer successfully.I deploy the *.pb on android,but I meet the issue like this:No OpKernel was registered to support Op 'All' with these attrs. Registered devices: [CPU], Registered kernels: .
I find the 'All' op used in the deepSense/rnn/All layer.
Could you give me some advices to fix the bug?
Thank you very much!
Hi, I want to learn about DeepSense using my own gait data collection but for gait recognition purpose not for activity recognition, recorded with 50Hz in a csv files, with 150 participants, the format is timestamp followed by 3 axis acc and 3 axis gyro. How I should preprocess the data to become like your eval or train dataset?
because my sampling rate is fix to get 50 samples per seconds, should I still perform the time interpolation?
I would really appreciate it if you have some explanation about your dataset. Thank you
I am using your network for my research and wanted to evaluate your network on android.
The Tensorflow version you used to generate the frozen graph is TF 1.4 according to the documentation.
I trained and generated a pb file but I am still confused how to run it on android. Currently Tensorflow has TF Lite. I tried to convert the pb file to tflite but I get errors of compatibility.
Tensorflow Lite was not available yet on TF 1.4.
How did you run the model on android without using TensorFlow lite?
(I have limited knowledge about android apps and java)
Hi,I am very new to Deep Learning .If anyone has implemented the code for GAIT ANALYSIS and a brief explanation of the code implemented ,that would be great. Also the code for preprocessing the data would also help.
Thanks
Hi,
Thank you very much for sharing your code. I downloaded your package and I tried to run it in Ubuntu 16.04 tf-1.1. I have received an error when the function tf.train.shuffle_batch is executed within the function input_line. The output error is:
Caused by op u'shuffle_batch', defined at:
File "deepSense_HHAR_tf.py", line 215, in
batch_feature, batch_label = input_pipeline(csvFileList, BATCH_SIZE)
File "deepSense_HHAR_tf.py", line 64, in input_pipeline
min_after_dequeue=min_after_dequeue)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 1214, in shuffle_batch
name=name)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/training/input.py", line 784, in _shuffle_batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 458, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 1328, in _queue_dequeue_many_v2
timeout_ms=timeout_ms, name=name)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
op_def=op_def)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/vol/users/gzc/DeepSense/my_project/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1228, in init
self._traceback = _extract_stack()
OutOfRangeError (see above for traceback): RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 8)
[[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
Suggestions?
Thanking in advance.
Gabriel.
Hi @yscacaca ,
Thank you for sharing your source code.
I tried to run your source code but faced the below error:
OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 64, current size 14
I tried to change tensorflow version 10 times between 1.1 to 1.10 with python 2.7 and it didn't solve the problem.
I've read the previous issues on this problem and didn't find a proper solution.
Is it possible to share the packages versions here? like python, tensorflow, ...
Or share all requirement packages versions as a yml file?
It gives me the error on the below line:
example_batch, label_batch = tf.train.shuffle_batch( [example, label], batch_size=batch_size, num_threads=16, capacity=capacity, min_after_dequeue=min_after_dequeue)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.