xinghaochen / pose-ren Goto Github PK

Hi! Thanks for sharing this code.
I wan't to retrain the dataset Hands17 but I'm confused about how to crop the hand from the depth images,which is from the Hands17 dataset. Is the same as the demo? I have already tried some ways but it dosen't work well.

hi, if I used c++, should I replace all the Python Layer as this :
layer {
name: "rois_6"
type: "Python"
bottom: "prev_pose"
top: "rois_6"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 6, 'img_h': 96}"
}
}

To

layer {
name: "rois_15"
type: "GenerateROI"
bottom: "prev_pose"
top: "rois_15"
generate_roi_param {
img_w: 96
roi_h: 6
spatial_mul: 8
roi_w: 6
joint_idx: 17
img_h: 96
}
}

I find it's hard for me to train your proposed model to get the same good result as you have achieved. Therefore, can you release the training code for us to better understand the whole training process or just provide the code for data preprocessing.

Are they predicted by the model or parameters provided by the camera?
Thank you very much.

Thank you for sharing the pre-trained net with the hands17 dataset.

I tried to run the real-time demo with the new network after renaming them, but failed.

I guess the the current code repository seems to be updated to support hands17.

I checked several places in the source codes, but lost to track.

Traceback (most recent call last):
File "realsense_realtime_demo_librealsense2.py", line 93, in
main()
File "realsense_realtime_demo_librealsense2.py", line 85, in main
results = hand_model.detect_image(depth)
File "/home/joohaeng/Downloads/Pose-REN-master/src/utils/model_pose_ren.py", line 74, in detect_image
return self.detect_images([img])[0, ...]
File "/home/joohaeng/Downloads/Pose-REN-master/src/utils/model_pose_ren.py", line 58, in detect_images
init_poses = self._net_init.forward()['fc3']
KeyError: 'fc3'

Hello!Thank you very much for sharing! I run your project with kinect2 online and the result is terrible! I don't know why. I've change the fx, fy, ux, uy,lower_,upper_ and dataset parameters as possible to fit my device. Can you help me to check the code? Thank you in advance!
realtime_kinect.txt

I want to detect the two hands of a picture at the same time. Firstly, I will find out the bounding boxes of the two hands with the method of object detection, and then I will bring out the depth information of the picture in the two enveloping boxes and use the model to detect them respectively. Is this feasible?

what should I do

Hi.

I am interested in your project and have some question.

How many times did you do data augmentation?

and, how much time did need all to train in your project?(baseline and pose-ren)

Last, I wonder that which GPU device perform training in your project?

Thank you!!

System Specs:
Ubuntu 18.04
NVIDIA RTX 2080 Ti (11GB)
NVIDIA Driver version: 430.50
CUDA version: 10.1
CUDNN version: 7.6.5
python: 2.7.17

I have compiled caffe-pose with CUDNN support and have added it to the python path.
I'm trying to run the realsense python demo using the command: python src/demo/realsense_realtime_demo_librealsense2.py

Following is the stack-trace of the crash:

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0217 15:53:40.758397 28195 net.cpp:51] Initializing net from parameters:
state {
phase: TEST
level: 0
}
layer {
name: "data"
type: "Input"
top: "data"
input_param {
shape {
dim: 1
dim: 1
dim: 96
dim: 96
}
}
}
layer {
name: "prev_pose"
type: "Input"
top: "prev_pose"
input_param {
shape {
dim: 1
dim: 48
}
}
}
layer {
name: "conv0"
type: "Convolution"
bottom: "data"
top: "conv0"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 16
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: "msra"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu0"
type: "ReLU"
bottom: "conv0"
top: "conv0"
}
layer {
name: "conv1"
type: "Convolution"
bottom: "conv0"
top: "conv1"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 16
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: "msra"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "pool1"
type: "Pooling"
bottom: "conv1"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "relu1"
type: "ReLU"
bottom: "pool1"
top: "pool1"
}
layer {
name: "conv2_0"
type: "Convolution"
bottom: "pool1"
top: "conv2_0"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 32
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: "msra"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu2_0"
type: "ReLU"
bottom: "conv2_0"
top: "conv2_0"
}
layer {
name: "conv2"
type: "Convolution"
bottom: "conv2_0"
top: "conv2"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 32
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: "msra"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu2"
type: "ReLU"
bottom: "conv2"
top: "conv2"
}
layer {
name: "conv3"
type: "Convolution"
bottom: "conv2"
top: "conv3"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 32
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: "msra"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "res1"
type: "Eltwise"
bottom: "conv2_0"
bottom: "conv3"
top: "res1"
}
layer {
name: "pool2"
type: "Pooling"
bottom: "res1"
top: "pool2"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "relu3"
type: "ReLU"
bottom: "pool2"
top: "pool2"
}
layer {
name: "conv3_0"
type: "Convolution"
bottom: "pool2"
top: "conv3_0"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 64
pad: 0
kernel_size: 1
stride: 1
weight_filler {
type: "msra"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu3_0"
type: "ReLU"
bottom: "conv3_0"
top: "conv3_0"
}
layer {
name: "conv4"
type: "Convolution"
bottom: "conv3_0"
top: "conv4"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: "msra"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu4"
type: "ReLU"
bottom: "conv4"
top: "conv4"
}
layer {
name: "conv5"
type: "Convolution"
bottom: "conv4"
top: "conv5"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
stride: 1
weight_filler {
type: "msra"
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "res2"
type: "Eltwise"
bottom: "conv3_0"
bottom: "conv5"
top: "res2"
}
layer {
name: "pool3"
type: "Pooling"
bottom: "res2"
top: "pool3"
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: "relu5"
type: "ReLU"
bottom: "pool3"
top: "pool3"
}
layer {
name: "rois_0"
type: "Python"
bottom: "prev_pose"
top: "rois_0"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 0, 'img_h': 96}"
}
}
layer {
name: "roi_pool_0"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_0"
top: "roi_pool_0"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_0"
type: "InnerProduct"
bottom: "roi_pool_0"
top: "fc1_0"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_0"
type: "ReLU"
bottom: "fc1_0"
top: "fc1_0"
}
layer {
name: "drop1_0"
type: "Dropout"
bottom: "fc1_0"
top: "fc1_0"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_1"
type: "Python"
bottom: "prev_pose"
top: "rois_1"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 1, 'img_h': 96}"
}
}
layer {
name: "roi_pool_1"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_1"
top: "roi_pool_1"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_1"
type: "InnerProduct"
bottom: "roi_pool_1"
top: "fc1_1"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_1"
type: "ReLU"
bottom: "fc1_1"
top: "fc1_1"
}
layer {
name: "drop1_1"
type: "Dropout"
bottom: "fc1_1"
top: "fc1_1"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_3"
type: "Python"
bottom: "prev_pose"
top: "rois_3"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 3, 'img_h': 96}"
}
}
layer {
name: "roi_pool_3"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_3"
top: "roi_pool_3"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_3"
type: "InnerProduct"
bottom: "roi_pool_3"
top: "fc1_3"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_3"
type: "ReLU"
bottom: "fc1_3"
top: "fc1_3"
}
layer {
name: "drop1_3"
type: "Dropout"
bottom: "fc1_3"
top: "fc1_3"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_4"
type: "Python"
bottom: "prev_pose"
top: "rois_4"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 4, 'img_h': 96}"
}
}
layer {
name: "roi_pool_4"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_4"
top: "roi_pool_4"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_4"
type: "InnerProduct"
bottom: "roi_pool_4"
top: "fc1_4"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_4"
type: "ReLU"
bottom: "fc1_4"
top: "fc1_4"
}
layer {
name: "drop1_4"
type: "Dropout"
bottom: "fc1_4"
top: "fc1_4"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_6"
type: "Python"
bottom: "prev_pose"
top: "rois_6"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 6, 'img_h': 96}"
}
}
layer {
name: "roi_pool_6"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_6"
top: "roi_pool_6"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_6"
type: "InnerProduct"
bottom: "roi_pool_6"
top: "fc1_6"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_6"
type: "ReLU"
bottom: "fc1_6"
top: "fc1_6"
}
layer {
name: "drop1_6"
type: "Dropout"
bottom: "fc1_6"
top: "fc1_6"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_7"
type: "Python"
bottom: "prev_pose"
top: "rois_7"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 7, 'img_h': 96}"
}
}
layer {
name: "roi_pool_7"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_7"
top: "roi_pool_7"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_7"
type: "InnerProduct"
bottom: "roi_pool_7"
top: "fc1_7"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_7"
type: "ReLU"
bottom: "fc1_7"
top: "fc1_7"
}
layer {
name: "drop1_7"
type: "Dropout"
bottom: "fc1_7"
top: "fc1_7"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_9"
type: "Python"
bottom: "prev_pose"
top: "rois_9"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 9, 'img_h': 96}"
}
}
layer {
name: "roi_pool_9"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_9"
top: "roi_pool_9"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_9"
type: "InnerProduct"
bottom: "roi_pool_9"
top: "fc1_9"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_9"
type: "ReLU"
bottom: "fc1_9"
top: "fc1_9"
}
layer {
name: "drop1_9"
type: "Dropout"
bottom: "fc1_9"
top: "fc1_9"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_10"
type: "Python"
bottom: "prev_pose"
top: "rois_10"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 10, 'img_h': 96}"
}
}
layer {
name: "roi_pool_10"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_10"
top: "roi_pool_10"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_10"
type: "InnerProduct"
bottom: "roi_pool_10"
top: "fc1_10"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_10"
type: "ReLU"
bottom: "fc1_10"
top: "fc1_10"
}
layer {
name: "drop1_10"
type: "Dropout"
bottom: "fc1_10"
top: "fc1_10"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_12"
type: "Python"
bottom: "prev_pose"
top: "rois_12"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 12, 'img_h': 96}"
}
}
layer {
name: "roi_pool_12"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_12"
top: "roi_pool_12"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_12"
type: "InnerProduct"
bottom: "roi_pool_12"
top: "fc1_12"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_12"
type: "ReLU"
bottom: "fc1_12"
top: "fc1_12"
}
layer {
name: "drop1_12"
type: "Dropout"
bottom: "fc1_12"
top: "fc1_12"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_13"
type: "Python"
bottom: "prev_pose"
top: "rois_13"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 13, 'img_h': 96}"
}
}
layer {
name: "roi_pool_13"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_13"
top: "roi_pool_13"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_13"
type: "InnerProduct"
bottom: "roi_pool_13"
top: "fc1_13"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_13"
type: "ReLU"
bottom: "fc1_13"
top: "fc1_13"
}
layer {
name: "drop1_13"
type: "Dropout"
bottom: "fc1_13"
top: "fc1_13"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "rois_15"
type: "Python"
bottom: "prev_pose"
top: "rois_15"
python_param {
module: "python_layers.py_generate_roi_layer"
layer: "PyGenerateROILayer"
param_str: "{'img_w': 96, 'roi_h': 6, 'spatial_mul': 8, 'roi_w': 6, 'joint_idx': 15, 'img_h': 96}"
}
}
layer {
name: "roi_pool_15"
type: "ROIPooling"
bottom: "pool3"
bottom: "rois_15"
top: "roi_pool_15"
roi_pooling_param {
pooled_h: 7
pooled_w: 7
spatial_scale: 0.125
}
}
layer {
name: "fc1_15"
type: "InnerProduct"
bottom: "roi_pool_15"
top: "fc1_15"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu6_15"
type: "ReLU"
bottom: "fc1_15"
top: "fc1_15"
}
layer {
name: "drop1_15"
type: "Dropout"
bottom: "fc1_15"
top: "fc1_15"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "concate_1_0"
type: "Concat"
bottom: "fc1_0"
bottom: "fc1_1"
bottom: "fc1_3"
top: "concate_1_0"
}
layer {
name: "fc2_0"
type: "InnerProduct"
bottom: "concate_1_0"
top: "fc2_0"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu7_0"
type: "ReLU"
bottom: "fc2_0"
top: "fc2_0"
}
layer {
name: "drop2_0"
type: "Dropout"
bottom: "fc2_0"
top: "fc2_0"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "concate_1_1"
type: "Concat"
bottom: "fc1_0"
bottom: "fc1_4"
bottom: "fc1_6"
top: "concate_1_1"
}
layer {
name: "fc2_1"
type: "InnerProduct"
bottom: "concate_1_1"
top: "fc2_1"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu7_1"
type: "ReLU"
bottom: "fc2_1"
top: "fc2_1"
}
layer {
name: "drop2_1"
type: "Dropout"
bottom: "fc2_1"
top: "fc2_1"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "concate_1_2"
type: "Concat"
bottom: "fc1_0"
bottom: "fc1_7"
bottom: "fc1_9"
top: "concate_1_2"
}
layer {
name: "fc2_2"
type: "InnerProduct"
bottom: "concate_1_2"
top: "fc2_2"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu7_2"
type: "ReLU"
bottom: "fc2_2"
top: "fc2_2"
}
layer {
name: "drop2_2"
type: "Dropout"
bottom: "fc2_2"
top: "fc2_2"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "concate_1_3"
type: "Concat"
bottom: "fc1_0"
bottom: "fc1_10"
bottom: "fc1_12"
top: "concate_1_3"
}
layer {
name: "fc2_3"
type: "InnerProduct"
bottom: "concate_1_3"
top: "fc2_3"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu7_3"
type: "ReLU"
bottom: "fc2_3"
top: "fc2_3"
}
layer {
name: "drop2_3"
type: "Dropout"
bottom: "fc2_3"
top: "fc2_3"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "concate_1_4"
type: "Concat"
bottom: "fc1_0"
bottom: "fc1_13"
bottom: "fc1_15"
top: "concate_1_4"
}
layer {
name: "fc2_4"
type: "InnerProduct"
bottom: "concate_1_4"
top: "fc2_4"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 2048
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
layer {
name: "relu7_4"
type: "ReLU"
bottom: "fc2_4"
top: "fc2_4"
}
layer {
name: "drop2_4"
type: "Dropout"
bottom: "fc2_4"
top: "fc2_4"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc_concat"
type: "Concat"
bottom: "fc2_0"
bottom: "fc2_1"
bottom: "fc2_2"
bottom: "fc2_3"
bottom: "fc2_4"
top: "fc_concat"
}
layer {
name: "fc3_0"
type: "InnerProduct"
bottom: "fc_concat"
top: "fc3_0"
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 48
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
}
}
}
I0217 15:53:40.759402 28195 layer_factory.hpp:77] Creating layer data
I0217 15:53:40.759414 28195 net.cpp:84] Creating Layer data
I0217 15:53:40.759421 28195 net.cpp:380] data -> data
I0217 15:53:40.766059 28195 net.cpp:122] Setting up data
I0217 15:53:40.766084 28195 net.cpp:129] Top shape: 1 1 96 96 (9216)
I0217 15:53:40.766091 28195 net.cpp:137] Memory required for data: 36864
I0217 15:53:40.766096 28195 layer_factory.hpp:77] Creating layer prev_pose
I0217 15:53:40.766108 28195 net.cpp:84] Creating Layer prev_pose
I0217 15:53:40.766113 28195 net.cpp:380] prev_pose -> prev_pose
I0217 15:53:40.766140 28195 net.cpp:122] Setting up prev_pose
I0217 15:53:40.766144 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766147 28195 net.cpp:137] Memory required for data: 37056
I0217 15:53:40.766151 28195 layer_factory.hpp:77] Creating layer prev_pose_prev_pose_0_split
I0217 15:53:40.766160 28195 net.cpp:84] Creating Layer prev_pose_prev_pose_0_split
I0217 15:53:40.766163 28195 net.cpp:406] prev_pose_prev_pose_0_split <- prev_pose
I0217 15:53:40.766170 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_0
I0217 15:53:40.766177 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_1
I0217 15:53:40.766183 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_2
I0217 15:53:40.766189 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_3
I0217 15:53:40.766196 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_4
I0217 15:53:40.766202 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_5
I0217 15:53:40.766208 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_6
I0217 15:53:40.766213 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_7
I0217 15:53:40.766219 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_8
I0217 15:53:40.766225 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_9
I0217 15:53:40.766232 28195 net.cpp:380] prev_pose_prev_pose_0_split -> prev_pose_prev_pose_0_split_10
I0217 15:53:40.766324 28195 net.cpp:122] Setting up prev_pose_prev_pose_0_split
I0217 15:53:40.766330 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766332 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766335 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766340 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766345 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766350 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766355 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766357 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766361 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766366 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766369 28195 net.cpp:129] Top shape: 1 48 (48)
I0217 15:53:40.766372 28195 net.cpp:137] Memory required for data: 39168
I0217 15:53:40.766376 28195 layer_factory.hpp:77] Creating layer conv0
I0217 15:53:40.766387 28195 net.cpp:84] Creating Layer conv0
I0217 15:53:40.766391 28195 net.cpp:406] conv0 <- data
I0217 15:53:40.766396 28195 net.cpp:380] conv0 -> conv0
F0217 15:53:40.766815 28195 cudnn_conv_layer.cpp:53] Check failed: status == CUDNN_STATUS_SUCCESS (1 vs. 0) CUDNN_STATUS_NOT_INITIALIZED
*** Check failure stack trace: ***
Aborted (core dumped)

I've traced the error to this line in the code, which is basically when the net is getting loaded. I get the same problem even whne I try it with the NYU model instead of the ICVL model.

On googling the error, most solutions point to this link, which basically suggests to compile caffe without CUDNN support. When I try it, it works perfectly giving me the required output (although at CPU speeds).

Any pointers about why it might be an issue? is it due to CUDA and CUDNN version mismatch? Any insights about how to solve this issue?

Thanks in advance.

你好，感谢你对手势估计的研究和领域信息的汇总。我想自己重新训练Pose-REN，但是在你提供的caffe-pose中没有发现可以读取手势数据集并引入数据增强的层的实现，请问下能否提供一下呢？还是说是通过别的方法，比如预先处理到本地再来读取的方式来实现。

I'm very interested in training Pose-REN from the scratch.

Do we need a special code for this, which is not included in this GitHub, or is it possible to train with a well-defined prototxt accompanied with well-augmented data?

If a control code is required, any outline hint will be appreciated!

Best,

Joo-Haeng

@xinghaochen Hi thanks for this wonderful repo , i had following queries

1. At what fps does it run
   2.does it run on CPU /GPU

I could build and run the real-time demo using R415 and 435 cameras. The estimation is not good as your demo with R300. Do I need to make some configuration changes for these cameras to get better results?

And, it will be great if I could test the net trained with Hands17 dataset.

Thank you for sharing your work.

hi,
thanks for sharing your project, I have two questions here,

1. will the cube size of hand influence the result? for example, change it from 300 to 250, how will the predict result be? have you done some experiments for comparison?
2. which is the better way for data augmentation? just augment for every sample or choose some of them (for example, half probability)?
   thank you!

Hi, thanks for this wonderful repo . I had following queries：
What is "import config" in src/testing/predict.py?Is there a config file missing?

I met some problem on installation of caffe.
When I built caffe-pose，it blocks me:

CXX .build_release/src/caffe/proto/caffe.pb.cc
CXX src/caffe/solvers/sgd_solver.cpp
CXX src/caffe/util/hdf5.cpp
CXX src/caffe/util/io.cpp
CXX src/caffe/util/math_functions.cpp
CXX src/caffe/layer.cpp
CXX src/caffe/syncedmem.cpp
CXX src/caffe/parallel.cpp
CXX src/caffe/layer_factory.cpp
CXX src/caffe/data_transformer.cpp
CXX src/caffe/layers/hdf5_output_layer.cpp
CXX src/caffe/layers/sigmoid_layer.cpp
CXX src/caffe/layers/threshold_layer.cpp
CXX src/caffe/layers/elu_layer.cpp
CXX src/caffe/layers/im2col_layer.cpp
CXX src/caffe/layers/lstm_layer.cpp
CXX src/caffe/layers/multinomial_logistic_loss_layer.cpp
src/caffe/layers/hdf5_output_layer.cpp:3:10: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
^~~~~~~~
compilation terminated.
Makefile:581: recipe for target '.build_release/src/caffe/layers/hdf5_output_layer.o' failed
make: *** [.build_release/src/caffe/layers/hdf5_output_layer.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from src/caffe/util/hdf5.cpp:1:
./include/caffe/util/hdf5.hpp:6:10: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
^~~~~~~~
compilation terminated.
Makefile:581: recipe for target '.build_release/src/caffe/util/hdf5.o' failed
make: *** [.build_release/src/caffe/util/hdf5.o] Error 1
In file included from src/caffe/solvers/sgd_solver.cpp:5:
./include/caffe/util/hdf5.hpp:6:10: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
^~~~~~~~
compilation terminated.
Makefile:581: recipe for target '.build_release/src/caffe/solvers/sgd_solver.o' failed
make: *** [.build_release/src/caffe/solvers/sgd_solver.o] Error 1

Thus, how can I resolve such issues?
Thanks & Regards!

你好，请问一下，你编译的caffe-pose是否对protobuf的版本有要求，是一定要2.6.1版本吗？

您好，请问如果已经安装过caffe，再安装caffe-pose会有什么影响吗？

Hi,thanks for sharing your project, I have two questions here：
1 Is the training data layer defined in python?
2 In the training label of icvl data set, there are 48 floating-point numbers. How to realize multi-label training?