tangyuhao / goturn-tensorflow Goto Github PK

I am honestly confused about the usage.
I would appreciate your guidance!
Say i have a video, and say i extract all the frames of the video:

00000.jpg
00001.jpg
00002.jpg
etc...

Now i have the following coordinates for the bounding box of frame 00000.jpg: <xmin, ymin, xmax, ymax>

how do i configure the test_set.txt file so that the network outputs bounding box predictions for all the other frames?

Do i need to preprocess the video so that the target in 00000.jpg is cropped and placed in the target folder?

INFO:tensorflow:Restoring parameters from ./checkpoints/checkpoint.ckpt-1
Traceback (most recent call last):

File "", line 1, in
runfile('/Users/davidparker/Desktop/MOT/code/tensorGoturn/load_and_test.py', wdir='/Users/davidparker/Desktop/MOT/code/tensorGoturn')

File "/Users/davidparker/anaconda3/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 705, in runfile
execfile(filename, namespace)

File "/Users/davidparker/anaconda3/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "/Users/davidparker/Desktop/MOT/code/tensorGoturn/load_and_test.py", line 104, in
saver.restore(sess, ckpt.model_checkpoint_path)

File "/Users/davidparker/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1743, in restore
err, "a Variable name or other graph key that is missing")

NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Key fc1_1/biases not found in checkpoint
[[Node: save_1/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save_1/Const_0_0, save_1/RestoreV2/tensor_names, save_1/RestoreV2/shape_and_slices)]]

I downloaded the ckpt from the given google drive address and only decompressed it in the root dir. The code is exactly the same as the git post.
Is anyone else also seeing this?

Hi,

I modified the code so that now it uses the predictions of the previous frame to crop the image and predict the object position in the current frame, but the tracker always predicts a bbox which is centred in the cropped frame even though the object is not there.
`The "Try" Loop

for i in range(0, int(len(train_box)/BATCH_SIZE)):
	cur_batch = sess.run(batch_queue)
	if(i> 0):
		print "before:",cur_batch[2]
		for x in range(BATCH_SIZE):
			cur_batch[2][x][0] = 10.0*((fc4.astype(np.float64)[x][0]*prev_frames_info[x][2]/10.0)+prev_frames_info[x][0])/227.0
			cur_batch[2][x][1] = 10.0*((fc4.astype(np.float64)[x][1]*prev_frames_info[x][3]/10.0)+prev_frames_info[x][1])/227.0
			cur_batch[2][x][2] = 10.0*((fc4.astype(np.float64)[x][2]*prev_frames_info[x][2]/10.0)+prev_frames_info[x][0])/227.0
			cur_batch[2][x][3] = 10.0*((fc4.astype(np.float64)[x][3]*prev_frames_info[x][3]/10.0)+prev_frames_info[x][1])/227.0
		print "after :",cur_batch[2]
		print "\n"
		prev_frames_info = []
	for c in range(BATCH_SIZE):
		ser = cur_batch[0][c]
		tar = cur_batch[1][c]
		[x1,y1,x2,y2]  = map(int,(227*cur_batch[2][c])/10.0)
		hi = int(round(abs(y2-y1)))
		y1 = max(y1-hi/2,1)
		wi = int(round(abs(x2-x1)))   
		x1 = max(x1-wi/2,1)
		ser_resz = ser[y1:y1+2*hi+1,x1:x1+2*wi+1]
		tar_resz = tar[y1:y1+2*hi+1,x1:x1+2*wi+1]
		if(i==0):
			prev_frames_info.append([x1,y1,ser_resz.shape[1],ser_resz.shape[0]])
			height_prev = ser_resz.shape[0]
			width_prev = ser_resz.shape[1]
		else:
			prev_frames_info.append([x1,y1,width_prev,height_prev])
		cur_batch[0][c] = scipy.misc.imresize(ser_resz,(227,227))
		cur_batch[1][c] = scipy.misc.imresize(tar_resz,(227,227))
	start_time = time.time()
	[batch_loss, fc4] = sess.run([tracknet.loss, tracknet.fc4],feed_dict={tracknet.image:cur_batch[0],tracknet.target:cur_batch[1], tracknet.bbox:cur_batch[2]})
	logging.info('batch box: %s' %(fc4))
	logging.info('gt batch box: %s' %(cur_batch[2]))
	logging.info('batch loss = %f'%(batch_loss))
	logging.debug('test: time elapsed: %.3fs.'%(time.time()-start_time))
	for j in range(BATCH_SIZE):
		imge = cur_batch[0][j]
		cv2.rectangle(imge,(int(round(227*(fc4[j][0])/10)),int(round(227*fc4[j][1]/10))),(int(round(227*(fc4[j][2])/10)),int(round(227*fc4[j][3]/10))),(127,127,255),5)
		var = cur_batch[3][j].split("/")
		cv2.imwrite(PATH,imge)

except KeyboardInterrupt:
print("get keyboard interrupt")`

Any thing that I am doing wrong. If I let it recalibrate after every alternate frame then also it fails, still predicts correct in the frame where gt bbox was provided but fails miserably in the next frame.

I download the model, and run the load_and_test.py .There isn't any information printed then the program end, which makes me even doubt that I had something wrong. So,how could I see the progress of test/training ?For example , there is a window showing the video with the predicted bounding box on it.
Thanks a lot!

After training on the ALOV300+++ dataset, the network outputs almost the same bounding box prediction (mean value) for whatever input. What could be going wrong?

I found that the pictures in target and searching folder are put in the wrong order, you should switch them

Hello, could you please tell me how to cut each picture of ALOV300++ video?

Hello,In your load_and_test.py there is a " f = open('pkl/right_order_goturn_weights.pkl', 'rb')",but there isn't that file ....I want some help

Hello. How can i test it in a new video?

Hi,
I wanted to draw bounding boxes into with the outputs of the network.
But I couldn't understand why the output values have negative values. What's more, the test example, why the bounding box value is between 0 and 1 and also has a negative value ?

test_example/target/000000.jpg,test_example/searching/000000.jpg,0.5259842519685036,0.02973154362416112,1.055905511811023,0.5781879194630871

Could you tell me the format of theses bounding boxes values and how to use the to draw a box in a image ?
Thanks

please give a new adress

Could you pelase explain why you permutate the order to be n * c * w * h?
in this line?
concat = tf.transpose(concat, perm=[0, 3, 1, 2])

Why does it need to be the same order as caffe?

thanks,
Omer

Would it be possible to know what is the training data for the pretrained model (that you provided)?
Since it is important to avoid using training data for testing the algorithm with the pretrained model,
it would be nice if you comment about what training data you used for the pretrained model.
Thanks.

Hi, I notice the input format has room for only one bounding box. Is it possible/wise to feed multiple bounding boxes for a single training image? Like with multiple lines?

Thanks a lot!
Matt

Hello,

I appreciate this open code for GOTURN.
By the way, merged_summary and train_writer have not defined in train.py

One more thing. Line 135 in train.py
"if (i - start > 1000):"
The variable 'i' is not clearly defined.

Please check them out.
Thank you.

Hi,
For training the net, we need the init the conv layers weights and biases from the caffeNet model as they do in the paper.
In the readme under 'Train' you didn't mention this.
Do you have some checkpoint (tensorflow model) for (only) the conv layer init?

Thanks,
Asaf

point1 = (int(fc4[0][0] * TWidth / 10), int(fc4[0][1] * THeight / 10))
在这里除10是什么意思？
非常感谢

Hi~ Thank you for your tensorflow implementation of GOTURN.
I have tried many times to run the caffe version of GOTURN, but I suffer from terrible environment compatibility problems due to my Windows OS. Could you offer the results on OTB benchmark? Or could this version be run on OTB benchmark for Win+Python3 environment?
Thank you so much!

I read "Learning to Track at 100 FPS with Deep Regression Networks", there is same size of target image and search image, but test image of "test_example" are not. Did I get it wrong?

Please tell how to make this train_file.txt for ALOV and imagenet?? or it is already available?

How to deal with error accumulation that happens from frame to frame? i.e. suppose we have some ideal bbox but in real life we predict bbox+eps if we make this eps mistake from frame to frame this can lead to bbox expand to whole image or shrink to single pixel.

it no every imgae ground-truth bounding box

Thank you for your code

Example of one line: train/target/000024.jpg, train/searching/000024.jpg, 0.29269230769230764, 0.22233115468409587, 0.7991794871794871, 0.7608061002178649

what's the meaning of the 0.29269230769230764, 0.22233115468409587, 0.7991794871794871, 0.7608061002178649?. And how did you calculate this training label form the bounding boxes of the current and previous frames?

I see the test data you given, how could I generate the training data ? For example, I have the video and corresponding bounding box location, I know the target object, how could I set the searching region ? Random setting the bounding box ??? It will be cool if you could shown how to pre-processing the dataset ! Thanks !

Hi there. Can you tell me what is the use of the target images ? Are those for training stages?
多谢老铁

can not download the pretrained model. Permission is required. could you please offer a new address?

Thanks !

Hi, Tang Yuhao~You once said that you suppose all the first frames' bounding box
are<0.25,0.25,0.75,0.75> . Since it's a huge work to crop the pics and make the bbox like that. Could it be possible to change the bbox of target pic??