hyseob / pymdnet Goto Github PK
View Code? Open in Web Editor NEWMDNet PyTorch implementation
License: Other
MDNet PyTorch implementation
License: Other
When I click the link to download the VGG-M model nothing happens?
Dear HyeonseobNam:
I am learning your excellent job.Do you mind show me:How do you pretrain the network? What is your train set,validation set and test set ?
Best wishes,
Maiji
Hi,
Has anybody find this problem? I went into an error:
File "run_tracker.py", line 164, in run_mdnet
feat_dim = pos_feats.size(-1)
AttributeError: 'tuple' object has no attribute 'size'
In the code ,the post_feats was defined in line 162:
pos_feats = (model, image, pos_examples), which is a tuple w/o size attribute.
Do you mean pos_feats[-1].size() ? Because pos_feats[-1] is a ndarray.
But in line 63, you called pos_feats.size(0) again. pos_feats(0)'s type is 'model.MDNet' which doesn't have a size attribute.
Hi all,
Before we are doing the tracking part, we must provide the bbox of groundtruth for each image. It seems like to generate the new bbox sample via original groundtruth bbox.
However, in the real case, when we are doing the tracking, we don't have the groundtruth information.
Is there any possible to track object without providing groundtruth bbox? To specific, how to generate samples and crop image without using groundtruth bbox values?
Thank you!
Hello, I ran your code and found that the number of frames in this run is far from the FPS in the paper.
why it need pretrain step?
how you get the mdnet_vot-otb.pth?
why in some case, the mdnet_vot-otb_new.pth is better than the mdnet_vot-otb.pth?
Hello.could you please tell me how to find the dataset include cfg.json
Traceback (most recent call last):
File "run_tracker.py", line 329, in
result, result_bb, fps = run_mdnet(img_list, init_bbox, gt=gt, savefig_dir=savefig_dir, display=display)
File "run_tracker.py", line 146, in run_mdnet
bbreg_feats = forward_samples(model, image, bbreg_examples)
File "run_tracker.py", line 34, in forward_samples
feat = model(regions, out_layer=out_layer)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "../modules/model.py", line 110, in forward
x = module(x)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/container.py", line 64, in forward
input = module(input)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/conv.py", line 237, in forward
self.padding, self.dilation, self.groups)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/functional.py", line 40, in conv2d
return f(input, weight, bias)
RuntimeError: CUDNN_STATUS_ARCH_MISMATCH
GPU: nvidia NVS 315
OS: Ubuntu 16.04
I want to know how long do you take training on ImageVid? thanks
elif args.json != '':
# load config from a json file
print("run json")
param = json.load(open(args.json,'r'))
seq_name = param['seq_name']
img_list = param['img_list']
init_bbox = param['init_bbox']
savefig_dir = param['savefig_dir']
result_path = param['result_path']
gt = None
how to run it?
My name is Nina, the director of communication of the open source project Tegu. Tegu is a GUI-based software that allows developers to implement computer vision algorithms in their work with minimal amount of coding.
We have recently noticed your repo py-MDNet. What you wrote is very valuable and we appreciate your contribution to the open source community.
We think py-MDNet will provide greater benefits to the community if we are able to incorporate it as part of the growing Tegu project. Per your LICENSE file, we would therefore like to request your permission for redistribution and potential commercial use of the software.
Tegu will remain open source in the future and your work will be cited as directed in your LICENSE file.
Looking forward to hearing from you.
I use 'python run_tracker.py -s DragonBaby -d -f',but it is wrong.
This is the information printed by the terminal:
Traceback (most recent call last):
File "run_tracker.py", line 330, in
result, result_bb, fps = run_mdnet(img_list, init_bbox, gt=gt, savefig_dir=savefig_dir, display=display)
File "run_tracker.py", line 147, in run_mdnet
bbreg_feats = forward_samples(model, image, bbreg_examples)
File "run_tracker.py", line 35, in forward_samples
feat = model(regions, out_layer=out_layer)
File "/home/casic/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/usr/projects/ObjectTracking/single/py-MDNet-master/model.py", line 110, in forward
x = module(x)
File "/home/casic/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/cs/.local/lib/python2.7/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/home/cs/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/usr/projects/ObjectTracking/single/py-MDNet-master/model.py", line 40, in forward
torch.cat((pad,pad,pad,pad,x_sq),2)),1)
RuntimeError: CUDA out of memory. Tried to allocate 1.24 GiB (GPU 0; 3.95 GiB total capacity; 1.77 GiB already allocated; 1.15 GiB free; 242.18 MiB cached)
Hello, how can I achieve the accuracy you show? Is there any special skills? I can't achieve the accuracy you show here when running your code on my computer.
hi,what do you mean the "ImageNet-VID dataset" ? Because I can find the 2015 :http://bvisionweb1.cs.unc.edu/ilsvrc2015/download-videos-3j16.php#vid and the 2017.
Can any guy give me a link of the ImageNet-VID dataset ?thanks a lot.
I have noticed the performance figures in python repo are worse than these in matlab repo, Why?
Thanks for your excellent work and your answering. I am quite interested that which part of modification mostly accelerate the FPS from about 2 to 5.
Looking forward to your answer.
Could you tell me how to do evaluation on OTB and VOT based on python?
Or could you publish the evaluation file?
I am learning tracking based on your architecture.
Thank you very much!
In 'run_tracker.py' , if the tracker lose the target in a frame(all sample scores are negative), you still update the target location. Shouldn't the target be locked at the previous location if all sample scores are negative?
if not success:
target_bbox = result[i-1]
bbreg_bbox = result_bb[i-1]
Thank you for you answer?
about the 44 frame processed ... run out of memory....running
python run_tracker.py -s DragonBaby -d
Any idea why this is happening?
return torch.cat(inputs, self.dim)
RuntimeError: cuda runtime error (2) : out of memory at /b/wheel/pytorch-src/torch/lib/THC/generic/THCStorage.cu:66
Hi Hyeonseob,
I found py-MDNet very interesting and I would like to use it a lot.
So, I tried to train a new model based on a new dataset.
However, when I was running the program for the 4th cycle,
it returns "RuntimeError: the given numpy array has zero-sized dimensions. Zero-sized dimensions are not supported in PyTorch"
.
It is from data_prov.py on line 63
neg_regions = torch.from_numpy(neg_regions).float()
.
called by train_mdnet.py on line 64
pos_regions, neg_regions = dataset[k].next()
Thank you.
Best,
Peratham
The file pretrain/data/imagenet_vid.pkl is missing from the repository. Any chance you can push it up? Thank you.
hi,do you draw the otb100 success using the results "Raw results of MDNet pretrained on Imagenet-VID: Google drive link" Or you use the other results.Because when I draw the line,I only get at 0.6761(based on imagenet) and 0.6718(based on vot).THey are different from what you show 0.680(vid) and 0.676(vid).I used the otb pyhton to draw the success line https://github.com/jwlim/tracker_benchmark.thanks a lot
Hello,
when I used your model to carry out my work,
I found that it is difficult to verify the validity of the ingredients during the ablation experiment.
Due to the randomness of the model
how did you solve it?
Is it the average of multiple training tests, or is it a fixed random seed?
If it is the former, can you briefly describe how to draw the accuracy curve under the condition that multiple results are averaged.
I do n’t know much about this, just contact with target tracking,
Looking forward to your reply
best~
when I run python run_'tracker.py -s DragonBaby', it happened in line 36 of bbreg.py, 'self.model.fit(X, Y)' ,
python-->3.6.2
torch-->1.0.0
sklearn-->0.19.0
opencv-->3.1.0
GPU-->tesla p40 and have 22G empty
In line168 of run_tracker.py, but I think it should be:
neg_examples = neg_generator(target_bbox, opts['n_neg_update'], opts['overlap_neg_update'])
rather than:
neg_examples = neg_generator(target_bbox, opts['n_neg_update'], opts['overlap_neg_init'])
Traceback (most recent call last):
File "/home/kai_xu/PycharmProjects/py-MDNet/pretrain/train_mdnet.py", line 100, in
train_mdnet()
File "/home/kai_xu/PycharmProjects/py-MDNet/pretrain/train_mdnet.py", line 71, in train_mdnet
pos_score = model(pos_regions, k)
File "/home/kai_xu/anaconda3/envs/pytorch/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "../modules/model.py", line 111, in forward
x = module(x)
File "/home/kai_xu/anaconda3/envs/pytorch/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "/home/kai_xu/anaconda3/envs/pytorch/lib/python2.7/site-packages/torch/nn/modules/container.py", line 64, in forward
input = module(input)
File "/home/kai_xu/anaconda3/envs/pytorch/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "/home/kai_xu/anaconda3/envs/pytorch/lib/python2.7/site-packages/torch/nn/modules/conv.py", line 237, in forward
self.padding, self.dilation, self.groups)
File "/home/kai_xu/anaconda3/envs/pytorch/lib/python2.7/site-packages/torch/nn/functional.py", line 40, in conv2d
return f(input, weight, bias)
RuntimeError: CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input.
could you give the link of the VOT database?
I cannot download it
thanks
wasson@wasson-MS-7971:~/yujia/py-MDNet-master/tracking$ python run_tracker.py -d DragonBaby
usage: run_tracker.py [-h] [-s SEQ] [-j JSON] [-f] [-d]
run_tracker.py: error: unrecognized arguments: DragonBaby
If I have a sequence with more than one target, what should the initial bounding box be set to in gen_config?
Is it not for multiple object tracking? when I run the code it is only tracking single person.
I have two questions:
Have you tried working in a fully convolution mode? I am thinking of changing it to a fully convolution,
So it may run faster. Do you think there are any reasons for that no to work well?
I saw that in your PyTorch code, what's taking most of the time during inference is the cropping, and not the feed froward part.
Have you tried addressing this issue? How do you think this can be resolved?
Cheers,
Avi
Dear @HyeonseobNam,
Thank you for your fantastic work.
I want to change the Base CNN model of your code via another one (e.g., ResNet18). Do you have any idea that how can I do that?
Hi,
As far as I understand, the code requires the ground truth for the first bounding box. While one can use some method to generate 1st proposals, the network has learned what a, say, ball looks like, how can I utilize that to generate some initial proposals? Moreover, there are several classes that the network can distinguish, a way to tell the network what to look for can be interesting. Let's discuss more.
Best regards,
I got some issue with inference step. I think it's pytorch + cuda issue.
what's the problem?
python run_tracker.py -s DragonBaby -f
File "run_tracker.py", line 329, in
result, result_bb, fps = run_mdnet(img_list, init_bbox, gt=gt, savefig_dir=savefig_dir, display=display)
File "run_tracker.py", line 131, in run_mdnet
model = model.cuda()
File "/home/dhlee/anaconda2/envs/tensorflow/lib/python2.7/site-packages/torch/nn/modules/module.py", line 147, in cuda
return self._apply(lambda t: t.cuda(device_id))
File "/home/dhlee/anaconda2/envs/tensorflow/lib/python2.7/site-packages/torch/nn/modules/module.py", line 118, in _apply
module._apply(fn)
File "/home/dhlee/anaconda2/envs/tensorflow/lib/python2.7/site-packages/torch/nn/modules/module.py", line 118, in _apply
module._apply(fn)
File "/home/dhlee/anaconda2/envs/tensorflow/lib/python2.7/site-packages/torch/nn/modules/module.py", line 118, in _apply
module._apply(fn)
File "/home/dhlee/anaconda2/envs/tensorflow/lib/python2.7/site-packages/torch/nn/modules/module.py", line 124, in _apply
param.data = fn(param.data)
File "/home/dhlee/anaconda2/envs/tensorflow/lib/python2.7/site-packages/torch/nn/modules/module.py", line 147, in
return self._apply(lambda t: t.cuda(device_id))
File "/home/dhlee/anaconda2/envs/tensorflow/lib/python2.7/site-packages/torch/_utils.py", line 65, in cuda
return new_type(self.size()).copy(self, async)
File "/home/dhlee/anaconda2/envs/tensorflow/lib/python2.7/site-packages/torch/cuda/init.py", line 275, in new
return super(_CudaBase, cls).new(cls, *args, **kwargs)
KeyboardInterrupt
@HyeonseobNam , hello, HyeonseobNam, thanks for your python implementation of the MDNet, this is really a wonderful wok. But i still have a question: do you evaluate the model which is provided by you on the OTB-100 dataset? I test it and find that the precision is far lower than the matlab version. I appreciate that if you can test it on the OTB-100 dataset. Thanks very much !
Are there any image size requirements during the training phase?
Hello Hyeonseob,
self.layers[i][0].weight.data = torch.from_numpy(np.transpose(weight, (3,2,0,1)))
Could you please explain to me, why did you transpose the shape of weights in the line
of code above, is there a difference between the weights of the initial model and the model
which you implemented in Pytorch ?
Thank you,
I'm f'tuning MDNet to my data. Only the first frame in the video is labelled with a bbox. After 8-10 frames the target score drop below 0. I played with learning rates/multipliers/trainable layers, but no success so far. Anything I could do or I need to label a larger number of frames in the dataset?
Hello, I encountered a problem when running "python tracking/run_tracker.py -s DragonBaby", the error below shows when I run the command:
Traceback (most recent call last):
File "tracking/run_tracker.py", line 325, in
result, result_bb, fps = run_mdnet(img_list, init_bbox, gt=gt, savefig_dir=savefig_dir, display=display)
File "tracking/run_tracker.py", line 150, in run_mdnet
train(model, criterion, init_optimizer, pos_feats, neg_feats, opts['maxiter_init'])
File "tracking/run_tracker.py", line 83, in train
score = model(batch_neg_feats[start:end], in_layer=in_layer)
File "C:\MIMIA\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 550, in call
result = self.forward(*input, **kwargs)
File ".\modules\model.py", line 120, in forward
x = module(x)
File "C:\MIMIA\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "C:\MIMIA\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\container.py", line 100, in forward
input = module(input)
File "C:\MIMIA\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "C:\MIMIA\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\linear.py", line 87, in forward
return F.linear(input, self.weight, self.bias)
File "C:\MIMIA\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\functional.py", line 1610, in linear
ret = torch.addmm(bias, input, weight.t())
RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)
Please help me. Thanks!!!!
In the groundtruth the frames without object is empty lines. When the objects joins the video a bit late the program starts to draw some rectangles. So, it causes the IOU value to be incorrect. What should be the GT lines when the object is missing?
why it need train the fully connected layers for 10 iterations in online update?
Hi,
In the run_mdnet
function in run_tracker.py
, why do you train the box regressor and the model? Isn't the function is for interpretation using a pre-trained model?
Best regards,
My PC spec. as follows.
Hardware: Xeon-E3 / 16G / GTX 650 /
Software: Ubuntu 16.04 / Anaconda 5.2(Python 2.7) / CUDA 9.0 / PyTorch 0.3.0
After I modified the size of opts['batch_test'] = 32
I can run following command.
$ python run_tracker.py -s DragonBaby
But when I want go further to do this:
$ python run_tracking.py -s DragonBaby -d -f
I got this information.
../modules/model.py:144: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
pos_loss = -F.log_softmax(pos_score)[:,1]
../modules/model.py:145: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
neg_loss = -F.log_softmax(neg_score)[:,0]
Traceback (most recent call last):
File "run_tracker.py", line 329, in
result, result_bb, fps = run_mdnet(img_list, init_bbox, gt=gt, savefig_dir=savefig_dir, display=display)
File "run_tracker.py", line 190, in run_mdnet
im = ax.imshow(image, aspect='normal')
File "/home/thwang/anaconda2/lib/python2.7/site-packages/matplotlib/init.py", line 1855, in inner
return func(ax, *args, **kwargs)
File "/home/thwang/anaconda2/lib/python2.7/site-packages/matplotlib/axes/_axes.py", line 5482, in imshow
self.set_aspect(aspect)
File "/home/thwang/anaconda2/lib/python2.7/site-packages/matplotlib/axes/_base.py", line 1345, in set_aspect
aspect = float(aspect) # raise ValueError if necessary
ValueError: could not convert string to float: normal
I can't make this work properly.
Could you help me solve this?
Dear @HyeonseobNam,
How one can change your code to work on CPU mode (not using any GPU)?
Hello, if I want to run my own sequences, do I need to pretrain using my image datasets or I can just run tracking? Thank you!
where is the parameter lr_decay? Ican't find it。
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.