First, thanks for your share of repo. When I clicked the download link of the trained model your gave in the Github, it did not work. And I tried it on other browsers, it also failed. If it's convenient, would you please sent it to me by email, my email is [email protected]. Thank you very much！

Hi, I saw your code. Is the prediction module of your code is slightly different from the one in your paper (Fig. 7)? In your code, the Top-left Corner Pooling is followed by 3x3 Conv-BN-ReLU.

Can you elaborate the speed comparison between YOLO, Retinanet and Faster RCNN

I encountered the SIGSEGV when running to line191-192 of kp.py, is it caused by pooling layer?

=====================================
interrupted by signal 11: SIGSEGV

I just replace the images with my own images of the file testdev2017，then run the python test.py CornerNet --testiter 500000 --split testing，and the error come up as follow：
cfg_file: ./config/CornerNet.json
loading all datasets...
split: testdev
loading from cache file: ./cache/coco_testdev2017.pkl
No cache file found...
loading annotations into memory...
Done (t=0.05s)
creating index...
index created!
20288it [00:08, 2410.16it/s]
loading annotations into memory...
Done (t=0.05s)
creating index...
index created!
system config...
{'batch_size': 49,
'cache_dir': './cache',
'chunk_sizes': [4, 5, 5, 5, 5, 5, 5, 5, 5, 5],
'config_dir': './config',
'data_dir': './data',
'data_rng': <mtrand.RandomState object at 0x7f847ac04438>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7f847ac04480>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': './results',
'sampling_function': 'kp_detection',
'snapshot': 5000,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
loading parameters at iteration: 500000
building neural network...
module_file: models.CornerNet
total parameters: 201035212
loading parameters...
loading model from ./cache/nnet/CornerNet/CornerNet_500000.pkl
locating kps: 0%| | 1/20288 [00:00<2:44:31, 2.06it/s]
Traceback (most recent call last):
File "test.py", line 99, in
test(testing_db, args.split, args.testiter, args.debug, args.suffix)
File "test.py", line 61, in test
testing(db, nnet, result_dir, debug=debug)
File "/home/dc2-user/CornerNet-master/test/coco.py", line 204, in testing
return globals()[system_configs.sampling_function](db, nnet, result_dir, debug=debug)
File "/home/dc2-user/CornerNet-master/test/coco.py", line 77, in kp_detection
height, width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
What should I do？Thanks。

thank you for releasing this repo. could you guide me to the steps to do a demo of CornerNet on a video (an mp4 file) ?

thanks,

Hi writer!I have some question about your code .
the nstack seted in CornerNet.py is 2,it means that，the outputs of your networks are two tl heatmaps and two br heatmaps (tags and regs are same).why we use four heatmaps to caculate the focal loss?
thanks for your answer!

If I want to train the model with my own images，how can I get the json file of annotations，such as instances_minival2014.json，instances_testval2014.json and instances_trainval2014.json？

In kp.py
# for tl_heat, br_heat in zip(self.tl_heats, self.br_heats):
# tl_heat[-1].bias.data.fill_(-2.19)
# br_heat[-1].bias.data.fill_(-2.19)

I met segmentation fault when calling corner pooling.
I have updated my gcc version to 4.9.4 and I am using Python 3.6.5.
After rebuilding the cpools I still came across the problem.
Can anyone give me some help?

Does this project support multi node training?Thanks.

hello, i train your code according to you provide steps, but occurs follow problem:
loading all datasets...
using 4 threads
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=14.30s)
creating index...
index created!
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=13.85s)
creating index...
index created!
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=13.05s)
creating index...
index created!
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=18.46s)
creating index...
index created!
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_minival2014.pkl
loading annotations into memory...
Done (t=0.41s)
creating index...
index created!
system config...
{'batch_size': 49,
'cache_dir': '/home/gosuncn/detection/CornerNet-master/cache',
'chunk_sizes': [4, 5, 5, 5, 5, 5, 5, 5, 5, 5],
'config_dir': '/home/gosuncn/detection/CornerNet-master/config',
'data_dir': '/home/gosuncn/detection/CornerNet-master/data/',
'data_rng': <mtrand.RandomState object at 0x7f2c28ed32d0>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7f2c28ed3360>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': '/home/gosuncn/detection/CornerNet-master/data/coco/results',
'sampling_function': 'kp_detection',
'snapshot': 5000,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
len of db: 118287
start prefetching data...
start prefetching data...
shuffling indices...
shuffling indices...
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-2:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-4:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
building model...
module_file: models.CornerNet
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-3:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
total parameters: 201035212
setting learning rate to: 0.00025
training start...
0%| | 0/500000 [00:00<?, ?it/s]
the path i also changed as follow:
self._configs["data_dir"] = "/home/gosuncn/detection/CornerNet-master/data"
self._configs["cache_dir"] = "/home/detection/CornerNet-master/cache"
self._configs["config_dir"] = "/home/detection/CornerNet-master/config"
self._configs["result_dir"] = "/home/detection/CornerNet-master/data/coco/results"
so i don't know why it cannot read images according to my path

hello!
I can't download the annotatios.zip

I want to know how fast CornerNet is，Thank you！

@heilaw
I am sorry to bother you,when I run git clone [email protected]:cocodataset/cocoapi.git coco
I got the error : Warning:Permanently added the RSA host key for IP address''to the list of known hosts.
Permission denied(publickey).
fatal:could not read from romote repository.
please make sure you have the correct access rights
and tht repository exists.
Is there any other website to get cooapi clone?

When I train the code , i have shape error as follow:

[sky@gpu01 CornerNet]$ python train.py CornerNet
loading all datasets...
using 4 threads
loading from cache file: ./cache/coco_trainval2014.pkl
No cache file found...
loading annotations into memory...
Done (t=11.85s)
creating index...
index created!
118287it [00:40, 2923.89it/s]
loading annotations into memory...
Done (t=11.86s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=10.91s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=10.25s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=11.62s)
creating index...
index created!
loading from cache file: ./cache/coco_minival2014.pkl
No cache file found...
loading annotations into memory...
Done (t=0.32s)
creating index...
index created!
5000it [00:01, 2962.00it/s]
loading annotations into memory...
Done (t=0.31s)
creating index...
index created!
system config...
{'batch_size': 49,
'cache_dir': './cache',
'chunk_sizes': [4, 5, 5, 5, 5, 5, 5, 5, 5, 5],
'config_dir': './config',
'data_dir': './data',
'data_rng': <mtrand.RandomState object at 0x7f2085918c60>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7f2085918ca8>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': './results',
'sampling_function': 'kp_detection',
'snapshot': 5000,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
len of db: 118287
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-1:
Traceback (most recent call last):
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-2:
Traceback (most recent call last):
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-3:
Traceback (most recent call last):
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-4:
Traceback (most recent call last):
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
building model...
module_file: models.CornerNet
start prefetching data...
shuffling indices...
total parameters: 201035212
setting learning rate to: 0.00025
training start...
0%| | 0/500000 [00:00<?, ?it/s]^CTraceback (most recent call last):

How can i solve it?Thanks a lot for your help !

Hi,

Great work!

You said that you use one-stage method for object detection. I want to know more details on the performance time with corner net.

Thank you!

How much does the loss value fall? and the network is converge.

Hi,

Thank you for your wonderful work. But after I simply change the path to my own dataset (all pictures with the size of 1280x720), I meet the segmentation fault. :( Can you please help me locate the bug?

Thanks for your excellent contribution. In CornerNet, 2 stacks of Hourglass is employed for keypoint detection. I'm intereseted in how better or worse if CornerNet is configured as 1 stack or 3 stacks? Or, is '2 stacks Hourglass' configuration the minimum requirement for Key Point detection？Thanks.

I'm sorry for bothering you !
In the paper, you said that CornerNet infer an image with average time 244ms on a TitanX GPU, that means the speed is around 5fps.
That is much slower than other single stage detectors. Why is it so slow even without anchors and fpn?
Does the time cost most during corner_pooling?
@ywchao @heilaw @anewell @jiadeng
Thanks a lot for your reply!

When i train my own 2047 training sets,its loss is around 4.The problem is as follows:

training loss at iteration 54405: 3.7331526279449463
training loss at iteration 54410: 3.6703977584838867
training loss at iteration 54415: 3.792213201522827
training loss at iteration 54420: 3.184262275695801
training loss at iteration 54425: 2.5833559036254883
training loss at iteration 54430: 3.983672618865967
training loss at iteration 54435: 2.3275046348571777
training loss at iteration 54440: 5.39171838760376
training loss at iteration 54445: 4.39638090133667
training loss at iteration 54450: 4.163412094116211
training loss at iteration 54455: 3.498041868209839
training loss at iteration 54460: 4.481285572052002
training loss at iteration 54465: 4.839260101318359
training loss at iteration 54470: 3.994492769241333
training loss at iteration 54475: 3.5584001541137695
training loss at iteration 54480: 4.685251712799072
training loss at iteration 54485: 4.015157222747803
training loss at iteration 54490: 4.063553810119629
training loss at iteration 54495: 3.18803334236145
training loss at iteration 54500: 4.06464147567749
validation loss at iteration 54500: 3.384662628173828
training loss at iteration 54505: 4.854454517364502
training loss at iteration 54510: 4.531534671783447
training loss at iteration 54515: 3.9393210411071777
training loss at iteration 54520: 3.8535616397857666
training loss at iteration 54525: 4.103610992431641
training loss at iteration 54530: 5.204475402832031
training loss at iteration 54535: 4.274611949920654
training loss at iteration 54540: 3.4196114540100098
training loss at iteration 54545: 3.8542287349700928
training loss at iteration 54550: 2.900420904159546
training loss at iteration 54555: 4.218133926391602
training loss at iteration 54560: 3.702637195587158
training loss at iteration 54565: 4.377786159515381
training loss at iteration 54570: 4.17509126663208
training loss at iteration 54575: 4.399285316467285
training loss at iteration 54580: 4.801181316375732
training loss at iteration 54585: 3.8167858123779297
training loss at iteration 54590: 3.7509422302246094
training loss at iteration 54595: 3.3366339206695557
training loss at iteration 54600: 4.960026264190674
validation loss at iteration 54600: 4.456686496734619
training loss at iteration 54605: 7.162835597991943
training loss at iteration 54610: 4.2326860427856445
training loss at iteration 54615: 5.217791557312012
training loss at iteration 54620: 5.991508483886719
training loss at iteration 54625: 3.405985116958618
training loss at iteration 54630: 8.021293640136719
training loss at iteration 54635: 4.542353630065918
training loss at iteration 54640: 5.705121040344238
47%|██████████████▍ | 39641/85000 [15:58:31<18:16:47, 1.45s/it]

Thank you! @heilaw

Training with 8 1080 Ti GPUs, which erred out after between 100 to 700 iterations:

Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "train.py", line 50, in pin_memory
    data = data_queue.get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
    res = self._recv()
  File "/home/user/tf/local/lib/python2.7/site-packages/torch/multiprocessing/queue.py", line 22, in recv
    return pickle.loads(buf)
  File "/usr/lib/python2.7/pickle.py", line 1388, in loads
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 864, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1139, in load_reduce
    value = func(*args)
  File "/home/user/tf/local/lib/python2.7/site-packages/torch/multiprocessing/reductions.py", line 68, in rebuild_storage_fd
    fd = multiprocessing.reduction.rebuild_handle(df)
  File "/usr/lib/python2.7/multiprocessing/reduction.py", line 155, in rebuild_handle
    conn = Client(address, authkey=current_process().authkey)
  File "/usr/lib/python2.7/multiprocessing/connection.py", line 169, in Client
    c = SocketClient(address)
  File "/usr/lib/python2.7/multiprocessing/connection.py", line 308, in SocketClient
    s.connect(address)
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 2] No such file or directory

Any suggestion? Thx.

When i train my own data , i meet this problem:

training loss at iteration 270: 8.091086387634277
training loss at iteration 275: 8.255900382995605
training loss at iteration 280: 8.269482612609863
training loss at iteration 285: 8.271137237548828
3%|█ | 286/10000 [07:02<3:59:10, 1.48s/it]Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/chase/application/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/chase/application/CornerNet/sample/coco.py", line 157, in kp_detection
tl_regrs[b_ind, tag_ind, :] = [fxtl - xtl, fytl - ytl]
IndexError: index 128 is out of bounds for axis 1 with size 128
Process Process-2:
Traceback (most recent call last):
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/chase/application/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/chase/application/CornerNet/sample/coco.py", line 157, in kp_detection
tl_regrs[b_ind, tag_ind, :] = [fxtl - xtl, fytl - ytl]
IndexError: index 128 is out of bounds for axis 1 with size 128
3%|█ | 287/10000 [07:03<3:59:08, 1.48s/it]Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 50, in pin_memory
data = data_queue.get()
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 70, in rebuild_storage_fd
fd = df.detach()
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/resource_sharer.py", line 57, in detach
with _resource_sharer.get_connection(self._id) as conn:
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/resource_sharer.py", line 87, in get_connection
c = Client(address, authkey=process.current_process().authkey)
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory

Thank @heilaw

Hi everyone. What happened?
python test.py CornerNet --testiter 500000
Traceback (most recent call last):
File "test.py", line 14, in
from nnet.py_factory import NetworkFactory
File "/home/jinxuany/CornerNet/nnet/py_factory.py", line 7, in
from models.py_utils.data_parallel import DataParallel
File "/home/jinxuany/CornerNet/models/py_utils/init.py", line 6, in
from ._cpools import TopPool, BottomPool, LeftPool, RightPool
File "/home/jinxuany/CornerNet/models/py_utils/_cpools/init.py", line 6, in
import top_pool, bottom_pool, left_pool, right_pool
ImportError: /home/jinxuany/.local/lib/python3.6/site-packages/cpools-0.0.0-py3.6-linux-x86_64.egg/top_pool.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN2at6detail20DynamicCUDAInterface10set_deviceE

as the title,if you have plan to write a demo.py for this repo?

I am trying to train the CornerNet on my machine but it produces the following error.

loading all datasets...
using 4 threads
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=8.28s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=8.38s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=9.22s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=8.18s)
creating index...
index created!
loading from cache file: ./cache/coco_minival2014.pkl
loading annotations into memory...
Done (t=0.23s)
creating index...
index created!
system config...
{'batch_size': 1,
'cache_dir': './cache',
'chunk_sizes': [1],
'config_dir': './config',
'data_dir': './data',
'data_rng': <mtrand.RandomState object at 0x7fd5039ad4c8>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7fd5039ad510>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': './results',
'sampling_function': 'kp_detection',
'snapshot': 100,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
len of db: 118287
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
building model...
module_file: models.CornerNet
shuffling indices...
total parameters: 201035212
setting learning rate to: 0.00025
training start...
0%| | 0/500000 [00:00<?, ?it/s]
Traceback (most recent call last):
File "train.py", line 195, in
train(training_dbs, validation_db, args.start_iter)
File "train.py", line 137, in train
training_loss = nnet.train(**training)
File "/home/ekrem/PycharmProjects/CornerNet/nnet/py_factory.py", line 81, in train
loss = self.network(xs, ys)
File "/home/ekrem/anaconda3/envs/CornerNet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/ekrem/PycharmProjects/CornerNet/models/py_utils/data_parallel.py", line 68, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/ekrem/anaconda3/envs/CornerNet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/ekrem/PycharmProjects/CornerNet/nnet/py_factory.py", line 20, in forward
loss = self.loss(preds, ys, **kwargs)
File "/home/ekrem/anaconda3/envs/CornerNet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/ekrem/PycharmProjects/CornerNet/models/py_utils/kp.py", line 297, in forward
pull, push = self.ae_loss(tl_tag, br_tag, gt_mask)
File "/home/ekrem/PycharmProjects/CornerNet/models/py_utils/kp_utils.py", line 197, in _ae_loss
dist = tag_mean.unsqueeze(1) - tag_mean.unsqueeze(2)
RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 2)

I have set batch_size = 1 and chunk_sizes = [1] to get rid of the out of memory error as mentioned in #4

Any suggestions ?
Thanks

I saw the function gaussian_radius but could not figure out how it worked, and the process was different with Formula of Root.

when I set the size (10,10) and iou 0.7, the returned radius is 2.73. However, when the box lies in the inner most side of ground truth box, the iou equals to [(10-22.73)^2]/(1010)=0.2.

So could you please explain the design idea behind it please? it bothers me for the whole night.

Dear all:

when i train my data, the category count is not 80 as coco, which code should i modify?

i find three part of code using "80" :

1. models/CornerNet.py: line 72
   class model(kp):
   def init(self, db):
   n = 5
   dims = [256, 256, 384, 384, 384, 512]
   modules = [2, 2, 2, 2, 2, 4]
   out_dim = 80 <---------------------- here need change to my category count???

2. db/detection.py: line 8
   self._configs["categories"] = 80 <---------------------- here need change to my category count???

3. config/CornerNet.json: line 45
   "categories": 80 <---------------------- here need change to my category count???

@heilaw Does all of these 3 place need to be change? and any other place need change?

ValueError('n (-0.21155001501952542) cannot be negative',) when i run the code 'conda create --name CornerNet --file conda_packagelist.txt
'

There is "locating kps: 0%| | 0/20288 [00:00<?, ?it/s]Segmentation fault" when i run test.py ,how can i solve this problem?

loading all datasets...
using 4 threads
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=11.78s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=8.33s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=7.80s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=11.06s)
creating index...
index created!
loading from cache file: ./cache/coco_minival2014.pkl
loading annotations into memory...
Done (t=0.38s)
creating index...
index created!
system config...
{'batch_size': 2,
'cache_dir': './cache',
'chunk_sizes': [4, 5, 5, 5, 5, 5, 5, 5, 5, 5],
'config_dir': './config',
'data_dir': './data',
'data_rng': <mtrand.RandomState object at 0x7f6494e26a68>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7f6494e26ab0>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': './results',
'sampling_function': 'kp_detection',
'snapshot': 5000,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
len of db: 118287
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
(480, 640, 3)
(480, 640, 3)
(427, 640, 3)
(434, 640, 3)
building model...
module_file: models.CornerNet
start prefetching data...
shuffling indices...
(480, 640, 3)
(375, 500, 3)
(480, 640, 3)
(427, 640, 3)
(333, 500, 3)
(640, 523, 3)
(424, 640, 3)
(480, 640, 3)
(375, 500, 3)
(500, 334, 3)
(480, 640, 3)
(500, 375, 3)
(416, 640, 3)
(500, 375, 3)
(425, 640, 3)
(480, 640, 3)
(375, 500, 3)
(640, 427, 3)
(640, 609, 3)
(383, 640, 3)
(428, 640, 3)
(426, 640, 3)
(427, 640, 3)
(640, 427, 3)
(640, 480, 3)
(555, 640, 3)
(480, 640, 3)
(416, 640, 3)
(375, 500, 3)
(375, 500, 3)
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1524584710464/work/aten/src/THC/THCTensorRandom.cu line=25 error=30 : unknown error

THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1525812548180/work/aten/src/THC/THCGeneral.cpp line=70 error=10 : invalid device ordinal
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/lianbo/anaconda3/envs/Pytorch/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/lianbo/anaconda3/envs/Pytorch/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 55, in pin_memory
data["xs"] = [x.pin_memory() for x in data["xs"]]
File "train.py", line 55, in
data["xs"] = [x.pin_memory() for x in data["xs"]]

I mean, using standard Convolution or Pooling operation, but gets similar performance...

Which pytorch version do you use?when I use your code,here is a problem with me .
import top_pool, bottom_pool, left_pool, right_pool
ImportError: /home/hdc/.local/lib/python3.6/site-packages/cpools-0.0.0-py3.6-linux-x86_64.egg/top_pool.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN2at15UndefinedTensor10_singletonE

Hi, I noticed that in your paper, there is an ablation study on using ground-truth offsets instead of predicted offsets. Is there any experiments comparing the performance of the model when using predicted offsets and not using offsets?
Thank you.

Thx for the project and fresh approach. What version of pytorch does this project support?

Tried pytorch 0.3.1 and 0.4.1, got import errors with "python setup.py install --user".

Hello, how can I test my own pic with your code? Have to create my own anotation? Is there a simple way?

hello:
When I use weights for testing, I get an error.
#------------------------------------------------------------------------------------------------
loading model from ./cache/nnet/CornerNet/CornerNet_500000.pkl
locating kps: 0%| | 0/5000 [00:00<?, ?it/s]
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
#------------------------------------------------------------------------------------------------

and there will be a segment error when debugging。Is this the reason for insufficient memory?My software environment is the same as yours，but my computer has less memory

How to fine tune your dataset on your trained coco model?
Could you provide the json file for fine-tunning params and update the load_pretrained_params() function in file : ./nnet/py_factory.py？

error:
src/right_pool.cpp: In function ‘std::vector<at::Tensor> pool_backward(at::Tensor, at::Tensor)’: src/right_pool.cpp:44:41: error: could not convert ‘torch::CUDA(((at::ScalarType)at::kFloat))’ from ‘at::TypeExtendedInterface’ to ‘at::IntList’ {aka ‘at::ArrayRef<long int>’} auto max_val = at::zeros(torch::CUDA(at::kFloat), {batch, channel, height}); ~~~~~~~~~~~^~~~~~~~~~~~ src/right_pool.cpp:45:41: error: could not convert ‘torch::CUDA(((at::ScalarType)at::kLong))’ from ‘at::TypeExtendedInterface’ to ‘at::IntList’ {aka ‘at::ArrayRef<long int>’} auto max_ind = at::zeros(torch::CUDA(at::kLong), {batch, channel, height}); ~~~~~~~~~~~^~~~~~~~~~~ src/right_pool.cpp:57:44: error: could not convert ‘torch::CUDA(((at::ScalarType)at::kByte))’ from ‘at::TypeExtendedInterface’ to ‘at::IntList’ {aka ‘at::ArrayRef<long int>’} auto gt_mask = at::zeros(torch::CUDA(at::kByte), {batch, channel, height}); ~~~~~~~~~~~^~~~~~~~~~~ src/right_pool.cpp:58:44: error: could not convert ‘torch::CUDA(((at::ScalarType)at::kFloat))’ from ‘at::TypeExtendedInterface’ to ‘at::IntList’ {aka ‘at::ArrayRef<long int>’} auto max_temp = at::zeros(torch::CUDA(at::kFloat), {batch, channel, height}); ~~~~~~~~~~~^~~~~~~~~~~~ error: command 'gcc' failed with exit status 1
solution:
change the invoking order for at::zeros (in bottom_pool.cpp, left_pool.cpp, right_pool.cpp, top_pool.cpp )in CornerNet/models/py_utils/_cpools/src,
For example:
change: auto max_val = at::zeros(torch::CUDA(at::kFloat), {batch, channel, height});
to : auto max_val = at::zeros( {batch, channel, height}, torch::CUDA(at::kFloat));

Your model is very creative. Can it be possible to build a Keras or TensorFlow version. Thank you very much.

when I train CornerNet, RuntimeError: CUDA error (10): invalid device ordinal (check_status at /pytorch/aten/src/ATen/cuda/detail/CUDAHooks.cpp:36) how to solver this problem