princeton-vl / cornernet

CornerNet: Training and Evaluation Code

Update (4/18/2019): please check out CornerNet-Lite, more efficient variants of CornerNet

Code for reproducing the results in the following paper:

CornerNet: Detecting Objects as Paired Keypoints
Hei Law, Jia Deng
European Conference on Computer Vision (ECCV), 2018

Getting Started

Please first install Anaconda and create an Anaconda environment using the provided package list.

conda create --name CornerNet --file conda_packagelist.txt

After you create the environment, activate it.

source activate CornerNet

Our current implementation only supports GPU so you need a GPU and need to have CUDA installed on your machine.

Compiling Corner Pooling Layers

You need to compile the C++ implementation of corner pooling layers.

cd <CornerNet dir>/models/py_utils/_cpools/
python setup.py install --user

Compiling NMS

You also need to compile the NMS code (originally from Faster R-CNN and Soft-NMS).

cd <CornerNet dir>/external
make

Installing MS COCO APIs

You also need to install the MS COCO APIs.

cd <CornerNet dir>/data
git clone [email protected]:cocodataset/cocoapi.git coco
cd <CornerNet dir>/data/coco/PythonAPI
make

Downloading MS COCO Data

• Download the training/validation split we use in our paper from here (originally from Faster R-CNN)
• Unzip the file and place annotations under <CornerNet dir>/data/coco
• Download the images (2014 Train, 2014 Val, 2017 Test) from here
• Create 3 directories, trainval2014, minival2014 and testdev2017, under <CornerNet dir>/data/coco/images/
• Copy the training/validation/testing images to the corresponding directories according to the annotation files

Training and Evaluation

To train and evaluate a network, you will need to create a configuration file, which defines the hyperparameters, and a model file, which defines the network architecture. The configuration file should be in JSON format and placed in config/. Each configuration file should have a corresponding model file in models/. i.e. If there is a <model>.json in config/, there should be a <model>.py in models/. There is only one exception which we will mention later.

To train a model:

python train.py <model>

We provide the configuration file (CornerNet.json) and the model file (CornerNet.py) for CornerNet in this repo.

To train CornerNet:

python train.py CornerNet

We also provide a trained model for CornerNet, which is trained for 500k iterations using 10 Titan X (PASCAL) GPUs. You can download it from here and put it under <CornerNet dir>/cache/nnet/CornerNet (You may need to create this directory by yourself if it does not exist). If you want to train you own CornerNet, please adjust the batch size in CornerNet.json to accommodate the number of GPUs that are available to you.

To use the trained model:

python test.py CornerNet --testiter 500000 --split <split>

If you want to test different hyperparameters in testing and do not want to overwrite the original configuration file, you can do so by creating a configuration file with a suffix (<model>-<suffix>.json). You DO NOT need to create <model>-<suffix>.py in models/.

To use the new configuration file:

python test.py <model> --testiter <iter> --split <split> --suffix <suffix>

We also include a configuration file for multi-scale evaluation, which is CornerNet-multi_scale.json, in this repo.

To use the multi-scale configuration file:

python test.py CornerNet --testiter <iter> --split <split> --suffix multi_scale

cornernet's Issues

Segmentation fault when corner pool is called

I met segmentation fault when calling corner pooling.
I have updated my gcc version to 4.9.4 and I am using Python 3.6.5.
After rebuilding the cpools I still came across the problem.
Can anyone give me some help?

undefined symbol: _ZN2at6detail20DynamicCUDAInterface10set_deviceE

Hi everyone. What happened?
python test.py CornerNet --testiter 500000
Traceback (most recent call last):
File "test.py", line 14, in
from nnet.py_factory import NetworkFactory
File "/home/jinxuany/CornerNet/nnet/py_factory.py", line 7, in
from models.py_utils.data_parallel import DataParallel
File "/home/jinxuany/CornerNet/models/py_utils/init.py", line 6, in
from ._cpools import TopPool, BottomPool, LeftPool, RightPool
File "/home/jinxuany/CornerNet/models/py_utils/_cpools/init.py", line 6, in
import top_pool, bottom_pool, left_pool, right_pool
ImportError: /home/jinxuany/.local/lib/python3.6/site-packages/cpools-0.0.0-py3.6-linux-x86_64.egg/top_pool.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN2at6detail20DynamicCUDAInterface10set_deviceE

The trained model can not be downloaded

First, thanks for your share of repo. When I clicked the download link of the trained model your gave in the Github, it did not work. And I tried it on other browsers, it also failed. If it's convenient, would you please sent it to me by email, my email is [email protected]. Thank you very much！

AttributeError: 'NoneType' object has no attribute 'shape'

hello, i train your code according to you provide steps, but occurs follow problem:
loading all datasets...
using 4 threads
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=14.30s)
creating index...
index created!
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=13.85s)
creating index...
index created!
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=13.05s)
creating index...
index created!
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=18.46s)
creating index...
index created!
loading from cache file: /home/gosuncn/detection/CornerNet-master/cache/coco_minival2014.pkl
loading annotations into memory...
Done (t=0.41s)
creating index...
index created!
system config...
{'batch_size': 49,
'cache_dir': '/home/gosuncn/detection/CornerNet-master/cache',
'chunk_sizes': [4, 5, 5, 5, 5, 5, 5, 5, 5, 5],
'config_dir': '/home/gosuncn/detection/CornerNet-master/config',
'data_dir': '/home/gosuncn/detection/CornerNet-master/data/',
'data_rng': <mtrand.RandomState object at 0x7f2c28ed32d0>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7f2c28ed3360>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': '/home/gosuncn/detection/CornerNet-master/data/coco/results',
'sampling_function': 'kp_detection',
'snapshot': 5000,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
len of db: 118287
start prefetching data...
start prefetching data...
shuffling indices...
shuffling indices...
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-2:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-4:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
building model...
module_file: models.CornerNet
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-3:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/gosuncn/detection/CornerNet-master/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/gosuncn/detection/CornerNet-master/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
total parameters: 201035212
setting learning rate to: 0.00025
training start...
0%| | 0/500000 [00:00<?, ?it/s]
the path i also changed as follow:
self._configs["data_dir"] = "/home/gosuncn/detection/CornerNet-master/data"
self._configs["cache_dir"] = "/home/detection/CornerNet-master/cache"
self._configs["config_dir"] = "/home/detection/CornerNet-master/config"
self._configs["result_dir"] = "/home/detection/CornerNet-master/data/coco/results"
so i don't know why it cannot read images according to my path

interrupted error in kp.py

I encountered the SIGSEGV when running to line191-192 of kp.py, is it caused by pooling layer?

=====================================
interrupted by signal 11: SIGSEGV

create environment error

ValueError('n (-0.21155001501952542) cannot be negative',) when i run the code 'conda create --name CornerNet --file conda_packagelist.txt
'

Training stops with queue error

Training with 8 1080 Ti GPUs, which erred out after between 100 to 700 iterations:

Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "train.py", line 50, in pin_memory
    data = data_queue.get()
  File "/usr/lib/python2.7/multiprocessing/queues.py", line 117, in get
    res = self._recv()
  File "/home/user/tf/local/lib/python2.7/site-packages/torch/multiprocessing/queue.py", line 22, in recv
    return pickle.loads(buf)
  File "/usr/lib/python2.7/pickle.py", line 1388, in loads
    return Unpickler(file).load()
  File "/usr/lib/python2.7/pickle.py", line 864, in load
    dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1139, in load_reduce
    value = func(*args)
  File "/home/user/tf/local/lib/python2.7/site-packages/torch/multiprocessing/reductions.py", line 68, in rebuild_storage_fd
    fd = multiprocessing.reduction.rebuild_handle(df)
  File "/usr/lib/python2.7/multiprocessing/reduction.py", line 155, in rebuild_handle
    conn = Client(address, authkey=current_process().authkey)
  File "/usr/lib/python2.7/multiprocessing/connection.py", line 169, in Client
    c = SocketClient(address)
  File "/usr/lib/python2.7/multiprocessing/connection.py", line 308, in SocketClient
    s.connect(address)
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 2] No such file or directory

Any suggestion? Thx.

RuntimeError: CUDA error (10): invalid device ordinal (check_status at /pytorch/aten/src/ATen/cuda/detail/CUDAHooks.cpp:36)

when I train CornerNet, RuntimeError: CUDA error (10): invalid device ordinal (check_status at /pytorch/aten/src/ATen/cuda/detail/CUDAHooks.cpp:36) how to solver this problem

Pytorch 1.0 change the interface for at::zeros, need to change the order while invoking it.

error:
src/right_pool.cpp: In function ‘std::vector<at::Tensor> pool_backward(at::Tensor, at::Tensor)’: src/right_pool.cpp:44:41: error: could not convert ‘torch::CUDA(((at::ScalarType)at::kFloat))’ from ‘at::TypeExtendedInterface’ to ‘at::IntList’ {aka ‘at::ArrayRef<long int>’} auto max_val = at::zeros(torch::CUDA(at::kFloat), {batch, channel, height}); ~~~~~~~~~~~^~~~~~~~~~~~ src/right_pool.cpp:45:41: error: could not convert ‘torch::CUDA(((at::ScalarType)at::kLong))’ from ‘at::TypeExtendedInterface’ to ‘at::IntList’ {aka ‘at::ArrayRef<long int>’} auto max_ind = at::zeros(torch::CUDA(at::kLong), {batch, channel, height}); ~~~~~~~~~~~^~~~~~~~~~~ src/right_pool.cpp:57:44: error: could not convert ‘torch::CUDA(((at::ScalarType)at::kByte))’ from ‘at::TypeExtendedInterface’ to ‘at::IntList’ {aka ‘at::ArrayRef<long int>’} auto gt_mask = at::zeros(torch::CUDA(at::kByte), {batch, channel, height}); ~~~~~~~~~~~^~~~~~~~~~~ src/right_pool.cpp:58:44: error: could not convert ‘torch::CUDA(((at::ScalarType)at::kFloat))’ from ‘at::TypeExtendedInterface’ to ‘at::IntList’ {aka ‘at::ArrayRef<long int>’} auto max_temp = at::zeros(torch::CUDA(at::kFloat), {batch, channel, height}); ~~~~~~~~~~~^~~~~~~~~~~~ error: command 'gcc' failed with exit status 1
solution:
change the invoking order for at::zeros (in bottom_pool.cpp, left_pool.cpp, right_pool.cpp, top_pool.cpp )in CornerNet/models/py_utils/_cpools/src,
For example:
change: auto max_val = at::zeros(torch::CUDA(at::kFloat), {batch, channel, height});
to : auto max_val = at::zeros( {batch, channel, height}, torch::CUDA(at::kFloat));

any plan to write demo.py ?

as the title,if you have plan to write a demo.py for this repo?

train error

When I train the code , i have shape error as follow:

[sky@gpu01 CornerNet]$ python train.py CornerNet
loading all datasets...
using 4 threads
loading from cache file: ./cache/coco_trainval2014.pkl
No cache file found...
loading annotations into memory...
Done (t=11.85s)
creating index...
index created!
118287it [00:40, 2923.89it/s]
loading annotations into memory...
Done (t=11.86s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=10.91s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=10.25s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=11.62s)
creating index...
index created!
loading from cache file: ./cache/coco_minival2014.pkl
No cache file found...
loading annotations into memory...
Done (t=0.32s)
creating index...
index created!
5000it [00:01, 2962.00it/s]
loading annotations into memory...
Done (t=0.31s)
creating index...
index created!
system config...
{'batch_size': 49,
'cache_dir': './cache',
'chunk_sizes': [4, 5, 5, 5, 5, 5, 5, 5, 5, 5],
'config_dir': './config',
'data_dir': './data',
'data_rng': <mtrand.RandomState object at 0x7f2085918c60>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7f2085918ca8>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': './results',
'sampling_function': 'kp_detection',
'snapshot': 5000,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
len of db: 118287
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-1:
Traceback (most recent call last):
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-2:
Traceback (most recent call last):
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
start prefetching data...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-3:
Traceback (most recent call last):
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
shuffling indices...
Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
Process Process-4:
Traceback (most recent call last):
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/sky/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/sky/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/sky/CornerNet/sample/coco.py", line 96, in kp_detection
image, detections = random_crop(image, detections, rand_scales, input_size, border=border)
File "/home/sky/CornerNet/sample/utils.py", line 57, in random_crop
image_height, image_width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
building model...
module_file: models.CornerNet
start prefetching data...
shuffling indices...
total parameters: 201035212
setting learning rate to: 0.00025
training start...
0%| | 0/500000 [00:00<?, ?it/s]^CTraceback (most recent call last):

How can i solve it?Thanks a lot for your help !

training on my own pictures

If I want to train the model with my own images，how can I get the json file of annotations，such as instances_minival2014.json，instances_testval2014.json and instances_trainval2014.json？

test my own pic

Hello, how can I test my own pic with your code? Have to create my own anotation? Is there a simple way?

Loss chages little

When i train my own 2047 training sets,its loss is around 4.The problem is as follows:

training loss at iteration 54405: 3.7331526279449463
training loss at iteration 54410: 3.6703977584838867
training loss at iteration 54415: 3.792213201522827
training loss at iteration 54420: 3.184262275695801
training loss at iteration 54425: 2.5833559036254883
training loss at iteration 54430: 3.983672618865967
training loss at iteration 54435: 2.3275046348571777
training loss at iteration 54440: 5.39171838760376
training loss at iteration 54445: 4.39638090133667
training loss at iteration 54450: 4.163412094116211
training loss at iteration 54455: 3.498041868209839
training loss at iteration 54460: 4.481285572052002
training loss at iteration 54465: 4.839260101318359
training loss at iteration 54470: 3.994492769241333
training loss at iteration 54475: 3.5584001541137695
training loss at iteration 54480: 4.685251712799072
training loss at iteration 54485: 4.015157222747803
training loss at iteration 54490: 4.063553810119629
training loss at iteration 54495: 3.18803334236145
training loss at iteration 54500: 4.06464147567749
validation loss at iteration 54500: 3.384662628173828
training loss at iteration 54505: 4.854454517364502
training loss at iteration 54510: 4.531534671783447
training loss at iteration 54515: 3.9393210411071777
training loss at iteration 54520: 3.8535616397857666
training loss at iteration 54525: 4.103610992431641
training loss at iteration 54530: 5.204475402832031
training loss at iteration 54535: 4.274611949920654
training loss at iteration 54540: 3.4196114540100098
training loss at iteration 54545: 3.8542287349700928
training loss at iteration 54550: 2.900420904159546
training loss at iteration 54555: 4.218133926391602
training loss at iteration 54560: 3.702637195587158
training loss at iteration 54565: 4.377786159515381
training loss at iteration 54570: 4.17509126663208
training loss at iteration 54575: 4.399285316467285
training loss at iteration 54580: 4.801181316375732
training loss at iteration 54585: 3.8167858123779297
training loss at iteration 54590: 3.7509422302246094
training loss at iteration 54595: 3.3366339206695557
training loss at iteration 54600: 4.960026264190674
validation loss at iteration 54600: 4.456686496734619
training loss at iteration 54605: 7.162835597991943
training loss at iteration 54610: 4.2326860427856445
training loss at iteration 54615: 5.217791557312012
training loss at iteration 54620: 5.991508483886719
training loss at iteration 54625: 3.405985116958618
training loss at iteration 54630: 8.021293640136719
training loss at iteration 54635: 4.542353630065918
training loss at iteration 54640: 5.705121040344238
47%|██████████████▍ | 39641/85000 [15:58:31<18:16:47, 1.45s/it]

Thank you! @heilaw

Which pytorch version do you use?

Which pytorch version do you use?when I use your code,here is a problem with me .
import top_pool, bottom_pool, left_pool, right_pool
ImportError: /home/hdc/.local/lib/python3.6/site-packages/cpools-0.0.0-py3.6-linux-x86_64.egg/top_pool.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN2at15UndefinedTensor10_singletonE

How much performance increase did offsets bring?

Hi, I noticed that in your paper, there is an ablation study on using ground-truth offsets instead of predicted offsets. Is there any experiments comparing the performance of the model when using predicted offsets and not using offsets?
Thank you.

after reading your code ,i still do not understand how the corner-pooling work

How can I predict the objects in my own pictures？

I just replace the images with my own images of the file testdev2017，then run the python test.py CornerNet --testiter 500000 --split testing，and the error come up as follow：
cfg_file: ./config/CornerNet.json
loading all datasets...
split: testdev
loading from cache file: ./cache/coco_testdev2017.pkl
No cache file found...
loading annotations into memory...
Done (t=0.05s)
creating index...
index created!
20288it [00:08, 2410.16it/s]
loading annotations into memory...
Done (t=0.05s)
creating index...
index created!
system config...
{'batch_size': 49,
'cache_dir': './cache',
'chunk_sizes': [4, 5, 5, 5, 5, 5, 5, 5, 5, 5],
'config_dir': './config',
'data_dir': './data',
'data_rng': <mtrand.RandomState object at 0x7f847ac04438>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7f847ac04480>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': './results',
'sampling_function': 'kp_detection',
'snapshot': 5000,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
loading parameters at iteration: 500000
building neural network...
module_file: models.CornerNet
total parameters: 201035212
loading parameters...
loading model from ./cache/nnet/CornerNet/CornerNet_500000.pkl
locating kps: 0%| | 1/20288 [00:00<2:44:31, 2.06it/s]
Traceback (most recent call last):
File "test.py", line 99, in
test(testing_db, args.split, args.testiter, args.debug, args.suffix)
File "test.py", line 61, in test
testing(db, nnet, result_dir, debug=debug)
File "/home/dc2-user/CornerNet-master/test/coco.py", line 204, in testing
return globals()[system_configs.sampling_function](db, nnet, result_dir, debug=debug)
File "/home/dc2-user/CornerNet-master/test/coco.py", line 77, in kp_detection
height, width = image.shape[0:2]
AttributeError: 'NoneType' object has no attribute 'shape'
What should I do？Thanks。

get cocoapi clone

@heilaw
I am sorry to bother you,when I run git clone [email protected]:cocodataset/cocoapi.git coco
I got the error : Warning:Permanently added the RSA host key for IP address''to the list of known hosts.
Permission denied(publickey).
fatal:could not read from romote repository.
please make sure you have the correct access rights
and tht repository exists.
Is there any other website to get cooapi clone?

some problem for test

hello:
When I use weights for testing, I get an error.
#------------------------------------------------------------------------------------------------
loading model from ./cache/nnet/CornerNet/CornerNet_500000.pkl
locating kps: 0%| | 0/5000 [00:00<?, ?it/s]
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
#------------------------------------------------------------------------------------------------

and there will be a segment error when debugging。Is this the reason for insufficient memory?My software environment is the same as yours，but my computer has less memory

speed comparison with YOLO, SSD and Retinanet

Can you elaborate the speed comparison between YOLO, Retinanet and Faster RCNN

RuntimeError:cuda runtime error(2):out of memory at /opt/conda/conda-bld/pytorh_1524584710464/work/aten/src/THC/GENERIC/thcsTORAGE.CU:58

test.py Segmentation fault

There is "locating kps: 0%| | 0/20288 [00:00<?, ?it/s]Segmentation fault" when i run test.py ,how can i solve this problem?

How to compute the radius around ground truth corner?

I saw the function gaussian_radius but could not figure out how it worked, and the process was different with Formula of Root.

when I set the size (10,10) and iou 0.7, the returned radius is 2.73. However, when the box lies in the inner most side of ground truth box, the iou equals to [(10-22.73)^2]/(1010)=0.2.

So could you please explain the design idea behind it please? it bothers me for the whole night.

How fast did you perform when detecting one image.

Hi,

Great work!

You said that you use one-stage method for object detection. I want to know more details on the performance time with corner net.

Thank you!

Segmentation fault when training on my own data.

Hi,

Thank you for your wonderful work. But after I simply change the path to my own dataset (all pictures with the size of 1280x720), I meet the segmentation fault. :( Can you please help me locate the bug?

What's the magic number -2.19 mean?

In kp.py
# for tl_heat, br_heat in zip(self.tl_heats, self.br_heats):
# tl_heat[-1].bias.data.fill_(-2.19)
# br_heat[-1].bias.data.fill_(-2.19)

How to fine-tunning your pretrained cooco model?

How to fine tune your dataset on your trained coco model?
Could you provide the json file for fine-tunning params and update the load_pretrained_params() function in file : ./nnet/py_factory.py？

Is there any alternative to replace Corner pooling layer?

I mean, using standard Convolution or Pooling operation, but gets similar performance...

How worse is the performance of CornerNet if it is configured as only one stack of Houglass Net?

Thanks for your excellent contribution. In CornerNet, 2 stacks of Hourglass is employed for keypoint detection. I'm intereseted in how better or worse if CornerNet is configured as 1 stack or 3 stacks? Or, is '2 stacks Hourglass' configuration the minimum requirement for Key Point detection？Thanks.

Can it be possible to build a Keras or TensorFlow version. Thank you~

Your model is very creative. Can it be possible to build a Keras or TensorFlow version. Thank you very much.

How much does the loss value fall? and the network is converge.

How much does the loss value fall? and the network is converge.

THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1524584710464/work/aten/src/THC/THCTensorRandom.cu line=25 error=30 : unknown error

loading all datasets...
using 4 threads
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=11.78s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=8.33s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=7.80s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=11.06s)
creating index...
index created!
loading from cache file: ./cache/coco_minival2014.pkl
loading annotations into memory...
Done (t=0.38s)
creating index...
index created!
system config...
{'batch_size': 2,
'cache_dir': './cache',
'chunk_sizes': [4, 5, 5, 5, 5, 5, 5, 5, 5, 5],
'config_dir': './config',
'data_dir': './data',
'data_rng': <mtrand.RandomState object at 0x7f6494e26a68>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7f6494e26ab0>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': './results',
'sampling_function': 'kp_detection',
'snapshot': 5000,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
len of db: 118287
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
(480, 640, 3)
(480, 640, 3)
(427, 640, 3)
(434, 640, 3)
building model...
module_file: models.CornerNet
start prefetching data...
shuffling indices...
(480, 640, 3)
(375, 500, 3)
(480, 640, 3)
(427, 640, 3)
(333, 500, 3)
(640, 523, 3)
(424, 640, 3)
(480, 640, 3)
(375, 500, 3)
(500, 334, 3)
(480, 640, 3)
(500, 375, 3)
(416, 640, 3)
(500, 375, 3)
(425, 640, 3)
(480, 640, 3)
(375, 500, 3)
(640, 427, 3)
(640, 609, 3)
(383, 640, 3)
(428, 640, 3)
(426, 640, 3)
(427, 640, 3)
(640, 427, 3)
(640, 480, 3)
(555, 640, 3)
(480, 640, 3)
(416, 640, 3)
(375, 500, 3)
(375, 500, 3)
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1524584710464/work/aten/src/THC/THCTensorRandom.cu line=25 error=30 : unknown error

annotations

hello!
I can't download the annotatios.zip

RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 2)

I am trying to train the CornerNet on my machine but it produces the following error.

loading all datasets...
using 4 threads
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=8.28s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=8.38s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=9.22s)
creating index...
index created!
loading from cache file: ./cache/coco_trainval2014.pkl
loading annotations into memory...
Done (t=8.18s)
creating index...
index created!
loading from cache file: ./cache/coco_minival2014.pkl
loading annotations into memory...
Done (t=0.23s)
creating index...
index created!
system config...
{'batch_size': 1,
'cache_dir': './cache',
'chunk_sizes': [1],
'config_dir': './config',
'data_dir': './data',
'data_rng': <mtrand.RandomState object at 0x7fd5039ad4c8>,
'dataset': 'MSCOCO',
'decay_rate': 10,
'display': 5,
'learning_rate': 0.00025,
'max_iter': 500000,
'nnet_rng': <mtrand.RandomState object at 0x7fd5039ad510>,
'opt_algo': 'adam',
'prefetch_size': 5,
'pretrain': None,
'result_dir': './results',
'sampling_function': 'kp_detection',
'snapshot': 100,
'snapshot_name': 'CornerNet',
'stepsize': 450000,
'test_split': 'testdev',
'train_split': 'trainval',
'val_iter': 100,
'val_split': 'minival',
'weight_decay': False,
'weight_decay_rate': 1e-05,
'weight_decay_type': 'l2'}
db config...
{'ae_threshold': 0.5,
'border': 128,
'categories': 80,
'data_aug': True,
'gaussian_bump': True,
'gaussian_iou': 0.7,
'gaussian_radius': -1,
'input_size': [511, 511],
'lighting': True,
'max_per_image': 100,
'merge_bbox': False,
'nms_algorithm': 'exp_soft_nms',
'nms_kernel': 3,
'nms_threshold': 0.5,
'output_sizes': [[128, 128]],
'rand_color': True,
'rand_crop': True,
'rand_pushes': False,
'rand_samples': False,
'rand_scale_max': 1.4,
'rand_scale_min': 0.6,
'rand_scale_step': 0.1,
'rand_scales': array([0.6, 0.7, 0.8, 0.9, 1. , 1.1, 1.2, 1.3]),
'special_crop': False,
'test_scales': [1],
'top_k': 100,
'weight_exp': 8}
len of db: 118287
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
shuffling indices...
start prefetching data...
building model...
module_file: models.CornerNet
shuffling indices...
total parameters: 201035212
setting learning rate to: 0.00025
training start...
0%| | 0/500000 [00:00<?, ?it/s]
Traceback (most recent call last):
File "train.py", line 195, in
train(training_dbs, validation_db, args.start_iter)
File "train.py", line 137, in train
training_loss = nnet.train(**training)
File "/home/ekrem/PycharmProjects/CornerNet/nnet/py_factory.py", line 81, in train
loss = self.network(xs, ys)
File "/home/ekrem/anaconda3/envs/CornerNet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/ekrem/PycharmProjects/CornerNet/models/py_utils/data_parallel.py", line 68, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/ekrem/anaconda3/envs/CornerNet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/ekrem/PycharmProjects/CornerNet/nnet/py_factory.py", line 20, in forward
loss = self.loss(preds, ys, **kwargs)
File "/home/ekrem/anaconda3/envs/CornerNet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/ekrem/PycharmProjects/CornerNet/models/py_utils/kp.py", line 297, in forward
pull, push = self.ae_loss(tl_tag, br_tag, gt_mask)
File "/home/ekrem/PycharmProjects/CornerNet/models/py_utils/kp_utils.py", line 197, in _ae_loss
dist = tag_mean.unsqueeze(1) - tag_mean.unsqueeze(2)
RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 2)

I have set batch_size = 1 and chunk_sizes = [1] to get rid of the out of memory error as mentioned in #4

Any suggestions ?
Thanks

training cornernet on 8 GPU takes approximately 20 days?

some issue about test speed

I'm sorry for bothering you !
In the paper, you said that CornerNet infer an image with average time 244ms on a TitanX GPU, that means the speed is around 5fps.
That is much slower than other single stage detectors. Why is it so slow even without anchors and fpn?
Does the time cost most during corner_pooling?
@ywchao @heilaw @anewell @jiadeng
Thanks a lot for your reply!

I want to know how fast CornerNet is，Thank you！

I want to know how fast CornerNet is，Thank you！

Can I use mscoco2017 data to train and validate the model?

CUDA out of memory with batchsize = 1 in one 1080Ti

Code is different from your paper?

Hi, I saw your code. Is the prediction module of your code is slightly different from the one in your paper (Fig. 7)? In your code, the Top-left Corner Pooling is followed by 3x3 Conv-BN-ReLU.

IndexError: index 128 is out of bounds for axis 1 with size 128

When i train my own data , i meet this problem:

training loss at iteration 270: 8.091086387634277
training loss at iteration 275: 8.255900382995605
training loss at iteration 280: 8.269482612609863
training loss at iteration 285: 8.271137237548828
3%|█ | 286/10000 [07:02<3:59:10, 1.48s/it]Traceback (most recent call last):
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/chase/application/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/chase/application/CornerNet/sample/coco.py", line 157, in kp_detection
tl_regrs[b_ind, tag_ind, :] = [fxtl - xtl, fytl - ytl]
IndexError: index 128 is out of bounds for axis 1 with size 128
Process Process-2:
Traceback (most recent call last):
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 46, in prefetch_data
raise e
File "train.py", line 42, in prefetch_data
data, ind = sample_data(db, ind, data_aug=data_aug)
File "/home/chase/application/CornerNet/sample/coco.py", line 182, in sample_data
return globals()[system_configs.sampling_function](db, k_ind, data_aug, debug)
File "/home/chase/application/CornerNet/sample/coco.py", line 157, in kp_detection
tl_regrs[b_ind, tag_ind, :] = [fxtl - xtl, fytl - ytl]
IndexError: index 128 is out of bounds for axis 1 with size 128
3%|█ | 287/10000 [07:03<3:59:08, 1.48s/it]Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 50, in pin_memory
data = data_queue.get()
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 70, in rebuild_storage_fd
fd = df.detach()
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/resource_sharer.py", line 57, in detach
with _resource_sharer.get_connection(self._id) as conn:
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/resource_sharer.py", line 87, in get_connection
c = Client(address, authkey=process.current_process().authkey)
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/home/chase/anaconda2/envs/CornerNet/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory

Thank @heilaw

loading parameters... loading model from ./cache/nnet/CornerNet/CornerNet_500.pkl locating kps: 100%|███████████████████████████████|

is it possible to detect four corners for rotated boxes?

when i train my data, the category count is not 80 as coco, which code should i modify?

Dear all:

when i train my data, the category count is not 80 as coco, which code should i modify?

i find three part of code using "80" :

1. models/CornerNet.py: line 72
   class model(kp):
   def init(self, db):
   n = 5
   dims = [256, 256, 384, 384, 384, 512]
   modules = [2, 2, 2, 2, 2, 4]
   out_dim = 80 <---------------------- here need change to my category count???

2. db/detection.py: line 8
   self._configs["categories"] = 80 <---------------------- here need change to my category count???

3. config/CornerNet.json: line 45
   "categories": 80 <---------------------- here need change to my category count???

@heilaw Does all of these 3 place need to be change? and any other place need change?

RuntimeError: cuda runtime error (10) : invalid device ordinal at /opt/conda/conda-bld/pytorch_1525812548180/work/aten/src/THC/THCGeneral.cpp:70

THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1525812548180/work/aten/src/THC/THCGeneral.cpp line=70 error=10 : invalid device ordinal
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/lianbo/anaconda3/envs/Pytorch/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/lianbo/anaconda3/envs/Pytorch/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "train.py", line 55, in pin_memory
data["xs"] = [x.pin_memory() for x in data["xs"]]
File "train.py", line 55, in
data["xs"] = [x.pin_memory() for x in data["xs"]]

Some question about code

Hi writer!I have some question about your code .
the nstack seted in CornerNet.py is 2,it means that，the outputs of your networks are two tl heatmaps and two br heatmaps (tags and regs are same).why we use four heatmaps to caculate the focal loss?
thanks for your answer!

Multi node training

Does this project support multi node training?Thanks.

supported pytorch version?

Thx for the project and fresh approach. What version of pytorch does this project support?

Tried pytorch 0.3.1 and 0.4.1, got import errors with "python setup.py install --user".

demo on own video.

thank you for releasing this repo. could you guide me to the steps to do a demo of CornerNet on a video (an mp4 file) ?

thanks,