GithubHelp home page GithubHelp logo

lilytopia / wakenet Goto Github PK

View Code? Open in Web Editor NEW
41.0 1.0 11.0 5.08 MB

A CNN-based optical image ship wake detector.

License: MIT License

Python 77.09% Makefile 0.09% C++ 1.75% Cuda 15.03% Shell 0.06% Cython 5.99%
remote-sensing object-detection radon-transform ship-wake-detection retinanet-pytorch

wakenet's Introduction

WakeNet

A CNN-based optical image ship wake detector. It uses SWIM as the benchmark dataset.

If you find this code helpful in your research, please cite this paper:

@ARTICLE{9618934,
author={Xue, Fuduo and Jin, Weiqi and Qiu, Su and Yang, Jie},
journal={IEEE Transactions on Geoscience and Remote Sensing},
title={Rethinking Automatic Ship Wake Detection: State-of-the-Art CNN-based Wake Detection via Optical Images},
year={2021},
volume={},
number={},
pages={1-1},
doi={10.1109/TGRS.2021.3128989}}

Introduction

Most existing wake detection algorithms use Radon transform (RT) due to the long streak features of ship wakes in SAR images. The high false alarm rate of RT requires that the algorithms have significant human intervention for image preprocessing. When processing optical images these algorithms are greatly challenged because it is not only sea clutter that interferes with the wake detection in the optical waveband, but many other environmental factors. To solve the problem, in this repo, we address the automatic ship wake detection task in optical images from the idea of the CNN to design an end-to end detector as a novel method. The detector processes all the wake textures clamped by the V-shaped Kelvin arms as the object, and conducts detection via the OBB. The extra regression of wake tip coordinates and Kelvin arm direction can improve the hard wake detection performance while predicting the wake heading. Some special structures and strategies are also adopted according to the wake features.

Introduction

Detection Results on SWIM dataset

Method Baseline Backbone mAP (AP)
R2CNN Faster R-CNN ResNet-101 65.24%
R-RetinaNet RetinaNet ResNet-101 72.22%
R-YOLOv3 YOLOv3 DarkNet-53 54.87%
BBAVectors CenterNet ResNet-101 66.19%
WakeNet (Ours) RetinaNet FcaNet-101 77.04%

Results

Dependencies

Python 3.6.12, Pytorch 1.7.0, OpenCV-Python 3.4.2, CUDA 11.0

Getting Started

Installation

Build modules for calculating OBB overlap and performing Radon transform:

cd $ROOT/utils
sh make.sh

Prepare the ship wake dataset

WakeNet uses SWIM dataset collected by us as the benchmark dataset. The images in SWIM dataset are uniformly cropped into the size of 768 × 768, with 6,960 images for training, 2,320 images for testing, and 2,320 images for evaluation.

  1. Download SWIM dataset from Kaggle into the $ROOT directory.
  2. Modify the dataset path in datasets/generate_imageset.py and then run it.
  3. Run datasets/evaluate/swim2gt.py to generate ground truth labels for evaluation.
  • You may modify the above two files to support other datasets.

Train model

You can modify the training hyperparameters with the default config file hyp.py. Then, run the following command.

python train.py

Evaluate model

Test the mAP on the SWIM dataset. Note that the VOC07 metric is defaultly used.

python eval.py

Test model

Generate visualization results.

python demo.py

Code reference

Thanks to Rotated-RetinaNet for providing a reference in our code writing.

wakenet's People

Contributors

lilytopia avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

wakenet's Issues

ERROR IN eval.py

Every time I run the eval.py file i get the error:
/opt/conda/lib/python3.10/site-packages/shapely/set_operations.py:133: RuntimeWarning: invalid value encountered in intersection
return lib.intersection(a, b, **kwargs)

I know it is related with the skewiou function in map.py and that the problem has to do with shapely.geometry.Polygon but I don´t know why this happens exactly and how to solve it.

Can anybody help me please?

According to the instructions in the README, I encountered a ValueError: Buffer dtype mismatch, expected 'int' but got 'long' in train.py at the 20th epoch.

Traceback (most recent call last):
File "/home/lab/libr/WakeNet/eval.py", line 100, in
evaluate(arg.target_size,
File "/home/lab/libr/WakeNet/eval.py", line 85, in evaluate
results = data_evaluate(model, target_size, test_path, conf, dataset)
File "/home/lab/libr/WakeNet/eval.py", line 43, in data_evaluate
dets = im_detect(model, im, target_sizes=target_size, conf=conf)
File "/home/lab/libr/WakeNet/utils/detect.py", line 12, in im_detect
return single_scale_detect(model, src, target_size=target_sizes[0], use_gpu=use_gpu, conf=conf)
File "/home/lab/libr/WakeNet/utils/detect.py", line 36, in single_scale_detect
scores, classes, boxes, landmarks = model(im, test_conf=conf)
File "/opt/anaconda3/envs/libr/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/home/lab/libr/WakeNet/models/WakeNet.py", line 106, in forward
return self.decoder(imgs, anchors_list[-1], cls_score, bbox_pred, land_pred, test_conf=test_conf)
File "/home/lab/libr/WakeNet/models/WakeNet.py", line 124, in decoder
anchors_nms_idx = nms(torch.cat([bboxes.float(), scores.float()], dim=2)[0, :, :], nms_thresh)
File "/home/lab/libr/WakeNet/utils/nms_wrapper.py", line 13, in nms
return cpu_nms(dets, thresh) # 加了个int
File "cpu_nms.pyx", line 153, in cpu_nms.cpu_nms
ValueError: Buffer dtype mismatch, expected 'int' but got 'long'

有关运行环境的一些问题

首先非常感谢作者分享了WaveNet的代码,我按照您的教程配置好了环境,但是在运行train.py时遇到了如下问题:

Fail to speed up training via apex.

Namespace(augment=True, backbone='fca34', dataset='SWIM', eval_path='/home/xu/WakeNet/SWIM_Dataset_1.0.0/test.txt', freeze_bn=False, hyp='hyp.py', load=False, multi_scale=False, resume=False, target_size=[768], train_path='/home/xu/WakeNet/SWIM_Dataset_1.0.0/train.txt', training_size=768, weight='/home/xu/WakeNet/models/pretrained/fca34.pth')
{'lr0': 0.0001, 'warmup_lr': 1e-05, 'warm_epoch': 1.0, 'num_classes': 1.0, 'epochs': 100.0, 'batch_size': 12.0, 'save_interval': 5.0, 'test_interval': 5.0, 'lambda1': 1.0, 'lambda2': 0.2}
Weight loaded.
Model Summary: 280 layers, 4.83088e+07 parameters, 4.83088e+07 gradients

 Epoch   gpu_mem       cls   reg_box   reg_ldm     total   targets  img_size

0%| | 0/580 [00:04<?, ?it/s]
Traceback (most recent call last):
File "/home/xu/WakeNet/train.py", line 220, in
train_model(arg, hyps)
File "/home/xu/WakeNet/train.py", line 126, in train_model
losses = model(ims, gt_boxes, gt_landmarks, process=epoch / epochs)
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 161, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 171, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
output.reraise()
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/_utils.py", line 428, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
output = module(*input, **kwargs)
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/xu/WakeNet/models/WakeNet.py", line 94, in forward
land_pred = torch.cat([self.ldm_head0(features[0]), self.ldm_head1(features[1]), self.ldm_head2(features[2]),
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/xu/WakeNet/models/MultiHeads.py", line 143, in forward
x2 = self.convs2(x2)
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 423, in forward
return self._conv_forward(input, self.weight)
File "/home/xu/anaconda3/envs/wakenet/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 420, in _conv_forward
self.padding, self.dilation, self.groups)
RuntimeError: CUDA error: no kernel image is available for execution on the device


以下是我环境的一些信息,不知您是否有遇到过这样的问题,非常感谢!
OS: ubuntu 16.04 LTS
GPU: Mon Nov 29 14:42:29 2021
image

nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Thu_Jun_11_22:26:38_PDT_2020
Cuda compilation tools, release 11.0, V11.0.194
Build cuda_11.0_bu.TC445_37.28540450_0

Pytorch安装时用的是如下命令:
pip install torch==1.7.0+cu110 torchvision==0.8.0+cu110 torchaudio==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
也尝试过使用conda安装:
conda install pytorch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 cudatoolkit=11.0 -c pytorch
但都会报错

不知是否是双卡的问题呢?恳请您回复解答一二,感谢!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.