caitinzhao / cvpr2019_pyramid-feature-attention-network-for-saliency-detection Goto Github PK

help me, pls

bafs@bafs-xavier:~/Saliency/cvpr2019_Pyramid-Feature-Attention-Network-for-Saliency-detection$ python3 train.py --train_file=train_pair.txt --model_weights=model/vgg16_no_top.h5
/usr/lib/python3/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.
2020-07-16 21:18:06.318968: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
train_file model/vgg16_no_top.h5
model_weights train_pair.txt
<class 'int'>   <class 'int'>
256   256
WARNING:tensorflow:From /usr/lib/python3/dist-packages/keras/backend/tensorflow_backend.py:63: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From /usr/lib/python3/dist-packages/keras/backend/tensorflow_backend.py:492: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /usr/lib/python3/dist-packages/keras/backend/tensorflow_backend.py:3630: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING:tensorflow:From /usr/lib/python3/dist-packages/keras/backend/tensorflow_backend.py:3458: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From /usr/lib/python3/dist-packages/keras/backend/tensorflow_backend.py:3013: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From /home/bafs/Saliency/cvpr2019_Pyramid-Feature-Attention-Network-for-Saliency-detection/bilinear_upsampling.py:44: The name tf.image.resize_bilinear is deprecated. Please use tf.compat.v1.image.resize_bilinear instead.

Traceback (most recent call last):
  File "train.py", line 71, in <module>
    model = VGG16(model_input,dropout=dropout, with_CPFE=with_CPFE, with_CA=with_CA, with_SA=with_SA)
  File "/home/bafs/Saliency/cvpr2019_Pyramid-Feature-Attention-Network-for-Saliency-detection/model.py", line 106, in VGG16
    C345 = ChannelWiseAttention(C345, name='C345_ChannelWiseAttention_withcpfe')
  File "/home/bafs/Saliency/cvpr2019_Pyramid-Feature-Attention-Network-for-Saliency-detection/attention.py", line 58, in ChannelWiseAttention
    attention = Dense(C / 4, activation='relu')(attention)
  File "/usr/lib/python3/dist-packages/keras/engine/topology.py", line 576, in __call__
    self.build(input_shapes[0])
  File "/usr/lib/python3/dist-packages/keras/layers/core.py", line 830, in build
    constraint=self.kernel_constraint)
  File "/usr/lib/python3/dist-packages/keras/legacy/interfaces.py", line 87, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/keras/engine/topology.py", line 397, in add_weight
    weight = K.variable(initializer(shape),
  File "/usr/lib/python3/dist-packages/keras/initializers.py", line 212, in __call__
    dtype=dtype, seed=self.seed)
  File "/usr/lib/python3/dist-packages/keras/backend/tensorflow_backend.py", line 3631, in random_uniform
    dtype=dtype, seed=seed)
  File "/home/bafs/.local/lib/python3.6/site-packages/tensorflow_core/python/ops/random_ops.py", line 245, in random_uniform
    rnd = gen_random_ops.random_uniform(shape, dtype, seed=seed1, seed2=seed2)
  File "/home/bafs/.local/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_random_ops.py", line 822, in random_uniform
    name=name)
  File "/home/bafs/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 632, in _apply_op_helper
    param_name=input_name)
  File "/home/bafs/.local/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 61, in _SatisfiesTypeConstraint
    ", ".join(dtypes.as_dtype(x).name for x in allowed_list)))
TypeError: Value passed to parameter 'shape' has DataType float32 not in list of allowed values: int32, int64

First：in train.py, line 28 and line 34

model_name = args.model_weights
train_path = args.train_file

Second: in attention.py line42, line 46, line58

C is float32 here, it should be int32 or int64, so add int() over C

Third: in bilinear_upsampling.py line 18

if you meet error here, please do the modification like below
conv_utils.normalize_data_format(data_format) to K.common.normalize_data_format(data_format)

Hi, could you please share the versions of the dependencies you used:

Tensorflow (-gpu)
Keras
numpy
opencv-python
matplotlib

I'm having some issues running the notebook test that I think are due to some version inconsistencies.

在你的代码中edge = tf.nn.conv2d(x, laplace, strides=(1, 1, 1, 1), padding='SAME')出现了以下错误
ValueError: Shape must be rank 4 but is rank 3 for 'loss/Conv2D' (op: 'Conv2D') with input shapes: [?,640,640], [3,3,1,1].
我的输入是一张二值图（x），请问我该如何更改呢？谢谢。

The saliency map size on ECSSD and HKU-IS are inconsistent with the original images. How to resize back?

I have read the code and the paper carefully, in the paper, SA is followed by the low-level feature and CA is followed by the high-level feature， but in the code, both CA and SA are followed by the high-level feature. line 112 in "model.py" , "SA=SpatialAttention(C345, 'spatial_attention')", why? Shouldn't that be "C12=SpatialAttention(C12, 'spatial_attention')"?

所下载的显著性图肯定不是输入的原图，它的具体形式是什么样，原图在哪里，怎么进行输入，请求解答，感谢。

Here https://github.com/CaitinZhao/cvpr2019_Pyramid-Feature-Attention-Network-for-Saliency-detection/blob/master/model.py#L119 you set kernel_size=3 for classification "head". But common solution is to use kernel_size=1.

Is it a mistake or you choose such kernel after some tests?

I tried to train the model for DUTS dataset using a GPU and i get the following error message

My dependencies are
Cuda 10.0.130
cudann 7.6.4
tensorflow-gpu=1.15.2
opencv=3.4.5.20
my GPU is RTX2080Ti
thank you

Dear author,

Thanks for your great work !

Could you provide the trained VGG16 model that I want to directly test on my personal data ?

Thank you !

Can anybody guide which file to use to get results in Figure 5. I have trained the model using the given instructions

I'm using tensorflow-gpu 2.5 and keras 2.2.4

at model_input = Input(shape=(target_size[0],target_size[1],3)):

I got the error AttributeError: module 'tensorflow' has no attribute 'get_default_graph'

I then changed

from keras.layers import Input

to

from tensorflow.keras.layers import Input

then got

AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'

at

x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv1')(img_input)

in model.py

at

model = VGG16(model_input,dropout=dropout, with_CPFE=with_CPFE, with_CA=with_CA, with_SA=with_SA)

then I changed

from keras.models import *

to

from tensorflow.keras.models import *

but get same error

There is a shape issue when running the inference with the vgg model provide. It says it accepts only int but float provided. Any ideas on how to run inference for the model provided?

We cannot get the score in the paper according to you code.

python train.py
Using TensorFlow backend.
train_file model/vgg16_no_top.h5
model_weights train_pair.txt
0 input_1
1 block1_conv1
2 block1_conv2
3 block1_pool
4 dropout_1
5 block2_conv1
6 block2_conv2
7 block2_pool
8 dropout_2
9 block3_conv1
10 block3_conv2
11 block3_conv3
12 block3_pool
13 dropout_3
14 block4_conv1
15 block4_conv2
16 block4_conv3
17 block4_pool
18 dropout_4
19 block5_conv1
20 block5_conv2
21 block5_conv3
22 dropout_5
23 C4_cfe_cfe0
24 C4_cfe_cfe1_dilation
25 C4_cfe_cfe2_dilation
26 C4_cfe_cfe3_dilation
27 C5_cfe_cfe0
28 C5_cfe_cfe1_dilation
29 C5_cfe_cfe2_dilation
30 C5_cfe_cfe3_dilation
31 C3_cfe_cfe0
32 C3_cfe_cfe1_dilation
33 C3_cfe_cfe2_dilation
34 C3_cfe_cfe3_dilation
35 C4_cfeconcatcfe
36 C5_cfeconcatcfe
37 C3_cfeconcatcfe
38 C4_cfe_BN
39 C5_cfe_BN
40 C3_cfe_BN
41 C4_cfe_relu
42 C5_cfe_relu
43 C3_cfe_relu
44 C4_cfe_up2
45 C5_cfe_up4
46 C345_aspp_concat
47 C345_ChannelWiseAttention_withcpfe_GlobalAveragePooling2D
48 dense_1
49 dense_2
50 C345_ChannelWiseAttention_withcpfe_reshape
51 C345_ChannelWiseAttention_withcpfe_repeat
52 C345_ChannelWiseAttention_withcpfe_multiply
53 C345_conv
54 C345_BN
55 C345_relu
56 C345_up4
57 spatial_attention_1_conv1
58 spatial_attention_2_conv1
59 attention1_1_BN
60 attention2_1_BN
61 C2_conv
62 attention1_1_relu
63 attention2_1_relu
64 C1_conv
65 C2_BN_BN
66 spatial_attention_1_conv2
67 spatial_attention_2_conv2
68 C1_BN_BN
69 C2_BN_relu
70 attention1_2_BN
71 attention2_2_BN
72 C1_BN_relu
73 C2_up2
74 attention1_2_relu
75 attention2_2_relu
76 C12_concat
77 spatial_attention_add
78 C12_conv
79 activation_1
80 C12_BN
81 repeat_1
82 C12_relu
83 C12_atten_mutiply
84 fuse_concat
85 sa
Traceback (most recent call last):
File "train.py", line 72, in
model.load_weights(model_name,by_name=True)
File "/home/lthpc/anaconda3/envs/py27/lib/python2.7/site-packages/keras/engine/topology.py", line 2566, in load_weights
f = h5py.File(filepath, mode='r')
File "/home/lthpc/anaconda3/envs/py27/lib/python2.7/site-packages/h5py/_hl/files.py", line 271, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/home/lthpc/anaconda3/envs/py27/lib/python2.7/site-packages/h5py/_hl/files.py", line 101, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/h5py_1490028130695/work/h5py/_objects.c:2846)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/h5py_1490028130695/work/h5py/_objects.c:2804)
File "h5py/h5f.pyx", line 78, in h5py.h5f.open (/home/ilan/minonda/conda-bld/h5py_1490028130695/work/h5py/h5f.c:2123)
IOError: Unable to open file (File signature not found)
What did you do about it？

@CaitinZhao

I use your 'train.py' to train the model, but cant get the result given in your paper. So I guess this training configuration is different from your final version. Could you help me and point out the difference? Thanks a lot.

You can check my PyTorch implementation of this paper here.

@CaitinZhao , it would be nice if you could add the link to the Pytorch code in the README, it would be easier for people to find it. :)