lim-anggun / fgsegnet Goto Github PK

When I test those remaining pictures,I found the testing results are bad.The segmentation mask max is about 0.3,I don't know why ,But I read your paper,you said the testing results are good,so I want to ask you how to test ,Can you help me ? I 'm very confused.Thank you!

我想提取画面中的人物轮廓，如果背景中有随风摆动的树叶，或者来往的车辆，还可以很好的提取人体轮廓吗？

Hi , i have find the file "pyramids.py" and find the function "pyramid_reduce",but i didn't find the line of
"out_rows = math.ceil(rows / float(downscale))
out_cols = math.ceil(cols / float(downscale)) ",any soluvation,thanks

Hello, I did not find the test file in the file you posted, can you publish the test file and the training parameter model?

I read your paper,In your paper. it said that it takes  around 23.7 minutes to train for 50 epochs. Considering a video sequence which has 1700 frames with spatial size of 320x240,with a 200-frame traing ,but when I run your code ,I found it takes more time than 23.7 minutes to train in the CDnet2014_dataset.I don't know why ,I use the more version GPU than yours. you mean you take around 23.7 minutes to train for 50 epochs for a category's  one scene or all categories' all scene  ?

I have a question ,you said the video sequence with size of 320x240,you mean you resize all video sequences to 320x240?but in the code ,I didn't find you process the picture size.
Anthor question , after training .I use the training model weights to test the picture , the result is bad ,the predict picture is black . what is the reason? Thank you!

Is it possible to use arbitrary image size for training and testing? In your test_prediction_demo,the input image size must be(240, 320, 3), how can I modify it to test an arbitrary image? Thanks a lot!

Dear @lim-anggun,
How one can apply your implemented method on a new video sequence (i.e., a new video with, for example 50 first frames of it)? The network must be trained again on the new video sequence?

@lim-anggun Hi
I wanna know that whether I can run this trained model directly on my videos as we can do with opencv foregroundextractor and other vision libraries foreground and background extractors
Thanks

I want test those remaining pictures. But I did not find the test program. Can you share the test program for me? I 'm very confused.Thank you!

Hello,
I want to use your model as the backbone of my research, I tried to run the model, but it gave errors. Also, I install TensorFlow-gpu == 1.1.0 and Keras == 2.0.6.
Error:
When I run the get_data function, there was an output of the data format:
My Output --->> (5, 240, 320, 3) (5, 120, 160, 2) (5, 60, 80, 1)
Your OutPut---->> (5, 240, 320, 3) (5, 120, 160, 3) (5, 60, 80, 3)

This is not the same as your output format. So, I kindly request to you please, upload the requirements.txt file.

hi, i want to organize the CDnet dataset into (train_images, train_labels), (test_images, test_labels) for training and testing my own systeme mainly based on CNN with keras.

How should I extract the foreground mask？

any idea?

Traceback (most recent call last):
File "/Users/alpullu/Downloads/FgSegNet-master/FgSegNet/FgSegNet.py", line 372, in
results = generateData(train_dir, dataset_dir, scene)
File "/Users/alpullu/Downloads/FgSegNet-master/FgSegNet/FgSegNet.py", line 242, in generateData
del X_list_temp, inlist, gtlist
UnboundLocalError: local variable 'inlist' referenced before assignment

->>> baseline / highway
Traceback (most recent call last):

File "", line 1, in
runfile('D:/ShiYanchnegxu/FgSegNet_v2-master/testing_scripts/extract_mask.py', wdir='D:/ShiYanchnegxu/FgSegNet_v2-master/testing_scripts')

File "D:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)

File "D:\ProgramData\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/ShiYanchnegxu/FgSegNet_v2-master/testing_scripts/extract_mask.py", line 239, in
model = load_model(mdl_path)

File "D:\ProgramData\Anaconda3\envs\TF1.1.0\lib\site-packages\keras\models.py", line 227, in load_model
with h5py.File(filepath, mode='r') as f:

File "D:\ProgramData\Anaconda3\lib\site-packages\h5py_hl\files.py", line 269, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)

File "D:\ProgramData\Anaconda3\lib\site-packages\h5py_hl\files.py", line 99, in make_fid
fid = h5f.open(name, flags, fapl=fapl)

File "h5py_objects.pyx", line 54, in h5py._objects.with_phil.wrapper

File "h5py_objects.pyx", line 55, in h5py._objects.with_phil.wrapper

File "h5py\h5f.pyx", line 78, in h5py.h5f.open

OSError: Unable to open file (unable to open file: name = 'FgSegNet_v2\models25\baseline\mdl_highway.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

I want to know the train speed for 3000 pictures with one object...and I have a 1080Ti...

I want to apply FgSegNet on my project ，which programmed by C++

where is the main function? how to train it?

Dear Long Ang Lim,

Thank you for providing your source code with detail guide to run it.

I successfully trained your method on several CDNet videos. However, when I tried to test it on remaining frames say highway sequence, it does not segment foreground object. The binary mask obtained shows only background values.
I tried every possible solution. All in vain.
Do you have any idea what going wrong?

Thank You.

hi :
I run your test_prediction.ipynb, and use my train weight xxx.h5,but I get a error "Unknown layer: MyUpSampling2D" when load_model(mdl_path), do you got this issue ?
thanks vary much

"A common approach for moving objects segmentation in a scene is to perform a background subtraction.Several methods have been proposed in this domain."
above in your paper, but I readed your code,
for scene in scene_list:
print ('Training ->>> ' + category + ' / ' + scene)

    # training frame path and dataset2014 path
    # '..' up dir 1 level 
    train_dir = os.path.join('..', 'FgSegNet_dataset2014', category, scene + str(num_frames))
    dataset_dir = os.path.join('..', 'CDnet2014_dataset', category, scene)
        
    mdl_path = os.path.join(mdl_dir, 'mdl_' + scene + '.h5')

    results = generateData(train_dir, dataset_dir, scene)
    train(results, scene, mdl_path, log_dir, vgg_weights_path)

I have a question
how many models after trainning cdw_dataset2014? is it common approach ？

I have a question, after training the model ,how to save the model's weight? For every scene,I need to save corresponding weight,?if so ,it needs to save 52 weight files, could you help me?

Hi @lim-anggun thank you for your research in this current field. I have a question. In this FgSegNet, I tried to train it on CDNet dataset and I have this weight model.

Soon I realized that the model names is same with video names. Is it means that this method generate different model files (one model one video) for comparison in CDNet dataset? Is it possible to make generalized model where all of video only use 1 model file?, and how about the performance? Suppose I have different video should I must have different model file?

Thank you.

I have a problem, do you know how to solve it, thank you very much!
Training ->>> turbulence / turbulence2
(50, 315, 645, 3) (50, 158, 323, 3) (50, 79, 162, 3)

ValueError Traceback (most recent call last)
/apsarapangu/disk6/wuting/FgSegNet/FgSegNet-master/FgSegNet/FgSegNet.py in ()
219
220 results = generateData(train_dir, dataset_dir, scene)
--> 221 train(results, scene, mdl_path, log_dir, vgg_weights_path)
222 del results

/apsarapangu/disk6/wuting/FgSegNet/FgSegNet-master/FgSegNet/FgSegNet.py in train(results, scene, mdl_path, log_dir, vgg_weights_path)
147 redu = keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=reduce_factor, patience=num_patience, verbose=1, mode='auto', epsilon=0.0001, cooldown=0, min_lr=0)
148 model.fit([results[0], results[1], results[2]], results[3], validation_split=0.2, epochs=epoch, batch_size=batch_size,
--> 149 callbacks=[redu, chk, tb], verbose=1, class_weight=results[4], shuffle = True)
150
151 del model, results, tb, chk, redu

/apsarapangu/disk6/wuting/Anaconda3/lib/python3.5/site-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, **kwargs)
1356 class_weight=class_weight,
1357 check_batch_axis=False,
-> 1358 batch_size=batch_size)
1359 # Prepare validation data.
1360 if validation_data:

/apsarapangu/disk6/wuting/Anaconda3/lib/python3.5/site-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_batch_axis, batch_size)
1232 self._feed_input_shapes,
1233 check_batch_axis=False,
-> 1234 exception_prefix='input')
1235 y = _standardize_input_data(y, self._feed_output_names,
1236 output_shapes,

/apsarapangu/disk6/wuting/Anaconda3/lib/python3.5/site-packages/keras/engine/training.py in _standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
138 ' to have shape ' + str(shapes[i]) +
139 ' but got array with shape ' +
--> 140 str(array.shape))
141 return arrays
142

ValueError: Error when checking input: expected ip_scale2 to have shape (None, 157, 322, 3) but got array with shape (50, 158, 323, 3)

Hello how to evaluate the result of the method in CDnet 2014 dataset? I follow your links and tried to follow everything but I got this error.

The file C:\dataset\badWeather\blizzard\stats.txt doesn't exist.
It means there was an error calling the comparator.
Traceback (most recent call last):
  File "modified_process_folder.py", line 129, in <module>
    main()
  File "modified_process_folder.py", line 46, in main
    processFolder(datasetPath, binaryRootPath)
  File "modified_process_folder.py", line 62, in processFolder
    confusionMatrix = compareWithGroungtruth(videoPath, binaryPath)  #STATS
  File "modified_process_folder.py", line 84, in compareWithGroungtruth
    return readCMFile(statFilePath)
  File "modified_process_folder.py", line 90, in readCMFile
    raise Exception('error')
Exception: error

Should we recompile the comparator?, but in original file comparator.exe is exist.

Anyone could upgrade it to at least tensorflow 1.14?

I have trained a model with your code, but it can not segment foreground object.
The acc become 0.8937 and 0.8999 at first two epoch, and it will remain unchanged.
Is this normal?

Epoch 55/60
40/40 [==============================] - 3s - loss: 1.7747 - acc: 0.8937 - val_loss: 1.6751 - val_acc: 0.8999
Epoch 56/60
40/40 [==============================] - 2s - loss: 1.7732 - acc: 0.8937 - val_loss: 1.6737 - val_acc: 0.8999
Epoch 57/60
40/40 [==============================] - 2s - loss: 1.7717 - acc: 0.8937 - val_loss: 1.6722 - val_acc: 0.8999
Epoch 58/60
40/40 [==============================] - 2s - loss: 1.7703 - acc: 0.8937 - val_loss: 1.6708 - val_acc: 0.8999
Epoch 59/60
40/40 [==============================] - 2s - loss: 1.7689 - acc: 0.8937 - val_loss: 1.6694 - val_acc: 0.8999
Epoch 60/60
40/40 [==============================] - 2s - loss: 1.7675 - acc: 0.8937 - val_loss: 1.6681 - val_acc: 0.8999

Hello @lim-anggun, I would like to use your model on a collection of image frames I have. However, the initModel methods always assert the dataset name:

assert dataset_name in ['CDnet', 'SBI', 'UCSD'], 'dataset_name must be either one in ["CDnet", "SBI", "UCSD"]]'

So, I'm sure I have to edit the initModel methods for my own use.

It seems like there are different calls to the MyUpSampling2D method for each dataset name I choose, and I'm confused with what this portion of code really does or why it's needed (especially why there are differences in the num_pixels argument):

# Scale 2
x2_ups = {'tunnelExit_0_35fps':(0,1),'tramCrossroad_1fps':(1,0),'bridgeEntry':(1,1),
                      'busyBoulvard':(1,0),'fluidHighway':(0,1),'streetCornerAtNight':(1,1), 
                      'tramStation':(2,0),'winterStreet':(1,0),'twoPositionPTZCam':(1,0),
                      'peopleInShade':(1,1),'turbulence2':(1,1),'turbulence3':(1,0),
                      'skating':(1,1), 'wetSnow':(0,0)}

            for key, val in x2_ups.items():
                if self.scene == key and self.scene in ['skating', 'wetSnow']:
                    x2 = Cropping2D(cropping=((1, 1), val))(x2)
                    break
                elif self.scene==key:
                    x2 = MyUpSampling2D(size=(1, 1), num_pixels=val)(x2)
                    break

(FgSegNet_M_S_module.py, line 153-165)

I would like to know how I would have to modify this method for my dataset. I would appreciate it if you can help me with this, and explain what initModel does and why it has to change slightly according to the dataset.

Thank you, and great work on this!

I followed the steps mentioned in the README. After I downloaded the dataset and placed it in the right directory, I used FgSegNet.py to start the training. Pasting below the error log:

(50, 240, 320, 3) (50, 120, 160, 3) (50, 60, 80, 3)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/FgSegNet/FgSegNet/FgSegNet.py in <module>()
    219 
    220         results = generateData(train_dir, dataset_dir, scene)
--> 221         train(results, scene, mdl_path, log_dir, vgg_weights_path)
    222         del results

/FgSegNet/FgSegNet/FgSegNet.py in train(results, scene, mdl_path, log_dir, vgg_weights_path)
    141     img_shape = results[0][0].shape
    142     model = FgSegNetModule(lr, reg, img_shape, scene, vgg_weights_path)
--> 143     model = model.initModel()
    144 
    145     tb = keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=0, batch_size=batch_size, write_graph=False, write_grads=True, write_images=True, embeddings_freq=0, embeddings_layer_names=None, embeddings_metadata=None)

/content/FgSegNet/FgSegNet/FgSegNetModule.py in initModel(self)
     85 
     86         input_1 = Input(shape=(h, w, d), name='ip_scale1')
---> 87         vgg_layer_output = self.VGG16(input_1)
     88         shared_model = Model(inputs=input_1, outputs=vgg_layer_output, name='shared_model')
     89         shared_model.load_weights(self.vgg_weights_path, by_name=True)

/content/FgSegNet/FgSegNet/FgSegNetModule.py in VGG16(self, x)
     27 
     28         # Block 1
---> 29         x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv1', data_format='channels_last')(x)
     30         x = Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv2')(x)
     31         x = MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool')(x)

/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89         return wrapper
     90     return legacy_support
---> 91 
     92 
     93 generate_legacy_method_interface = functools.partial(generate_legacy_interface,

/usr/local/lib/python3.6/dist-packages/keras/layers/convolutional.py in __init__(self, filters, kernel_size, strides, padding, data_format, dilation_rate, activation, use_bias, kernel_initializer, bias_initializer, kernel_regularizer, bias_regularizer, activity_regularizer, kernel_constraint, bias_constraint, **kwargs)
    439                  bias_constraint=None,
    440                  **kwargs):
--> 441         super(Conv2D, self).__init__(
    442             rank=2,
    443             filters=filters,

TypeError: super(type, obj): obj must be an instance or subtype of type 

Did I miss something?