ascust / sec-mxnet Goto Github PK
View Code? Open in Web Editor NEWMXNet implementation of SEC
License: MIT License
MXNet implementation of SEC
License: MIT License
I just have one object. My CLASS_NUM=2.When I run train_fg_cues.py, I got this:
Epoch[1] Batch[10] Speed: 29.01 samples/sec MultiLogisticLoss=0.100427 time=17/01/2019--21:28:53
Epoch[1] Batch[20] Speed: 29.69 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:28:58
Epoch[1] Batch[30] Speed: 29.92 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:03
Epoch[1] Batch[40] Speed: 29.68 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:08
Epoch[1] Batch[50] Speed: 29.80 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:13
Epoch[1] Batch[60] Speed: 28.67 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:18
Epoch[1] Batch[70] Speed: 29.99 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:23
Epoch[1] Batch[80] Speed: 28.36 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:29
Epoch[1] Batch[90] Speed: 28.10 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:34
Epoch[1] Batch[100] Speed: 28.93 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:39
Epoch[1] Batch[110] Speed: 28.58 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:44
Epoch[1] Batch[120] Speed: 29.89 samples/sec MultiLogisticLoss=0.000000 time=17/01/2019--21:29:49
U have used Segmentation classes for creating Augmented images.Will the Augmented images be used for training?If so aren't we using fully supervised data for training? Please let me know
generating cue file
cue file generated
{'VOC_TRAIN_LIST': 'train_list.txt', 'WD': 0.0005, 'EPOCH_SIZE': 1000, 'WORKSPACE': 512, 'CRF_BI_W': 10, 'BG_CUE_FILE': 'bg_cue_initsec.p', 'EPOCH_SIZE_BG': 8000, 'EPOCH_SIZE_FG': 8000, 'MAX_EPOCH': 8, 'LOG_FOLDER': 'log', 'CPU_WORKER_NUM': 8, 'CRF_BI_XY_STD': 80, 'BATCH_SIZE': 15, 'OUTPUT_FOLDER': 'outputs', 'LR_DECAY': 2000, 'LR': 0.001, 'CRF_POS_XY_STD': 3, 'CLASS_NUM': 21, 'VOCDEVKIT_PATH': '/home/hkt/Videos/VOCdevkit/VOC2012/', 'MOMENTUM': 0.9, 'CRF_BI_RGB_STD': 10, 'Q_FG': 0.996, 'FG_CUE_FILE': 'fg_cue_initsec.p', 'Q_BG': 0.999, 'CRF_POS_W': 3, 'INPUT_SIZE_SEC': 320, 'LR_FG': 0.001, 'BASE_NET': 'vgg16', 'SNAPSHOT_FOLDER': 'snapshots', 'SBD_PATH': '/home/hkt/Videos/benchmark_RELEASE/dataset/', 'BATCH_SIZE_FG': 15, 'VOC_VAL_IM_FOLDER': 'val_images', 'BATCH_SIZE_BG': 15, 'LR_BG': 0.001, 'DATASET_PATH': 'dataset', 'VOC_VAL_LIST': 'val_list.txt', 'LD_DECAY_BG': 2000, 'MEAN_RGB': [123, 117, 104], 'DOWN_SAMPLE_SEC': 8, 'VOC_TRAIN_MULTI_FILE': 'voc_multi_file.p', 'CACHE_PATH': 'cache', 'SALIENCY_TH_FG': 0.2, 'SALIENCY_TH_BG': 0.1, 'CUE_FILE_INITSEC': 'sec_cue.p', 'VOC_TRAIN_IM_FOLDER': 'train_images', 'VOC_VAL_MASK_FOLDER': 'val_masks', 'LR_DECAY_FG': 2000}
start training model SEC_vgg16
Traceback (most recent call last):
File "/home/hkt/PycharmProjects/SEC-MXNet-master/train_SEC.py", line 58, in
seg_net = net_symbol.create_training(class_num=conf.CLASS_NUM, outputsize=output_size, workspace=conf.WORKSPACE)
File "/home/hkt/PycharmProjects/SEC-MXNet-master/cores/symbols/SEC_vgg16.py", line 75, in create_training
bi_w=10, maxiter=10, scale_factor=12.0, min_prob=0.0001)
File "", line 52, in Custom
File "/home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/site-packages/mxnet/_ctypes/symbol.py", line 125, in _symbol_creator
ctypes.byref(sym_handle)))
File "/home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/site-packages/mxnet/base.py", line 251, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [13:15:07] src/operator/custom/custom.cc:104: Check failed: CustomOperator::Get()->Find(params.op_type) != nullptr Cannot find custom operator crf_layer
Stack trace returned 10 entries:
[bt] (0) /home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x39008a) [0x7fcf0796608a]
[bt] (1) /home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x3906c1) [0x7fcf079666c1]
[bt] (2) /home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x56d564) [0x7fcf07b43564]
[bt] (3) /home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x71811fa) [0x7fcf0e7571fa]
[bt] (4) /home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/site-packages/mxnet/libmxnet.so(MXSymbolCreateAtomicSymbol+0x6dc) [0x7fcf0a098e1c]
[bt] (5) /home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c) [0x7fcf2fbcdec0]
[bt] (6) /home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/lib-dynload/../../libffi.so.6(ffi_call+0x22d) [0x7fcf2fbcd87d]
[bt] (7) /home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/lib-dynload/_ctypes.so(_ctypes_callproc+0x4de) [0x7fcf2fde48de]
[bt] (8) /home/hkt/anaconda3/envs/paddlepaddle/lib/python2.7/lib-dynload/_ctypes.so(+0x9b31) [0x7fcf2fddab31]
[bt] (9) /home/hkt/anaconda3/envs/paddlepaddle/bin/../lib/libpython2.7.so.1.0(PyObject_Call+0x43) [0x7fcf3bccd973]
Process finished with exit code 1
just as the title says, may i ask whether you get the identical seeds as provided by the original repo, as I find the localization cue important for training the segmentation network
Hi, May I ask you questions about your implementation of softmax function?
in ./cores/symbols/SEC_syms.py, you implement softmax as follows:
def create_softmax(preds, min_prob=0.0001):
min_prob_const = mx.sym.ones((1, 1, 1, 1)) * min_prob
preds_max = mx.sym.max(preds, keepdims=True, axis=1, name="softmax_max")
preds_exp = mx.sym.exp(mx.sym.broadcast_minus(preds, preds_max, name="softmax_minus"))
tmp1 = mx.sym.sum(preds_exp, keepdims=True, axis=1)
tmp2 = mx.sym.broadcast_div(preds_exp, tmp1)
tmp3 = mx.sym.broadcast_add(tmp2, min_prob_const)
probs = mx.sym.broadcast_div(tmp3, mx.sym.sum(tmp3, axis=1, keepdims=True))
return probs
In my mind, the variable 'tmp2' is exactly the softmax output, why do you add another small value and normalize again?
I trained the model for generating the seed cues for my own dataset.But the foreground seed cue file contains blank arrays.Can you please suggest me how to correct it?
first i follow the readme training voc for vgg16, val miou = 0.5005 , result is ok.
then i training voc for resnet50 ,i only change the core.config.py file params for these :
conf.BASE_NET = "resnet50",
conf.LOG_FOLDER = "log_resnet50"
conf.SNAPSHOT_FOLDER = "snapshots_resnet50"
conf.OUTPUT_FOLDER = "outputs_resnet50"
val miou = 0.4883, result even became lower than vgg16, i did not know why? hope you can help me!
there are some final loss for vgg16 resnet50 log, first is vgg16,second is resnet50
//train_bg_cue_net.log
Train-MultiLogisticLoss= 0.008143 0.027573
//train_fg_cue_net.log
Train-MultiLogisticLoss= 0.057217 0.065230
//train_sec_model.log final loss
Train-SEC_seed_loss=0.246602 0.336656
Train-SEC_constrain_loss=0.226370 0.324345
Train-SEC_expand_loss=0.861525 0.847647
Hi, Thank you very much for sharing the code. There was an error while running: no such file or directory: 'dataset/voc_multi_file.p', bg_cue_net-1.params, fg_cue_net-1.params. Can you share this file?
do you test the single-gpu training performance?
I cann't find the net_symbol file.You used it in train_SEC.py
I have used your code to generate cues for a 2 category dataset.I got this runtime error while training the cues.Please do look into it
start training fg cues for SEC.
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/mxnet/symbol/symbol.py", line 1623, in simple_bind
ctypes.byref(exe_handle)))
File "/usr/local/lib/python3.5/dist-packages/mxnet/base.py", line 253, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [08:47:13] src/storage/storage.cc:119: Compile with USE_CUDA=1 to enable GPU usage
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train_fg_cues.py", line 56, in
momentum=conf.MOMENTUM, lr_decay=conf.LR_DECAY_FG, workspace=conf.WORKSPACE)
File "/home/rashmi/SEC-MXNet-master/cores/train_multi_wrapper.py", line 40, in train_multi_wrapper
label_shapes=data_iter.provide_label)
File "/usr/local/lib/python3.5/dist-packages/mxnet/module/module.py", line 429, in bind
state_names=self._state_names)
File "/usr/local/lib/python3.5/dist-packages/mxnet/module/executor_group.py", line 279, in init
self.bind_exec(data_shapes, label_shapes, shared_group)
File "/usr/local/lib/python3.5/dist-packages/mxnet/module/executor_group.py", line 375, in bind_exec
shared_group))
File "/usr/local/lib/python3.5/dist-packages/mxnet/module/executor_group.py", line 662, in _bind_ith_exec
shared_buffer=shared_data_arrays, **input_shapes)
File "/usr/local/lib/python3.5/dist-packages/mxnet/symbol/symbol.py", line 1629, in simple_bind
raise RuntimeError(error_msg)
RuntimeError: simple_bind error. Arguments:
label: (15, 20)
data: (15, 3, 320, 320)
[08:33:20] src/storage/storage.cc:119: Compile with USE_CUDA=1 to enable GPU usage
The URL of VGG16 is broken. Could you update the URL?
Sorry I'm new to MXNet, but the usage of your crf layer in SEC_vgg16.py and SEC_resnet50.py seems incorrect. I have looked into the CRF layer, it requires the input unary to be 'log(prob)' form, as indicated below:
def inference(self, im, unary):
#im: image HxWx3
#unary: CxHxW should be "log(p)". take log of probability
H, W = im.shape[:2]
C = unary.shape[0]
d = dcrf.DenseCRF2D(W, H, C)
d.setUnaryEnergy(-unary.reshape(C, -1))
however, in SEC_vgg16.py and SEC_resnet50.py, when you use the CRF layer, your input unary term seems not satisfy the requirement. for example, in SEC_vgg16.py
preds = create_classifier(body, class_num=class_num, workspace=workspace, lr_type="alex10")
probs = SEC_syms.create_softmax(preds)
crf = mx.symbol.Custom(data=preds, small_ims=small_ims, name='crf', op_type='crf_layer',
pos_xy_std=3, pos_w=3, bi_xy_std=80, bi_rgb_std=13, bi_w=10,
maxiter=10, scale_factor=12.0, min_prob=0.0001)
variable 'preds' is the predicted score map and I don't see you perform any futher transformations in 'crf_layer' before feeding it to inference function,
I just have one object. So My CLASS_NUM=2. After I run train_fg_cues.py and train_bg_cues.py, I run train_SEC.py, and I got Epoch[1] Batch[10] Speed: 20.38 samples/sec SEC_seed_loss=nan SEC_constrain_loss=nan SEC_expand_loss=nan time=17/01/2019--22:11:03.
Why this happened?
Hi
I trained bg_cues and fg_cues successfully. After that, when I train SEC then loss for S,E and C are nan with default setting of conf.Q_FG = 0.996, conf.Q_BG = 0.999 and others. When I tweak different parameters by lowering learning rate etc., no changes in loss as nan before. So after tweaking conf.Q_FG and conf.Q_BG by lowering it, loss giving some values and reducing till 5th batch. And then again giving nan for each. Problem lies in this when this warning comes then loss become nan.
Projects/SEC-MXNet/cores/symbols/crf_layer.py:24: RuntimeWarning: invalid value encountered in less
self.result[self.result<self.min_prob] = self.min_prob
Please enlighten me about that and conf.Q_FG & conf.Q_BG
More details:
def inference(self, im, unary):
#im: image HxWx3
#unary: CxHxW should be "log(p)". take log of probability
This unary is nan actually
unary = in_data[0].asnumpy()
small_ims = in_data[1].asnumpy()
N = unary.shape[0]
print 'in_data: ', in_data[0], in_data[1]
self.result = np.zeros(unary.shape, dtype=np.float32)
for i in range(N):
self.result[i] = self.CRF.inference(small_ims[i], unary[i])
# print np.min(unary[i]), np.max(unary[i]), np.mean(np.abs(unary[i]))
# print np.any(np.isnan(unary[i])), np.any(np.isnan(self.result[i]))
#print 'self.min_prob: ', self.result, self.min_prob
self.result[self.result<self.min_prob] = self.min_prob
@ascust Hello, your work is very amazing! I just want to segment humans and backgrounds, and I don't want to segment other objects on my own datasets. I have about 5000 images, 2500 of them contain humans, 2500 of them have no humans. However, in my dataset, the images contain humans are taken in the same indoor scene (scene A) and images without humans are taken in different indoor scenes (scene B, scene C, scene D ... ). None of the images without humans is taken in the scene A. Also, the images without humans may contains other objects, such as bottles, books, chairs, desks, etc. Can I generate accurate seeds to locate only human seeds as foreground in this case? Thank you very much!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.