vicoslab / mixed-segdec-net-comind2021 Goto Github PK

Where can we download this file download_and_extract.py

hi
This prj is GPU version, do you have CPU version? tks

Usually in other computer vision models, the data augmentation is used in training to improve the performance on the test data.
Why is the Image augmentation not used here to train the model?
Is there any specific reason?

Hello author

May I ask which scripts you use to split data sets into pyb files

I wonder why in every input_ksdd2.py input_ksdd.py input_dagm,py or other dataset, the len of the dataset should be 2*len(pos_samples) for training, why not len(pos_samples) + len(neg_samples)?

self.len = 2 * len(pos_samples) if self.kind in ['TRAIN'] else len(pos_samples) + len(neg_samples)

Thank you :)

In dataset.py, line 24
self.pos_imgs_permutation = np.random.permutation(self.num_neg)
Should self.num_neg be self.num_pos?

Thank you for your work, I want to ask that how I can train on other datasets besides the dataset mentioned in your article

hello author,

i am interested in your evaluation code to find the precision recall for each class. can i use this if my dataset has labels as png images. can you tell me the approach to convert the label colored images with many classes to other annotation format which is suitable for evaluation.

thank you

Hi, this model can extend to support multi-classes defects detection? Thanks

The training code runs too slow, please provide batch training code!. Thank you

Thanks for your great work!
can you provide a light version on windows system?

First, thanks for your public nice code!

I just wonder whether the AP metric in paper is about classification rather than semantic segmentation.
And in your "end2end.py " L246, it seems only evaluate for "predictions" which is result of classification.

By the way, I think "weakly supervision" is not precise in your mix supervision setting, because if your final evaluation is only about classification, class-level, pixel-level labels will all be full-supervised label for classification.

Could you open source the code?

Thanks for this great work. When I use a GTX2080TI 11G with an epoch of 50 rounds, the result of the SDD2 data set cannot be reproduced, which is 95.4%. My best result was only 92.6%. Can you give me some suggestions？

Thank you so much for sharing the scripts.

In line 129 of "End2End.py", "training_iteration()" returns "total_loss_seg + total_loss_dec" instead of "total_loss".
Is it supposed to mean something?

KSDD datasets are binary classification. If every picture in my datasets is an object with two or more defects, how can I modify the algorithm to be multi-classified?I'd appreciate it if you could help me！Thanks！

Hi, thanks for this great work. But some problems arise when I want to get your perfect results. I can't get the correct ROC and PR curve by training only on the DAGM dataset. Is there anything wrong? Could you please give me some advice on training work?
Thanks very much.

There are some details about my training log:
"0603_2_FOLD_1 Epoch 50/50 ==> avg_loss_seg=0.00284, avg_loss_dec=0.00041, avg_loss=0.00006, correct=158/158, in 19.77s/epoch (fwd/bck in 16.59s/epoch)
0603_2_FOLD_1 VALIDATION || AUC=nan, and AP=nan, with best thr=1.000000 at f-measure=nan and FP=5, FN=0, TOTAL SAMPLES=575"

Dear JakobBozic
my work requires the classification of defect categories for this data set, and I noticed that the public data set you provided does not classify defect categories.
Could you please provide a data set with the categories divided? Or, if I want to divide it myself, can you give me some suggestions?

Could you please share the dataset split code?

Error encountered during pytorch2onnx:
AttributeError:'SegDecNet' Object has no attribute 'glob_max_lr_mlutiplier_mask'

Problems when using my own dataset. How can I annotation multi-tagged images?Is it possible to use pseudo-color annotation like paddle?https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.5/docs/data/marker/marker.md

Directory structure

COCO

--IMAGE

--LABEL

D:\mixed-segdec-net-comind2021-master\mixed-segdec-net-comind2021-master>python -u train_net.py  --GPU=1 --DATASET=COCO --RUN_NAME=COCO --DATASET_PATH=datasets/COCO --RESULTS_PATH=save/results --SAVE_IMAGES=True --DILATE=7 --EPOCHS=50 --LEARNING_RATE=1.0 --DELTA_CLS_LOSS=0.01 --BATCH_SIZE=1 --WEIGHTED_SEG_LOSS=True --WEIGHTED_SEG_LOSS_P=2 --WEIGHTED_SEG_LOSS_MAX=1 --DYN_BALANCED_LOSS=True --GRADIENT_ADJUSTMENT=True --FREQUENCY_SAMPLING=True  --NUM_SEGMENTED=0 --FOLD=2
COCO Executing run with path save/results\COCO\COCO
COCO BATCH_SIZE                : 1
COCO DATASET                   : COCO
COCO DATASET_PATH              : datasets/COCO
COCO DELTA_CLS_LOSS            : 0.01
COCO DILATE                    : 7
COCO DYN_BALANCED_LOSS         : True
COCO EPOCHS                    : 50
COCO FOLD                      : 2
COCO FREQUENCY_SAMPLING        : True
COCO GPU                       : 1
COCO GRADIENT_ADJUSTMENT       : True
COCO INPUT_CHANNELS            : 1
COCO INPUT_HEIGHT              : 400
COCO INPUT_WIDTH               : 2448
COCO LEARNING_RATE             : 1.0
COCO MEMORY_FIT                : 1
COCO NUM_SEGMENTED             : 0
COCO ON_DEMAND_READ            : False
COCO REPRODUCIBLE_RUN          : False
COCO RESULTS_PATH              : save/results
COCO SAVE_IMAGES               : True
COCO TRAIN_NUM                 : None
COCO USE_BEST_MODEL            : False
COCO VALIDATE                  : True
COCO VALIDATE_ON_TEST          : True
COCO VALIDATION_N_EPOCHS       : 5
COCO WEIGHTED_SEG_LOSS         : True
COCO WEIGHTED_SEG_LOSS_MAX     : 1.0
COCO WEIGHTED_SEG_LOSS_P       : 2.0
datasets/COCO
268
264
536
datasets/COCO
268
264
532
536
536
COCO Saving current model state to save/results\COCO\COCO\models\ep_00.pth
COCO Returning seg_loss_weight 1.0 and dec_loss_weight 0.0
COCO Returning dec_gradient_multiplier 0
Traceback (most recent call last):
  File "train_net.py", line 63, in <module>
    end2end.train()
  File "D:\mixed-segdec-net-comind2021-master\mixed-segdec-net-comind2021-master\end2end.py", line 60, in train
    train_results = self._train_model(device, model, train_loader, loss_seg, loss_dec, optimizer, validation_loader, tensorboard_writer)
  File "D:\mixed-segdec-net-comind2021-master\mixed-segdec-net-comind2021-master\end2end.py", line 160, in _train_model
    for iter_index, (data) in enumerate(train_loader):
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 517, in __next__
    data = self._next_data()
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 557, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "C:\ProgramData\Anaconda3\envs\pytorch\lib\site-packages\torch\utils\data\_utils\fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "D:\mixed-segdec-net-comind2021-master\mixed-segdec-net-comind2021-master\data\dataset.py", line 49, in __getitem__
    ix = self.neg_imgs_permutation[ix]
IndexError: index 266 is out of bounds for axis 0 with size 264

input_coco.py

class COCODataset(Dataset):
    def __init__(self, kind: str, cfg):
        super(COCODataset, self).__init__(cfg.DATASET_PATH, cfg, kind)
        self.read_contents()

    def read_contents(self):
        pos_samples, neg_samples = [], []
        print(self.path)
        img_num = 0
        for image_name in glob.glob(self.path + '/IMAGE/*.jpg'):
            img_num += 1
            image_path = image_name
            image = self.read_img_resize(image_path, self.grayscale, self.image_size)
            img_name_short = image_name[:-4]
            jpg_name = img_name_short.split('\\')[-1]
            seg_mask_path = os.path.join(self.path, "LABEL", f"{jpg_name}.jpg")
            if os.path.exists(seg_mask_path):
                seg_mask, _ = self.read_label_resize(seg_mask_path, self.image_size, dilate=self.cfg.DILATE)
                image = self.to_tensor(image)
                seg_loss_mask = self.distance_transform(seg_mask, self.cfg.WEIGHTED_SEG_LOSS_MAX,
                                                        self.cfg.WEIGHTED_SEG_LOSS_P)
                seg_mask = self.to_tensor(self.downsize(seg_mask))
                seg_loss_mask = self.to_tensor(self.downsize(seg_loss_mask))
                pos_samples.append((image, seg_mask, seg_loss_mask, True, image_path, None, img_name_short))
            else:
                seg_mask = np.zeros_like(image)
                image = self.to_tensor(image)
                seg_loss_mask = self.to_tensor(self.downsize(np.ones_like(seg_mask)))
                seg_mask = self.to_tensor(self.downsize(seg_mask))
                neg_samples.append((image, seg_mask, seg_loss_mask, True, image_path, seg_mask_path, img_name_short))
        self.pos_samples = pos_samples
        self.neg_samples = neg_samples
        self.num_pos = len(pos_samples)
        print(len(pos_samples))
        self.num_neg = len(neg_samples)
        print(len(neg_samples))
        self.len = 2 * len(pos_samples) if self.kind in ['TRAIN'] else len(pos_samples) + len(neg_samples)
        print(self.len)
        self.init_extra()

end2end.py, line113:
total_correct += (decision > 0.5).item() == is_pos_.item()
decision is the output of model(fc layer)，it‘s not a probability, Is function sigmoid() missed?

Thank you for this great work.

Could you please update the license information, sir?

Thank you in advance.

Hello, I have been trying to use a pretrained segmentation network like UNET, deeplabv3 or deeplabv3plus inplace of the segmentation network in this paper but cannot seem to go by it. Can I get any help from this community?

Thanks for your great project, and I need to use jit.trace convert model to libtorch(C++) model ,code as below

model.to(device)
model.cuda()
model.eval()
example = torch.rand(1, 3, INPUT_HEIGHT,INPUT_WIDTH).cuda()
traced_script_module = torch.jit.trace(model, example)
traced_script_module.save("model_for_libtorch.pt")

RuntimeError:
Could not export Python function call 'GradientMultiplyLayer'. Remove calls to Python functions before export. Did you forget to add @script or @script_method annotation? If this is a nn.ModuleList, add it to constants:

I read the README.md file, but I am still confused about the implementation of the read_contents() method, so that I cannot currently train on my own data set. I hope you can give me some suggestions for my problem, thank you!

Traceback (most recent call last):
File "D:\project\mixed-segdec-net-comind2021-master\train_net.py", line 63, in
end2end.train()
File "D:\project\mixed-segdec-net-comind2021-master\end2end.py", line 41, in train
self.print_run_params()
File "D:\project\mixed-segdec-net-comind2021-master\end2end.py", line 363, in print_run_params
k, v = l.split(":")
ValueError: too many values to unpack (expected 2)

I want to get the solution about this code error.

Where is the weak supervision and mixed supervision training api you mentioned that I did not find in the code, please tell me thank you, Only the fully supervised api has been found.

Thank you for sharing. I have a problem with my dataset. When the maximum value of the mask is not 1.0 but 0.9 or 0.8, it will affect the correct statistics, even loss calculation. I guess it is a problem of downsampling, it may have to be normalized again.

hello, the number of the positive data is 246, all same in five different splits, is there anything wrong?
I think that the number in the name of the split represents the number of the positive data, do i make something wrong?

Thansk for your great work!

I was just wondering if it is possible to get image resulst from running demo_eval_single_image.py
It seems that only score is availale as a result.
An input image with a localization of defects is needed.

Hi ,
can you please tell me as to what parameters to use to train the model to get the AP 99.99% (as mentioned in paper)? I tried to train using below parameters on STEEL dataset but achieved AP 75% only. Also could you please tell me how is the class information given to the model for this dataset?

python -u train_net.py  \
        --GPU=0 \
        --DATASET=STEEL \
        --RUN_NAME=steel_server_1 \
        --DATASET_PATH=datasets/STEEL \
        --RESULTS_PATH=datasets/steel_result_1 \
        --SAVE_IMAGES=True \
        --DILATE=7 \
        --EPOCHS=50 \
        --VALIDATION_N_EPOCHS=5 \
        --LEARNING_RATE=0.1 \
        --DELTA_CLS_LOSS=0.1 \
        --BATCH_SIZE=500 \
        --WEIGHTED_SEG_LOSS=True \
        --WEIGHTED_SEG_LOSS_P=2 \
        --WEIGHTED_SEG_LOSS_MAX=1 \
        --DYN_BALANCED_LOSS=True \
        --GRADIENT_ADJUSTMENT=True \
        --FREQUENCY_SAMPLING=True \
        --TRAIN_NUM=3000 \
        --NUM_SEGMENTED=3000 \
        --FOLD=0

Thanks

I noticed that there will be exceptions if a sample contains less negative then positive samples. The issues occur in dataset.py, __getitem__. Ensuring the random choice sample size is not too big (size=min(self.num_neg, self.num_negatives_per_one_positive * self.num_pos), dataset.py, line 39) and using num_neg for the mod (ix = index % self.num_neg , dataset.py, line 48) would fix this.

Here is a bug in dataset.py as below figure

Hi, how to download KolektorSDD2 ? I can't find any downloading link at https://www.vicos.si/Downloads/KolektorSDD2