kaiyangzhou / ssdg-benchmark Goto Github PK

Hi,
Did you use the validation set in the model training? Considering that in PACS and Office-Home datasets, the number of the validation set is much larger than the number of the training set under semi-supervised DG setting, I think it is unreasonable to use such large validation sets to select hyperparameters or the final models because it is inconsistent with reality. I observe that you keep validation sets in the data splits, what are they used for?

Hi! I run your code and the data number of PACS is 9990 (6926+806+2048), but the paper (https://arxiv.org/pdf/1710.03077) says PACS has 9991 data.

***** Dataset statistics *****
Dataset: SSDGPACS Source domains: ['cartoon', 'photo', 'sketch']
Target domains: ['art_painting']
#classes: 7
#train_x: 210
#train_u: 6,926
#val: 806
#test: 2,048

Hi, it's me again. Sorry to disturb you. These days, i ran your codes on office_home data set and found problems. I download office_home data set though your link but it could not be used. And i could not fix it. First is the name of documents and then there is no file named "train" in office_home_dg/art/. Waiting for your help!!

Dear author,
When I follow your directions and run the command:bash run_ssdg.sh ssdg_pacs 210 v1, there is a error that Key error:"Img0". So I ran the command print(batch_x) and I found that there was no key called "img0". Only "img" and "img2" in that dict.
Here is the shot screen:

Good morning. Please i was trying to explore the semi supervised domain Generalization. I have followed the readme file to the last line but it came out with this line

Parsing files in output/ssdg_pacs/nlab_210/StyleMatch/resnet18/v1/art_painting
Traceback (most recent call last):
File "parse_test_res.py", line 188, in
main(args, end_signal)
File "parse_test_res.py", line 147, in main
end_signal=end_signal
File "parse_test_res.py", line 97, in parse_function
assert len(outputs) > 0, f'Nothing found in {directory}'
AssertionError: Nothing found in output/ssdg_pacs/nlab_210/StyleMatch/resnet18/v1/art_painting

Please what can i do?

Hi, I ran fixmatch about five times for PACS with 210 labels.
When the target domain is A, I only got 77.07% accuracy, which is 1% lower than the result in the paper.

Hi,
Could you share the val set lists of the Office-Home dataset?

Hi,
How about the experimental results if there is no stop-gradient operation on the weakly augmented branch?
Since there is no stop-gradient operation in the original FixMatch paper, what is the role of stop-gradient in SSDG?

I tried running your code and couldn't able to reproduce the numbers for the PACS dataset. The results I obtained for 5 samples per class and the numbers reported in the paper are as follows.

Art_Painting  77.18   78.54(reported)
Cartoon  73.74  74.44(reported)
Photo   89.35    89.25(reported)
Sketch  76.5   79.06(reported)
Avg    79.1925 80.32 (reported)

I can understand the 1% percentage point difference in the first three domains but

1. The numbers reported for a sketch are 3 percentage points higher than what I obtained
2. The number quoted for 5 samples per class is greater than the numbers obtained in the paper for 10 samples per class. I am a bit confused about this.

Can you please clarify this?

Thanks for ur interesting work. I am wondering how the datasets were split into labeled and unlabeled images, just stochastic? or selected by human. Thank you.

Looking through the code, I found this

 metric1 = {
        'name': 'accuracy',
        'regex': re.compile(r'\* accuracy: ([\.\deE+-]+)%')
    }

    metric2 = {
        'name': 'error',
        'regex': re.compile(r'\* error: ([\.\deE+-]+)%')
    }

Turns out that inside parse_function, the regex isn't matched with what is in the log.txt. This makes it not populate the output variable of the code shown below.

 for metric in metrics:
        match = metric['regex'].search(line)
        if match and good_to_go:
            print("good_to_go")
            if 'file' not in output:
                output['file'] = fpath
            num = float(match.group(1))
            name = metric['name']
            output[name] = num

What am I doing wrong?