nus-hpc-ai-lab / datm Goto Github PK

Dear NUS-HPC-AL-Lab,

I hope this message finds you well. I would like to extend my gratitude for the exceptional work that you and your team have conducted. Recently, I have developed a keen interest in the topic of data distillation and had the opportunity to read your paper on the subject.

While attempting to replicate the experiments detailed in your paper, I encountered a potential issue in the code concerning the evaluation process. Specifically, the configuration file sets num_eval=1 during the distillation process. I am considering modifying this setting to num_eval=5, which would obviate the need to run evaluation.py separately. Instead, I could directly use the maximum accuracy (Max_acc) and standard deviation (Max_std) recorded in wandb to obtain the results reported in your paper.

Could you please confirm if this modification would be appropriate or suggest any alternative approaches if necessary?

Thank you for your attention to this matter, and I look forward to your valuable feedback.

Best regards

Hello, we found that during the reproduction of your article, the "num_eval" in the distillation process is set to 1. Could you please explain why? If it's only one evaluation, how did you obtain the standard deviation (std)? In your paper, don't you mention the results averaged over five experiments?

In the CFG.py file, "num_eval" is set to 5, but during the distillation process, the configuration used is "config/xx/xxx.yaml," where all the YAML files have "num_eval" set to 1. This has caused us significant confusion in our progress. We would appreciate it if we could receive your response as soon as possible. Thank you.

Hello, could you please share the .pt files or the code for each IPC under the Random method mentioned in Table 1? Thank you.

Hello, when I was replicating your paper, I wanted to know if training expert trajectories requires training on the same dataset with and without ZCA whitening, and the same applies to distillation, right? My current question is whether I also need to train different expert trajectories based on alpha=0.3 or 1. It seems necessary according to the FTD paper, but buffer_FTD.py does not seem to handle the expert trajectory file paths for different alpha datasets.

When running DATM.py on CIFAR10 and CIFAR100, CIFAR10 hardly increases memory usage, but CIFAR100 significantly increases memory usage each time the following code is executed. What could be the reason for this?

for step in range(args.syn_steps):
       .......
       grad = torch.autograd.grad(ce_loss, student_params[-1], create_graph=True)[0]
       .......

print("loading file {}".format(expert_files[expert_id[file_idx]]))
IndexError: list index out of range

Hi, appreciate your work. I got a problem when I try to apply your method to larger dataset of 90,000 images with 224*224 size. It seems that during buffer, the loading of images will automatically be killed at round 1800/90000 at "Train ZCA". Is there any way to solve such problem?

Greetings, I'm interested in visualizing the distilled images provided on the paper's GitHub page via Matplotlib. It would be a great help if you could send the script to show these images.
Thank you in advance for your attention and congratulations on the paper.

Hello,

I am attempting to reproduce the results presented in the paper [Title of the Paper]. Could you please clarify whether it is necessary to modify the config files provided in the repository? If so, could you provide guidance on which specific parameters or sections need to be adjusted to achieve the same experimental setup as described in the paper?

Thank you for your assistance.

Hello, I encountered an error "CFG:AssertionError: Invalid type <class 'NoneType'> for key batch_syn; valid types = {<class 'bool'>, <class 'tuple'>, <class 'float'>, <class 'list'>, <class 'str'>, <class 'int'>}" when using your code during distillation. Is this a version issue, or is there a problem with your code?

Hello, I encountered resource limitations while running experiments. It seems that CIFAR100's IPC10 requires 240GB of VRAM. I noticed that your paper mentioned using four A100 cards; is that really enough? I haven't tried CIFAR100's IPC50 yet, but I estimate it might need more than 320GB. Do you have any solutions? This includes when running tinyImageNet later, which I can't even imagine how demanding it will be. The paper mentioned reducing the synthesis steps, right? Are there any other methods?

Hello, while I was reviewing your code, I noticed that the configurations for each dataset and each ipc in the configuration files (configs/xxx/xxx.yaml) seem to differ from those listed in the final parameter table in your paper.
For example, for CIFAR10, the paper states a batch_syn of 10 for IPC1, but the code specifies 1000, and for IPC10, the paper states 100, but the code specifies 500. Nearly every configuration does not match.
Additionally, for CIFAR100, the three T values for IPC1 in the paper are 0, 10, and 20, but in the code, they are 0, 20, and 30.
I also do not understand why there are two opposite values of zca in the same code configuration. Which one is used during training?
If you have time, could you please answer these three questions so that we can better replicate your results? I would greatly appreciate it.

Excuse me for interrupting, I would like to ask how I can obtain the expert trajectory of TinyImage. When I run python buffer_FTD.py --dataset=Tiny --model=ConvNetD4 --train_epochs=100 --num_experts=100 --zca --buffer_path=../buffer_storage/ --data_path=../dataset/ --rho_max=0.01 --rho_min=0.01 --alpha=0.3 --lr_teacher=0.01 --mom=0. --batch_train=256, it reports an error saying the 'dataset/train' file does not exist