nus-hpc-ai-lab / datm Goto Github PK
View Code? Open in Web Editor NEWICLR 2024, Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching
Home Page: https://gzyaftermath.github.io/DATM/
ICLR 2024, Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching
Home Page: https://gzyaftermath.github.io/DATM/
Dear NUS-HPC-AL-Lab,
I hope this message finds you well. I would like to extend my gratitude for the exceptional work that you and your team have conducted. Recently, I have developed a keen interest in the topic of data distillation and had the opportunity to read your paper on the subject.
While attempting to replicate the experiments detailed in your paper, I encountered a potential issue in the code concerning the evaluation process. Specifically, the configuration file sets num_eval=1 during the distillation process. I am considering modifying this setting to num_eval=5, which would obviate the need to run evaluation.py separately. Instead, I could directly use the maximum accuracy (Max_acc) and standard deviation (Max_std) recorded in wandb to obtain the results reported in your paper.
Could you please confirm if this modification would be appropriate or suggest any alternative approaches if necessary?
Thank you for your attention to this matter, and I look forward to your valuable feedback.
Best regards
Hello, we found that during the reproduction of your article, the "num_eval" in the distillation process is set to 1. Could you please explain why? If it's only one evaluation, how did you obtain the standard deviation (std)? In your paper, don't you mention the results averaged over five experiments?
In the CFG.py file, "num_eval" is set to 5, but during the distillation process, the configuration used is "config/xx/xxx.yaml," where all the YAML files have "num_eval" set to 1. This has caused us significant confusion in our progress. We would appreciate it if we could receive your response as soon as possible. Thank you.
Hello, could you please share the .pt files or the code for each IPC under the Random method mentioned in Table 1? Thank you.
Hello, when I was replicating your paper, I wanted to know if training expert trajectories requires training on the same dataset with and without ZCA whitening, and the same applies to distillation, right? My current question is whether I also need to train different expert trajectories based on alpha=0.3 or 1. It seems necessary according to the FTD paper, but buffer_FTD.py does not seem to handle the expert trajectory file paths for different alpha datasets.
When running DATM.py on CIFAR10 and CIFAR100, CIFAR10 hardly increases memory usage, but CIFAR100 significantly increases memory usage each time the following code is executed. What could be the reason for this?
for step in range(args.syn_steps):
.......
grad = torch.autograd.grad(ce_loss, student_params[-1], create_graph=True)[0]
.......
print("loading file {}".format(expert_files[expert_id[file_idx]]))
IndexError: list index out of range
Hi, appreciate your work. I got a problem when I try to apply your method to larger dataset of 90,000 images with 224*224 size. It seems that during buffer, the loading of images will automatically be killed at round 1800/90000 at "Train ZCA". Is there any way to solve such problem?
Greetings, I'm interested in visualizing the distilled images provided on the paper's GitHub page via Matplotlib. It would be a great help if you could send the script to show these images.
Thank you in advance for your attention and congratulations on the paper.
Hello,
I am attempting to reproduce the results presented in the paper [Title of the Paper]. Could you please clarify whether it is necessary to modify the config files provided in the repository? If so, could you provide guidance on which specific parameters or sections need to be adjusted to achieve the same experimental setup as described in the paper?
Thank you for your assistance.
Hello, I encountered an error "CFG:AssertionError: Invalid type <class 'NoneType'> for key batch_syn; valid types = {<class 'bool'>, <class 'tuple'>, <class 'float'>, <class 'list'>, <class 'str'>, <class 'int'>}" when using your code during distillation. Is this a version issue, or is there a problem with your code?
Hello, I encountered resource limitations while running experiments. It seems that CIFAR100's IPC10 requires 240GB of VRAM. I noticed that your paper mentioned using four A100 cards; is that really enough? I haven't tried CIFAR100's IPC50 yet, but I estimate it might need more than 320GB. Do you have any solutions? This includes when running tinyImageNet later, which I can't even imagine how demanding it will be. The paper mentioned reducing the synthesis steps, right? Are there any other methods?
Hello, while I was reviewing your code, I noticed that the configurations for each dataset and each ipc in the configuration files (configs/xxx/xxx.yaml) seem to differ from those listed in the final parameter table in your paper.
For example, for CIFAR10, the paper states a batch_syn of 10 for IPC1, but the code specifies 1000, and for IPC10, the paper states 100, but the code specifies 500. Nearly every configuration does not match.
Additionally, for CIFAR100, the three T values for IPC1 in the paper are 0, 10, and 20, but in the code, they are 0, 20, and 30.
I also do not understand why there are two opposite values of zca in the same code configuration. Which one is used during training?
If you have time, could you please answer these three questions so that we can better replicate your results? I would greatly appreciate it.
Excuse me for interrupting, I would like to ask how I can obtain the expert trajectory of TinyImage. When I run python buffer_FTD.py --dataset=Tiny --model=ConvNetD4 --train_epochs=100 --num_experts=100 --zca --buffer_path=../buffer_storage/ --data_path=../dataset/ --rho_max=0.01 --rho_min=0.01 --alpha=0.3 --lr_teacher=0.01 --mom=0. --batch_train=256, it reports an error saying the 'dataset/train' file does not exist
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.