yandex-research / tab-ddpm Goto Github PK

In your code, the GPU you are using is named "cuda:1", but this fails on some computers.

For instance, on my computer, this dummy piece of code fails :

import torch
device = torch.device("cuda:1")
a = torch.tensor([1, 2, 3])
a.to(device)  # Fails

The error is

RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1

I had the same error with this repository, and I fixed this error by replacing all occurences of "cuda:1" to "cuda:0"

The files involved are:

• tab-ddpm/scripts/train.py

  Line 12 in b476257

  def __init__(self, diffusion, train_iter, lr, weight_decay, steps, device=torch.device('cuda:1')):

• tab-ddpm/scripts/train.py

  Line 91 in b476257

  device = torch.device('cuda:1'),

• tab-ddpm/scripts/pipeline.py

  Line 41 in b476257

  device = torch.device('cuda:1')

• tab-ddpm/scripts/sample.py

  Line 34 in b476257

  device = torch.device('cuda:1'),

• all the files in exp/*/*/config.toml

Hi,
When I run : !python scripts/pipeline.py --config /content/drive/MyDrive/tab-ddpm/data/industry/config.toml --train --sample

The error occured:
mlp
File "/content/drive/MyDrive/tab-ddpm/scripts/tab_ddpm/modules.py", line 470, in forward
emb += F.silu(self.label_emb(y))
RuntimeError: The size of tensor a (256) must match the size of tensor b (39) at non-singleton dimension 1

and I print out the emb size which is torch.Size([256, 128]); and label_emb(y) size is [256,39,128], where y shape is [256,39];

I tried to replace emb += F.silu(self.label_emb(y)) with emb += F.silu(self.label_emb(y.sequeeze())) as ResNetDiffusion did in module.py, but it does not work, still print out the same error message.

Hi I am having the following errors..
I am very new to running the code through cmd, please help me to fix this..

(tddpm) C:\Users\namjo\Downloads\tab-ddpm-main\tab-ddpm-main>python scripts/tune_ddpm.py churn2 6500 synthetic catboost ddpm_tune --eval_seeds
[I 2023-07-14 11:54:06,300] A new study created in memory with name: no-name-5aa9ef04-90d2-4ef6-8808-158a7117fe5f
C:\Users\namjo\anaconda3\envs\tddpm\lib\site-packages\optuna\progress_bar.py:47: ExperimentalWarning: Progress bar is experimental (supported from v1.2.0). The interface can change in the future.
self._init_valid()
[W 2023-07-14 11:54:06,314] Trial 0 failed because of the following error: FileNotFoundError(2, 'The system cannot find the file specified', None, 2, None)
Traceback (most recent call last):
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\site-packages\optuna\study_optimize.py", line 213, in _run_trial
value_or_values = func(trial)
File "C:\Users\namjo\Downloads\tab-ddpm-main\tab-ddpm-main\scripts\tune_ddpm.py", line 85, in objective
subprocess.run(['python3.9', f'{pipeline}', '--config', f'{exps_path / "config.toml"}', '--train', '--change_val'], check=True)
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\subprocess.py", line 505, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\subprocess.py", line 951, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\subprocess.py", line 1420, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
0%| | 0/50 [00:00<?, ?it/s]
Traceback (most recent call last):
File "C:\Users\namjo\Downloads\tab-ddpm-main\tab-ddpm-main\scripts\tune_ddpm.py", line 113, in
study.optimize(objective, n_trials=50, show_progress_bar=True)
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\site-packages\optuna\study\study.py", line 400, in optimize
_optimize(
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\site-packages\optuna\study_optimize.py", line 66, in _optimize
_optimize_sequential(
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\site-packages\optuna\study_optimize.py", line 163, in _optimize_sequential
trial = _run_trial(study, func, catch)
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\site-packages\optuna\study_optimize.py", line 264, in _run_trial
raise func_err
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\site-packages\optuna\study_optimize.py", line 213, in _run_trial
value_or_values = func(trial)
File "C:\Users\namjo\Downloads\tab-ddpm-main\tab-ddpm-main\scripts\tune_ddpm.py", line 85, in objective
subprocess.run(['python3.9', f'{pipeline}', '--config', f'{exps_path / "config.toml"}', '--train', '--change_val'], check=True)
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\subprocess.py", line 505, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\subprocess.py", line 951, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\namjo\anaconda3\envs\tddpm\lib\subprocess.py", line 1420, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

Hi, thanks for providing the code to your Paper.
You wrote:

It takes approximately 7min to run the script above (NVIDIA GeForce RTX 2080 Ti).

to run your TabDDPM pipeline.

I was wondering if this was also the time you used for training in order to produce the results in your Paper (with an RTX 2080Ti)?
If not, for how long did you approximately do the training and on what kind of GPU?

Cheers, Sven

I am trying to reproduce the results presented in Figure 2 (individual feature distributions) of the paper but encountering difficulties despite following the methodology and code provided. Any guidance on accurately reproducing these results would be greatly appreciated.

Hi,

I found TabDDPM needs to tune the hyerparameters for each dataset (even each ML evaluators). I want to know if there are any recommended parameters that can performe good on every dataset? Or TabDDPM is sensitive to the hyperparameters?

Looking forward to your reply!
Best,

Great work here, I'm curious if there is a supported way to reconstruct the synthetic data into a single DataFrame / CSV with the original column names preserved?

Hi,

I want to know how you deal with imbalance label problem? When the label distribution is highly skewed, the generative model may only generate one class label, which cannot use to evaluate ROC AUC, even with cross validation. Besides, should I use cross validation for real data? I cannot find the codes to tackle this problem, looking forward to your reply! Thanks!

Hi, Thanks for uploading the code with the paper.

I just wanted to work this code,
After setting up the environment using conda and downloading this code and the dataset, I ran the TabDDPM pipeline as you wrote on GitHub Repository.

python scripts/pipeline.py --config exp/churn2/ddpm_cb_best/config.toml --train --sample

However, the commander made some error.

(tddpm) C:\Users\Desktop\Code\tab-ddpm-main>python scripts/pipeline.py --config exp/churn2/ddpm_cb_best/config.toml --train --sample
Traceback (most recent call last):
  File "C:\Users\Desktop\Code\tab-ddpm-main\scripts\pipeline.py", line 5, in <module>
    from train import train
  File "C:\Users\Desktop\Code\tab-ddpm-main\scripts\train.py", line 6, in <module>
    from tab_ddpm import GaussianMultinomialDiffusion
ModuleNotFoundError: No module named 'tab_ddpm'

May I know the reason why tab_ddpm is not imported?
I did not modify any code, and just wanted to work this code.
I just turned the code, but I cannot understand why the error is coming out.

I am using provided churn dataset as input and in output I am getting generated categorical and numerical columns in .npy format....there are 4 categorical and 7 numerical columns . How to identify names of those columns?

I would suggest to include a bare bone basic pipeline of your code. The current layout of the code is every difficult to understand with a lot of boilerplate useless coding in-between (which makes sense in API's) but is not really required to understand the concept.

Hello, I'm getting an access error to the provided dataset link it seems the link is no longer valid: https://www.dropbox.com/s/rpckvcs3vx7j605/data.tar?dl=0

Is it possible to have a valid link for the used dataset?

Thank you.

Hello,

Thanks for this code, from the paper it looks very promising, so I'd like to integrate it in my workflow. I'd like not to multiply virtual environments to run different synthetic data generators, so my request is to use pytorch version 2.0.x, and I wonder if there is a good reason why you do not allow this ?

Hello, I'am struggling adapting your code to apply the principals on my own datasets. Could you add a short explanation on how to do this?

thanks a lot

It appears that this repository does not contain a License file. Could you please add a license file to clarify how people can use or adapt the code? Many thanks!

I've had success in training/generating data with this package on a variety of different datasets, but I have noticed when there is a very high cardinality feature present in a dataset this package fails with a very uninformative error message: "Killed" and nothing else. As soon as I remove the high-cardinality feature, it runs fine. By high-cardinality I mean on the order of tens of thousands of unique values for a given column. Not sure how to debug or where to start given the uninformative nature of the error message. The last line of code that seems to be executed before it gets killed is line 579 in lib/data,py. Any ideas? Anyone else have this same issue?

The server of the email address which is related to the first author of the paper "TABDDPM: MODELLING TABULAR DATA WITH
DIFFUSION MODELS" is so aggressiv that it is impossible for me to write you an email: "Message rejected under suspicion of SPAM"

in:
https://github.com/rotot0/tab-ddpm/blob/5ac62c686ab177afcf7ae97492e15ac99984a14a/scripts/tune_evaluation_model.py#L121

train_func is train_catboost which returns a MetricsReport object.
This object cannot be unpacked into the 4 variables (same in this line)

I assume _m1 & _m2 refer to "Metrics 1" and "Metrics 2" for validation and testing set.

Since only val_m2 is used i would suggest to use:

score = train_func( (...) ).get_metric(split="val", metric="[acc|f1|roc_auc]")

What metric do you think is best suited for tuning the Catboost model / what metric is val_m2 supposed to be?

Thank you very much for your great work!
However, I am a little bit confused about this line, which will return nan and makes the loss_multi loss be nan. Thank you very much for your reply!

First I would like to thank for the sharing of the code, the idea of using diffusion model to solve tabular data is mind blowing.

I am trying to reproduce the project after I gather all the package I need
I run the command
python scripts/pipeline.py --config exp/churn2/ddpm_cb_best/config.toml --train --sample

and it raise the error message

File "D:\Dev\tab_ddpm\scripts\pipeline.py", line 5, in
from train import train
File "D:\Dev\tab_ddpm\scripts\train.py", line 8, in
from utils_train import get_model, make_dataset, update_ema
File "D:\Dev\tab_ddpm\scripts\utils_train.py", line 3, in
import lib
File "D:\Dev\tab_ddpm\lib_init_.py", line 8, in
from .data import * # noqa
File "D:\Dev\tab_ddpm\lib\data.py", line 22, in
from .metrics import calculate_metrics as calculate_metrics_
File "D:\Dev\tab_ddpm\lib\metrics.py", line 17, in
class MetricsReport:
File "D:\Dev\tab_ddpm\lib\metrics.py", line 37, in MetricsReport
def get_splits_names(self) -> list(str):
TypeError: 'type' object is not iterable

I have googled it, but most of the answer tell me to rewrite the class object. I still have no clue how to fix the bus

What are the xyz_cv.json files in the tuned_models folder for?

Hello Everyone,
I want to use TABDDPM model to generate synthetic data for a specific dataset. I am figuring out the detailed understanding of GitHub Repository for the model. If someone could guide me how I can use the source code of the model in Jupyter notebook, it would be really helpful for me. Or if anyone can provide any resources where I can get detailed knowledge of understanding GitHub repositories and the usage of it for different datasets, it would be appreciated.

Thankyou so much!

Hi,
First of all: Great work and well written code! It's easy to follow and pretty self-explanatory.

I am trying to recreate your results from the paper and got the code running (only using the Adult Dataset so far).
I have a few questions and was wondering if you might be able to help me out:

Can you explain to me, what is shown on Figure 2 in your Paper.
Lets take Adult (AD) as an example:
Q1: what do you mean with categorical_feature 3?

• What exactly do you mean with "feature" (--> the columns?).
• What does 1 individual bar in one bar chart represent (each bar = each column)? What does their height mean?
• what does the "3" mean?

Q2: Is the code to recreate Figure 2 & 3 also publicly available?

Q3.1: Did I understand the function of your scripts correct:
tune_ddpm.py is used to train multiple different model versions with different hyperparameter by internally calling pipeline.py with different configs and storing the best found model in the end.
The best found model version can be evaluated over multiple different seeds using the eval_seeds.py script.
Q3.2: Is the above order of script execution correct to recreate your results?

Q4: I am not quite sure for what the tune_evaluation_model.py script is used? Aren't we training the evaluation models already in eval_seeds.py?

Cheers, Sven

Hi, I'm trying to generate the synthetic data by following the README.
The training works well, and I'm able to find the numerical generated values, but I can't get the categorical generated values.
In detail,exp/churn2/ddpm_mlp_best/X_cat_train.npy is empty.
On the other hand, I can see the values in exp/churn2/ddpm_mlp_best/X_num_train.npy.
Is this a bug, or am I missing anything? Thanks!

python scripts/pipeline.py --config exp/churn2/ddpm_mlp_best/config.toml --train --sample 

[3 2 2 2]
16
{'num_classes': 2, 'is_y_cond': True, 'rtdl_params': {'d_layers': [512, 1024, 1024, 1024, 1024, 256], 'dropout': 0.0}, 'd_in': 16}
mlp
Step 500/30000 MLoss: 0.713 GLoss: 0.9151 Sum: 1.6280999999999999
Step 1000/30000 MLoss: 0.7205 GLoss: 0.6157 Sum: 1.3362
Step 1500/30000 MLoss: 0.7073 GLoss: 0.5363 Sum: 1.2436
Step 2000/30000 MLoss: 0.718 GLoss: 0.506 Sum: 1.224
Step 2500/30000 MLoss: 0.7069 GLoss: 0.484 Sum: 1.1909
Step 3000/30000 MLoss: 0.7195 GLoss: 0.4682 Sum: 1.1877
Step 3500/30000 MLoss: 0.721 GLoss: 0.4381 Sum: 1.1591
Step 4000/30000 MLoss: 0.7193 GLoss: 0.4203 Sum: 1.1396000000000002
Step 4500/30000 MLoss: 0.7173 GLoss: 0.412 Sum: 1.1293
Step 5000/30000 MLoss: 0.7159 GLoss: 0.4084 Sum: 1.1242999999999999
Step 5500/30000 MLoss: 0.7206 GLoss: 0.4068 Sum: 1.1274
Step 6000/30000 MLoss: 0.7188 GLoss: 0.4046 Sum: 1.1234
Step 6500/30000 MLoss: 0.7161 GLoss: 0.3968 Sum: 1.1129
Step 7000/30000 MLoss: 0.7148 GLoss: 0.3964 Sum: 1.1112
Step 7500/30000 MLoss: 0.7152 GLoss: 0.3985 Sum: 1.1137
Step 8000/30000 MLoss: 0.7154 GLoss: 0.3928 Sum: 1.1082
Step 8500/30000 MLoss: 0.6898 GLoss: 0.3933 Sum: 1.0831
Step 9000/30000 MLoss: 0.7257 GLoss: 0.3968 Sum: 1.1225
Step 9500/30000 MLoss: 0.7103 GLoss: 0.3969 Sum: 1.1072
Step 10000/30000 MLoss: 0.7095 GLoss: 0.3844 Sum: 1.0939
Step 10500/30000 MLoss: 0.7171 GLoss: 0.3905 Sum: 1.1076
Step 11000/30000 MLoss: 0.7225 GLoss: 0.3903 Sum: 1.1128
Step 11500/30000 MLoss: 0.6993 GLoss: 0.3919 Sum: 1.0912000000000002
Step 12000/30000 MLoss: 0.7088 GLoss: 0.3844 Sum: 1.0932
Step 12500/30000 MLoss: 0.726 GLoss: 0.385 Sum: 1.111
Step 13000/30000 MLoss: 0.7013 GLoss: 0.3832 Sum: 1.0845
Step 13500/30000 MLoss: 0.7045 GLoss: 0.382 Sum: 1.0865
Step 14000/30000 MLoss: 0.6991 GLoss: 0.3807 Sum: 1.0798
Step 14500/30000 MLoss: 0.7229 GLoss: 0.3812 Sum: 1.1040999999999999
Step 15000/30000 MLoss: 0.7096 GLoss: 0.3825 Sum: 1.0921
Step 15500/30000 MLoss: 0.7081 GLoss: 0.3788 Sum: 1.0869
Step 16000/30000 MLoss: 0.7158 GLoss: 0.3816 Sum: 1.0974
Step 16500/30000 MLoss: 0.7067 GLoss: 0.377 Sum: 1.0836999999999999
Step 17000/30000 MLoss: 0.6844 GLoss: 0.3759 Sum: 1.0603
Step 17500/30000 MLoss: 0.699 GLoss: 0.373 Sum: 1.072
Step 18000/30000 MLoss: 0.692 GLoss: 0.3802 Sum: 1.0722
Step 18500/30000 MLoss: 0.6822 GLoss: 0.3725 Sum: 1.0547
Step 19000/30000 MLoss: 0.7103 GLoss: 0.3775 Sum: 1.0878
Step 19500/30000 MLoss: 0.7161 GLoss: 0.373 Sum: 1.0891
Step 20000/30000 MLoss: 0.6964 GLoss: 0.3737 Sum: 1.0701
Step 20500/30000 MLoss: 0.6908 GLoss: 0.3772 Sum: 1.068
Step 21000/30000 MLoss: 0.687 GLoss: 0.3743 Sum: 1.0613000000000001
Step 21500/30000 MLoss: 0.6917 GLoss: 0.3731 Sum: 1.0648
Step 22000/30000 MLoss: 0.6878 GLoss: 0.3727 Sum: 1.0605
Step 22500/30000 MLoss: 0.7067 GLoss: 0.3743 Sum: 1.081
Step 23000/30000 MLoss: 0.6958 GLoss: 0.3685 Sum: 1.0643
Step 23500/30000 MLoss: 0.7281 GLoss: 0.3687 Sum: 1.0968
Step 24000/30000 MLoss: 0.7148 GLoss: 0.3705 Sum: 1.0853
Step 24500/30000 MLoss: 0.7021 GLoss: 0.3696 Sum: 1.0716999999999999
Step 25000/30000 MLoss: 0.6956 GLoss: 0.3667 Sum: 1.0623
Step 25500/30000 MLoss: 0.6925 GLoss: 0.3654 Sum: 1.0579
Step 26000/30000 MLoss: 0.6943 GLoss: 0.3682 Sum: 1.0625
Step 26500/30000 MLoss: 0.6899 GLoss: 0.3646 Sum: 1.0545
Step 27000/30000 MLoss: 0.6971 GLoss: 0.36 Sum: 1.0571000000000002
Step 27500/30000 MLoss: 0.7294 GLoss: 0.3662 Sum: 1.0956000000000001
Step 28000/30000 MLoss: 0.7321 GLoss: 0.3631 Sum: 1.0952
Step 28500/30000 MLoss: 0.6992 GLoss: 0.3626 Sum: 1.0618
Step 29000/30000 MLoss: 0.7026 GLoss: 0.3621 Sum: 1.0647
Step 29500/30000 MLoss: 0.711 GLoss: 0.3618 Sum: 1.0728
Step 30000/30000 MLoss: 0.6822 GLoss: 0.3598 Sum: 1.042
mlp
Sample timestep    0
Sample timestep    0
Sample timestep    0
Sample timestep    0
Sample timestep    0
Sample timestep    0
Discrete cols: [2, 4]
Num shape:  (52000, 7)
Elapsed time: 0:13:52

Adding issue because I spent time with this while this was just simple error.
Trying to run:

python scripts/pipeline.py --config exp/churn2/ddpm_mlp_best/config.toml --train --sample

It is throwing error:

FileNotFoundError: [Errno 2] No such file or directory: 'archive\\y_train.npy'

Where is this data folder?

Hi,

Thanks for your wonderfull paper and code! I have a small question about the evulation process:

In my understanding, tuned_models/ stores the tuned parameters for each evaluation model (CatBoost or MLP) on real data, and when we tune a new synthesis method like Tab-DDPM, we just use tuned parameters to evaluate the synthesized data? Is that right?

If so, I am wondering whether it is possible that for synthesized data, there are better parameters that can reach the better performance on CatBoost or MLP?

Thanks and looking forward to your rely!

Hi team!

I was facing issues in running the code on google colab.

it was not able to read the module inside packages.

Hey guys,

I really appreciate your code and implementation. I am trying to reconstruct the experiments. However, there is a library called 'zero' that is used in the util file that does not have the attributes used. I looked into the documentation and it doesn't exist there either. Do you know if this is a custom library or if it using a previous version? I also looked through the requirements.txt and 'zero' is not present there either. This is where it is used in the code:

lib/util.py:
22. import zero
49. class Timer(zero.Timer):
271. 'gpus': zero.hardware.get_gpus_info(),

Thanks in advance for your time.

Best regards

I am trying to reproduce the provided churn2 dataset, and I am trying to save the generated data from my persepctive the data come from
./scripts/sample.py

Q1. So I was wondering whether the generated dataset is x_gen and y_gen . if it does, whether I just write np.save() code to store the generated data? And why did the output show the synthetic data is loaded from /tmp/tmpmtq5gchu and I can't find the dic on my environment.

Q2. I am not sure how the catboost is beed executed. from the code,

My guess is you use synthetic data to train catboost then evaluate on real testing dataset to see if the model can be trained by synthetic dataset, I am not sure whether this thougt is correct. Please help.

Thanks for reading.

line 79. real_data_path = 'data/higgs-small',

Is this corrext? Shouldn't the data set that is chose be taken from the input arguments?