antreasantoniou / dagan

DAGAN: Data Augmentation Generative Adversarial Networks

Antreas Antoniou

I am a Machine Learning Research Associate at the University of Edinburgh, supervised by Prof. Amos Storkey. I am a member of the BayesWatch research group and the Adaptive and Neural Computation (ANC) research institute.

Leading my research interests is Multi-Modal Learning, specifically targeting the synergistic integration of text, images, audio, and video data. This is followed by the development of Self-Supervised Methods, inspired by mechanisms of infant learning and principles of evolutionary computation.

Additional key areas include Meta-Learning, Adversarial Learning, and Optimization Techniques Inspired by Evolutionary Optimization. These are applied across both differentiable and gradient-free optimization paradigms. Other relevant research dimensions include Inductive Biases, Scalability, Computational Efficiency, and Memory-Augmented Neural Networks.

For more information see my website: https://antreas.io/home/

dagan's Issues

ValueError: No variables to optimize

I'm trying to run the command given in the readme to train the DAGAN. At first, I had to fix some compatibility issues between tensorflow versions 1.x and 2.9.1 such as tf_slim instead of tf.contrib and using keras layers. It compiles now, but during runtime, I get the error "No variables to optimize" in the train() method of dagan_networks_wgan.py on the following line:
opt_ops["g_opt_op"] = opts["g_opt"].minimize(losses["g_losses"],
var_list=self.g.variables,
colocate_gradients_with_ops=True)

It seems like there are no variables in the graph during the first time the train function is called? Has anyone gotten this to work in recent tensorflow versions?

BUG report: change height and width in architecture

Hi,

When I use my own dataset whose width and height is not the same. I noticed that there is a small mistake in the codes.

Because the data you used has the same width and height, so it's ok for your current version. But when data with different height and width is used, it can not work.

In file "dagan_architectures.py",

line 270, it should be
w = concat_shape[len(encoder_layers) - 1 - i][2]

line 318, it should be
w_size=upscale_shape[2],

line 319, it shoule be
h_size=upscale_shape[1]

line 355, it should be
dim_upscale=True, local_inner_layers=decoder_inner_layers, w_size=upscale_shape[2],

line 336, it shoule be
h_size=upscale_shape[1], dropout_rate=dropout_rate)

Question about wgan-gp computation

Hi, Antreas,

I have a question regarding wgan-gp computation.
In your code,
slopes = tf.sqrt(tf.reduce_sum(tf.square(gradients), reduction_indices=[1])).
Here, "reduction_indices=[1]" leads to the shape of (32, 28, 1) not (32, 1) when Omniglot training.
Is it correct?

Thanks,
Sungwoong.

augment new images

Hi,

I have 224x224 images belong different classes. To augment my image set should I convert these to .npy format and put them datasets folder and run the code? Is it enough?

Regards.

AC-GAN like Discriminator

The synthesized results of VGG-Face dataset seem to contain many other identities.
Have you tried AC-GAN like discriminator to avoid it? Or Is it not suitable for one-shot generation task?

can you give your dataset?

Hello, thanks for a good repo.. I need your dataset to see the results.. can you provide your dataset?

how to download the data, the author connection can not be opened,

hello，I would like to ask, how to download the data, the author connection can not be opened, can you share it?

how to use emnist

just as the title shows, how to use emnist to train a model

OSError

Thanks for this interesting repository. I tried it using the instructions but got the error below:
OSError: Failed to interpret file 'datasets/omniglot_data.npy' as a pickle

Do we need resampling of noise variable for updating generator parameters ?

Normally while training W-GAN we iteratively update the discriminator parameters and generator parameters sequentially. Before updating the generator parameters, we resample the noise variable again as shown in Page 8 of https://arxiv.org/pdf/1701.07875.pdf. But in your code in Lines 165-168 of https://github.com/AntreasAntoniou/DAGAN/blob/master/dagan_networks_wgan.py you use the same noise variables for generator and discriminator update. Is there any particular reason for doing so or both works ?

training time

Thank you very much for sharing your code. I am learning your method, but can you tell me how long it takes to run and train two data sets? I'm not sure if there was an error during the run, but it has taken a lot of time.
look forward to your reply.

Any suggestion of generate 128*128 RGB images?

I'm trying to use this network to generate 128*128 images by current setting,
python train_omniglot_dagan.py --batch_size 32 --generator_inner_layers 3 --discriminator_inner_layers 5 --num_generations 64 --experiment_title omniglot_dagan_experiment_default --num_of_gpus 1 --z_dim 100 --dropout_rate_value 0.5
but the result never get better during 12 hours training.
Is there any suggestion about the parameter setting?
For example, generator_layers, discriminator_layers, generator_inner_layers, discriminator_inner_layers, etc.

can I use DAGAN for binary classes

Hello,
I really need your help, it will be very much appreciated as I can't solve it, unfortunately.
I have a training dataset of classes 1 and 0. 6000x50x50x3 per each class. I want to generate more data for both classes. Can I do it using DAGAN? If yes I was wondering how to define this: " Then you need to choose which classes go to each of your training, validation and test sets".

In your code snippet from load_dataset will this be correct?
x_train, x_test, x_val = self.x[:1], self.x[1:2], self.x[1:2]
x_train = x_train[:gan_training_index]
return x_train, x_test, x_val
I guess here synthetic images for only one class will be generated, what should I do to generate for another class as well?

Where is the .ckpt files?

I tried to replay the code with the given dataset and without any code modification, but it does not work at all.

I downloaded the 'vgg_face_data.npy' from here(https://drive.google.com/drive/folders/15x2C11OrNeKLMzBDHrv8NPOwyre6H3O5) and put it in /datasets.

(Command)
python train_face_dagan.py --batch_size 32 --generator_inner_layers 3 --discriminator_inner_layers 5 --num_generations 64 --num_of_gpus 8 --z_dim 100 --dropout_rate_value 0.5 --continue_from_epoch 38 > output.txt

(Error)
File "train_face_dagan.py", line 12, in
experiment.run_experiment()
File "experiment_builder.py", line 101, in run_experiment
ignore_missing_vars=True)
File "lib/python3.7/site-packages/tensorflow_core/contrib/framework/python/ops/variables.py", line 742, in assign_from_checkpoint_fn
reader = pywrap_tensorflow.NewCheckpointReader(model_path)
File "lib/python3.7/site-packages/tensorflow_core/python/pywrap_tensorflow_internal.py", line 885, in init
this = _pywrap_tensorflow_internal.new_CheckpointREader(filename)

tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceREader constructor: Failed to find any matching files for omniglot/dagan/experiment/saved_models/train_saved_model_omniglot_dagan_experiment_38.ckpt

tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceREader constructor: Failed to find any matching files for omniglot/dagan/experiment/saved_models/train_saved_model_omniglot_dagan_experiment_38.ckpt
tensorflow.python.gramework.errors_impl.NotFoundError:: command not found

The directory "omniglot/dagan/experiment/saved_models", was made by "build_experiment_folder(self.experiment_name)", so it exists.
However, I cannot find the code that makes "train_saved_model_omniglot_dagan_experiment_38.ckpt".

Do I need to add "train_saved_model_omniglot_dagan_experiment_38.ckpt" additionaly?
If not, why this code is not working?

I do not changed anything, I just did git clone this repository.

And I am totally newcomer for tensorflow.

Cross Domain Dataset

I'm wondering if this method can be applied to datasets containing different domains such as dataset of ImageNet, which we have cats, flowers, dog, etc?

How to make *.npy file?

Images from "vgg_face_data" have so many different sizes.

For example,

https://www.contactmusic.com/pics/ld/active_for_life_arrivals_090110/a.j_buckley_2706152.jpg is 500x750

http://www.kino-teatr.ru/acter/album/73932/50682.jpg is 400x600

and so on.

'vgg_face_data.npy' is reshaped into (2354, 100, 64, 64, 3).

It might be, 64x64 images with 3 channels RGB.

2354 means number of classes and 100 stands number of samples.

But the size of vgg face data set is various, all the images would be reshaped into 64x64?

Question about experiment setting in the paper

Hi Antoniou !

I cannot understand the following statement in your paper.

The real or fake label was also passed to the network

It means that adding one extra class "fake" to final layer of CNN?

Thank you!

ResourceExhaustedError when using a new dataset with only 3000 images

I try to use a new dataset to train DAGAN. The commend is this:
nohup python train_face_dagan.py --batch_size 8 --generator_inner_layers 3 --discriminator_inner_layers 5 --num_generations 64
--experiment_title pubfig_dagan_experiment_default --num_of_gpus 3 --z_dim 5 --dropout_rate_value 0.5 >dagan_face_log.txt 2>&1 &

I tried different batch size(32, 16, 8), and the ResourceExhaustedError always exists. May I ask the resourse requirement of this code? I have 3 GTX TITANX GPUs.

Is the purpose of this code deliberately creating an unbalanced data set?

Is the purpose of this code deliberately creating an unbalanced data set?

Quoted from data.OmniglotImbalancedDAGANDataset.load_dataset

for i in range(x.shape[0]):
    choose_samples = np.random.choice([i for i in range(1, 15)])
    x_temp.append(x[i, :choose_samples])
self.x = np.array(x_temp)

This means that I don't need to perform this weird step when actually loading an unbalanced data set. Is my understanding correct?

Thanks for your help and code!

Can I get the model that you trained by the default paramaters.

Hello,can I get the model that you trained by the default paramaters, cause my devices is not in good condition if i train by myslef it cost nearly about 200 days. Thank you very much.

Issues met when training DAGAN for new dataset

Hi, I am trying to train the DAGAN for a new dataset, which has been transformed into the standard numpy format [10, 28, 64, 64, 3] (10 people and 28 pictures for each)
I think this dataset is small enough. However, after I trained the model for over 200 epochs the loss of generator never converge. And images generated are not as expected. In the generated 16 by 16 images result, except for original images on the left top, all other images are the same, as shown below. I think the network didn't converge.

And this is the loss I got.

Could you please give me some suggestion/tricks on training the model? I will really appreciate!

DenseNet classifier

Thank you for your impressive work!
I want to replicate classification experiment with my own dataset.

1. I know the detail is well explained in the paper, but it's very helpful if you could tell me which DenseNet repository you used for classification experiment.
2. Plus, did you normalize generated data to 0-1 before train classifier?

Thank you in advance!

how many times it will cost to train the model

Feeding in Tabular/vectorized data?

Interested in using this to augment some tabular (well, tabular turned vectorized) data. Is there an established way to feed this in?

Performance on Vanilla GAN ?

Did you trying using the Vanilla GAN before using W-GAN ? I tried your approach using Vanilla GAN on toy 2D data and it does not work and wanted to ask whether you faced similar problems.

Creating new DAGAN architecture.

I'm trying to implement a new architecture for a DAGAN i would like to use from the paper below.

https://pdfs.semanticscholar.org/e681/f8172dafea9a7c147859d73dc23fd40ab634.pdf

I'm ok working with keras and pytorch although i don't typically use tensorflow directly. Could you point me in the right direction of any libraries or other repos you know that might help?

Issues met when generating data

Hi, I finished training with omniglot and am trying to generate data, but Error happen.
Error code is shown below. I think that using tensorflow-gpu latest version is the reason ,but I could not modify your code.

Traceback (most recent call last):
File "gen_omniglot_dagan.py", line 24, in
experiment.run_experiment()
File "/ ////generation_builder.py", line 89, in run_experiment
ignore_missing_vars=True)
File "///////python3.6/site-packages/tensorflow/contrib/framework/python/ops/variables.py", line 674, in assign_from_checkpoint_fn
reader = pywrap_tensorflow.NewCheckpointReader(model_path)
File "///////python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 290, in NewCheckpointReader
return CheckpointReader(compat.as_bytes(filepattern), status)
File "///////python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 519, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for omniglot/dagan/experiment/default/saved_models//omniglot_dagan_experiment_default_4.ckpt

how to make the environment

I see the requirements in your file,but I want to know if I am not make the envs like that, will I run your code success fully,because I can not find the version that too earlier, and if I install the version you given ,that would remained me that there some conflict way in your installation.wish you could help me.

What is the role of Linear Projection?

Hi, Antoniou!

What is the role of Linear Projection form z to representation space?
I wonder there is a problem if we just put Gaussian noise on representation space.

preparing dataset for training

can you please tell me how to prepare my own dataset? Is just making the .npy file of my data enough?

vgg_face_data.npy dtype

Thank you for sharing your code. But when I tried to load the file 'vgg_face_data.npy', I got the error below:
ValueError: cannot reshape array of size 1073479997 into shape (2354,100,64,64,3)

Could you tell me the the np.dtype when you packed the VGG-Face datasets? Look forward to your kind reply.