zshuai8 / fedgmm_icml2023 Goto Github PK

If we set n_clusters=3 (n_component=3) when generating non-iid clients, then theoretically how to set n_learner and n_gmm to get best results? is n_learner=3 and n_gmm=3 right?
Please help to understand relations between the n_clusters and the n_learner, n_gmm?

Run FedEM
==> Clients initialization..
===> Building data iterators..
Traceback (most recent call last):
File "run_experiment.py", line 244, in
run_experiment(args)
File "run_experiment.py", line 142, in run_experiment
clients = init_clients(args_,
File "run_experiment.py", line 31, in init_clients
get_loaders(
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/utils/utils.py", line 375, in get_loaders
inputs, targets = get_cifar10(dist_shift, dp)
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/datasets.py", line 1170, in get_cifar10
cifar10_data = d_norm(cifar10_data)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 221, in forward
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 336, in normalize
tensor.sub(mean).div_(std)
RuntimeError: The size of tensor a (32) must match the size of tensor b (3) at non-singleton dimension 1

I train on cifar10 dataset and have this problem.

Dear author, I uncomment lines 42-81 in the run_experiment.py file to get PCA projection file. I modify all_data_tensor = all_data_tensor.view(-1,3,32,32) since I run experiment on cifar10.

# all_data_tensor = []
# for cur_data in train_iterators:
#     all_data_tensor.append(cur_data.dataset.data)
# all_data_tensor = torch.cat(all_data_tensor, dim=0)
#
# model = models.resnet18(pretrained=True)
#
# del model.fc
# all_data_tensor = all_data_tensor.view(-1,1,32,32)
# x = all_data_tensor
# if all_data_tensor.shape[1] == 1:
#     x = all_data_tensor.repeat(1, 3, 1, 1)
# x = model.conv1(x.float())
# x = model.bn1(x)
# x = model.relu(x)
# x = model.maxpool(x)
#
# x = model.layer1(x)
# x = model.layer2(x)
# x = model.layer3(x)
# x = model.layer4(x)
#
# # Extract the feature maps produced by the encoder
# encoder_output = x.squeeze()
# U, S, V = torch.svd(encoder_output)
# global PCA_V
# PCA_V = V
# print(PCA_V.size())
# with open(f"data/cifar10/all_data/PCA.pkl" , 'wb') as f:
#     pickle.dump(PCA_V, f)
# raise
#
#
# encoder_output = encoder_output.view(encoder_output.size(0), -1)
# pca_transformer = PCA(n_components=emb_size)
# # Fit the PCA transformer to your data
#
# X_pca = pca_transformer.fit_transform(encoder_output.detach().numpy())
# # Convert the resulting principal components to a PyTorch tensor
# projected = torch.from_numpy(X_pca).float().cuda()

However, I have new error as follows:

==> Clients initialization..
===> Building data iterators..
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:00<00:00, 456.40it/s]
===> Initializing clients..
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:44<00:00, 1.41s/it]
==> Test Clients initialization..
===> Building data iterators..
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 465.28it/s]
===> Initializing clients..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:08<00:00, 1.11s/it]
Traceback (most recent call last):
File "run_experiment.py", line 244, in
run_experiment(args)
File "run_experiment.py", line 185, in run_experiment
get_aggregator(
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/utils/utils.py", line 662, in get_aggregator
return ACGCentralizedAggregator(
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/aggregator.py", line 412, in init
super().init(clients,
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/aggregator.py", line 123, in init
self.write_logs()
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/aggregator.py", line 581, in write_logs
self.update_test_clients()
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/aggregator.py", line 434, in update_test_clients
client.update_sample_weights()
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/client.py", line 202, in update_sample_weights
self.samples_weights = self.learners_ensemble.calc_samples_weights(self.val_iterator)
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/learners/learners_ensemble.py", line 805, in calc_samples_weights
all_losses = self.gather_losses(iterator).T # n * m2
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/learners/learners_ensemble.py", line 872, in gather_losses
all_losses[learner_id] = learner.gather_losses(iterator)
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/learners/learner.py", line 248, in gather_losses
y_pred = self.model(x)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torchvision/models/mobilenetv2.py", line 198, in forward
return self._forward_impl(x)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torchvision/models/mobilenetv2.py", line 191, in _forward_impl
x = self.features(x)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/container.py", line 119, in forward
input = module(input)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/container.py", line 119, in forward
input = module(input)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 399, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 395, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [32, 3, 3, 3], expected input[128, 32, 32, 3] to have 3 channels, but got 32 channels instead