zshuai8 / fedgmm_icml2023 Goto Github PK
View Code? Open in Web Editor NEWPersonalized Federated Learning under Mixture of Distributions
License: MIT License
Personalized Federated Learning under Mixture of Distributions
License: MIT License
If we set n_clusters=3 (n_component=3) when generating non-iid clients, then theoretically how to set n_learner and n_gmm to get best results? is n_learner=3 and n_gmm=3 right?
Please help to understand relations between the n_clusters and the n_learner, n_gmm?
Run FedEM
==> Clients initialization..
===> Building data iterators..
Traceback (most recent call last):
File "run_experiment.py", line 244, in
run_experiment(args)
File "run_experiment.py", line 142, in run_experiment
clients = init_clients(args_,
File "run_experiment.py", line 31, in init_clients
get_loaders(
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/utils/utils.py", line 375, in get_loaders
inputs, targets = get_cifar10(dist_shift, dp)
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/datasets.py", line 1170, in get_cifar10
cifar10_data = d_norm(cifar10_data)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 221, in forward
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 336, in normalize
tensor.sub(mean).div_(std)
RuntimeError: The size of tensor a (32) must match the size of tensor b (3) at non-singleton dimension 1
I train on cifar10 dataset and have this problem.
Dear author, I uncomment lines 42-81 in the run_experiment.py file to get PCA projection file. I modify all_data_tensor = all_data_tensor.view(-1,3,32,32) since I run experiment on cifar10.
# all_data_tensor = []
# for cur_data in train_iterators:
# all_data_tensor.append(cur_data.dataset.data)
# all_data_tensor = torch.cat(all_data_tensor, dim=0)
#
# model = models.resnet18(pretrained=True)
#
# del model.fc
# all_data_tensor = all_data_tensor.view(-1,1,32,32)
# x = all_data_tensor
# if all_data_tensor.shape[1] == 1:
# x = all_data_tensor.repeat(1, 3, 1, 1)
# x = model.conv1(x.float())
# x = model.bn1(x)
# x = model.relu(x)
# x = model.maxpool(x)
#
# x = model.layer1(x)
# x = model.layer2(x)
# x = model.layer3(x)
# x = model.layer4(x)
#
# # Extract the feature maps produced by the encoder
# encoder_output = x.squeeze()
# U, S, V = torch.svd(encoder_output)
# global PCA_V
# PCA_V = V
# print(PCA_V.size())
# with open(f"data/cifar10/all_data/PCA.pkl" , 'wb') as f:
# pickle.dump(PCA_V, f)
# raise
#
#
# encoder_output = encoder_output.view(encoder_output.size(0), -1)
# pca_transformer = PCA(n_components=emb_size)
# # Fit the PCA transformer to your data
#
# X_pca = pca_transformer.fit_transform(encoder_output.detach().numpy())
# # Convert the resulting principal components to a PyTorch tensor
# projected = torch.from_numpy(X_pca).float().cuda()
However, I have new error as follows:
==> Clients initialization..
===> Building data iterators..
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:00<00:00, 456.40it/s]
===> Initializing clients..
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:44<00:00, 1.41s/it]
==> Test Clients initialization..
===> Building data iterators..
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 465.28it/s]
===> Initializing clients..
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:08<00:00, 1.11s/it]
Traceback (most recent call last):
File "run_experiment.py", line 244, in
run_experiment(args)
File "run_experiment.py", line 185, in run_experiment
get_aggregator(
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/utils/utils.py", line 662, in get_aggregator
return ACGCentralizedAggregator(
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/aggregator.py", line 412, in init
super().init(clients,
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/aggregator.py", line 123, in init
self.write_logs()
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/aggregator.py", line 581, in write_logs
self.update_test_clients()
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/aggregator.py", line 434, in update_test_clients
client.update_sample_weights()
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/client.py", line 202, in update_sample_weights
self.samples_weights = self.learners_ensemble.calc_samples_weights(self.val_iterator)
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/learners/learners_ensemble.py", line 805, in calc_samples_weights
all_losses = self.gather_losses(iterator).T # n * m2
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/learners/learners_ensemble.py", line 872, in gather_losses
all_losses[learner_id] = learner.gather_losses(iterator)
File "/mnt/traffic1/data/jxt/Personalized_Federated_Learning/FedGMM/learners/learner.py", line 248, in gather_losses
y_pred = self.model(x)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torchvision/models/mobilenetv2.py", line 198, in forward
return self._forward_impl(x)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torchvision/models/mobilenetv2.py", line 191, in _forward_impl
x = self.features(x)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/container.py", line 119, in forward
input = module(input)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/container.py", line 119, in forward
input = module(input)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 399, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/mnt/traffic/home/jxt/miniconda3/envs/FedGMM32/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 395, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [32, 3, 3, 3], expected input[128, 32, 32, 3] to have 3 channels, but got 32 channels instead
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.