FedMLB about flower

[alessiomora]

Hi @jafermarq. I confirm I am based in a territory among the ones listed above (Italy).

[alessiomora]

Hi. I would like to provide the implementation of this paper. I would like to firstly focus on reproducing the results for CIFAR-100 (with distribution-based label skew, Dirichlet concentration param $\alpha \in[0.3, 0.6]$) with ResNet-18 as model architecture.

[jafermarq]

Hey @alessiomora , great to hear you want to reproduce FedMLB.
Could you point us to the specific table/plot for those experiments you mention? is this for the 100 client or 500client setup? also, what point of reference would you include for FedMLB? (i.e. what baseline in the FedMLB paper) -- we have some info about this in the first point in our FAQ: https://flower.dev/summer/#faq

[alessiomora]

Hi @jafermarq. I will start by reproducing the left part of Table 1 in the paper, that is the results for CIFAR-100 with (a) Moderate-scale with Dir(0.3), 100 clients, 5% participation and (b) Large-scale with Dir(0.3), 500 clients, 2% participation, for FedAvg and FedMLB. Those results are also plotted in Fig. 2.

Then, I will reproduce the left part of Table 10 (appendix) in the paper, that is the results for CIFAR-100 with (a) Moderate-scale with Dir(0.6), 100 clients, 5% participation and (b) Large-scale with Dir(0.6), 500 clients, 2% participation, for FedAvg and FedMLB. Those results are also plotted in Fig. 6 (c) and 6 (d), in appendix.

I can also reproduce Table 2 (for FedAvg, FedMLB and FedAvg+KD). Those results are also plotted in Fig. 3.

Then, I am planning to reproduce the results with Tiny-Imagenet as well (the right part of Table 1). I will provide details about this later on.

[jafermarq]

Hi @alessiomora This sounds perfect! I've ticked some of the check list above in the issue. Please take a look at the info regarding the point in step 2 about reward eligibility. If all looks good, feel free to ✅ it.

[jafermarq]

Many thanks for confirming! Everything in step 1 and 2 is completed. Looking forward to see FedMLB in action 🙌.