In LLM, where is the adapter for clients and servers to interact? about federatedscope

[rayrayraykk]

Hi,

Thanks for reaching out. The adapter's integration is detailed in the adapter module of the codebase.

FederatedScope/federatedscope/llm/model/adapter_builder.py

Line 147 in 7f08694

class AdapterModel(nn.Module):

Traditional FL is sending the full model, and adapter is used to fine-tune a small portion of the LLM's parameters.
For full definition and details, you can refer to our paper https://arxiv.org/abs/2309.00363 . And let me know if you have any further questions.

Best regards,

[haoquan24]

Hi,

Thanks for your time. I have another question. In the yaml file, what does "local_update_steps" represent?

Is the amount of data used for each client training "local_update_steps * batch_size" instead of all?

If not, what data does each client use in one round of training?

Best wishs.

[rayrayraykk]

When cfg.train.batch_or_epoch = 'batch', the amount of data used for each client training is "local_update_steps * batch_size". And When cfg.train.batch_or_epoch = 'epoch', the data used are "local_update_steps * num_data".

In federatedscope/core/configs/cfg_training.py:

    cfg.train.local_update_steps = 1
    cfg.train.batch_or_epoch = 'batch'