FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion
Today data is often scattered among billions of resource-constrained edge devices with security and privacy constraints. Federated Learning (FL) has emerged as a viable solution to learn a global model while keeping data private, but the model complexity of FL is impeded by the computation resources...
Сохранить в:
| Главные авторы: | , , , |
|---|---|
| Формат: | Статья |
| Опубликовано: |
2021
|
| Предметы: | |
| Online-ссылка: | https://arxiv.org/abs/2110.11027 http://arxiv.org/abs/2110.11027 |
| Метки: |
Нет меток, Требуется 1-ая метка записи!
|
| _version_ | 1879173416305557504 |
|---|---|
| author | Cheng, Sijie Wu, Jingwen Xiao, Yanghua Liu, Yang Liu, Yang |
| author_browse | Cheng, Sijie Liu, Yang Wu, Jingwen Xiao, Yanghua |
| author_facet | Cheng, Sijie Wu, Jingwen Xiao, Yanghua Liu, Yang Liu, Yang |
| author_sort | Cheng, Sijie |
| building | arXiv |
| collection | Acceso abierto |
| description | Today data is often scattered among billions of resource-constrained edge devices with security and privacy constraints. Federated Learning (FL) has emerged as a viable solution to learn a global model while keeping data private, but the model complexity of FL is impeded by the computation resources of edge nodes. In this work, we investigate a novel paradigm to take advantage of a powerful server model to break through model capacity in FL. By selectively learning from multiple teacher clients and itself, a server model develops in-depth knowledge and transfers its knowledge back to clients in return to boost their respective performance. Our proposed framework achieves superior performance on both server and client models and provides several advantages in a unified framework, including flexibility for heterogeneous client architectures, robustness to poisoning attacks, and communication efficiency between clients and server on various image classification tasks. |
| first_indexed | 2026-10-15T23:56:59Z |
| format | Article |
| fullrecord | <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ https://www.openarchives.org/OAI/2.0/oai_dc.xsd"><identifier>arxiv-2110.11027</identifier><datestamp>2021-12-08</datestamp>
<dc:title>FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion</dc:title>
<dc:creator>Cheng, Sijie</dc:creator>
<dc:creator>Wu, Jingwen</dc:creator>
<dc:creator>Xiao, Yanghua</dc:creator>
<dc:creator>Liu, Yang</dc:creator>
<dc:creator>Liu, Yang</dc:creator>
<dc:subject>Machine Learning</dc:subject>
<dc:subject>Artificial Intelligence</dc:subject>
<dc:description>Today data is often scattered among billions of resource-constrained edge devices with security and privacy constraints. Federated Learning (FL) has emerged as a viable solution to learn a global model while keeping data private, but the model complexity of FL is impeded by the computation resources of edge nodes. In this work, we investigate a novel paradigm to take advantage of a powerful server model to break through model capacity in FL. By selectively learning from multiple teacher clients and itself, a server model develops in-depth knowledge and transfers its knowledge back to clients in return to boost their respective performance. Our proposed framework achieves superior performance on both server and client models and provides several advantages in a unified framework, including flexibility for heterogeneous client architectures, robustness to poisoning attacks, and communication efficiency between clients and server on various image classification tasks. </dc:description>
<dc:description>Under review as a conference paper at ICLR 2022</dc:description>
<dc:date>2021-10-21</dc:date>
<dc:date>2021-12-07</dc:date>
<dc:type>text</dc:type>
<dc:identifier>https://arxiv.org/abs/2110.11027</dc:identifier>
<dc:identifier>http://arxiv.org/abs/2110.11027</dc:identifier>
</oai_dc:dc> |
| id | arxiv-2110.11027 |
| institution | arXiv |
| last_indexed | 2026-10-15T23:56:59Z |
| publishDate | 2021 |
| publishDateRange | 2021 |
| publishDateSort | 2021 |
| record_format | oai_dc |
| spelling | arxiv-2110.110272021-12-08 FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion Cheng, Sijie Wu, Jingwen Xiao, Yanghua Liu, Yang Liu, Yang Machine Learning Artificial Intelligence Today data is often scattered among billions of resource-constrained edge devices with security and privacy constraints. Federated Learning (FL) has emerged as a viable solution to learn a global model while keeping data private, but the model complexity of FL is impeded by the computation resources of edge nodes. In this work, we investigate a novel paradigm to take advantage of a powerful server model to break through model capacity in FL. By selectively learning from multiple teacher clients and itself, a server model develops in-depth knowledge and transfers its knowledge back to clients in return to boost their respective performance. Our proposed framework achieves superior performance on both server and client models and provides several advantages in a unified framework, including flexibility for heterogeneous client architectures, robustness to poisoning attacks, and communication efficiency between clients and server on various image classification tasks. Under review as a conference paper at ICLR 2022 2021-10-21 2021-12-07 text https://arxiv.org/abs/2110.11027 http://arxiv.org/abs/2110.11027 |
| spellingShingle | Machine Learning Artificial Intelligence Cheng, Sijie Wu, Jingwen Xiao, Yanghua Liu, Yang Liu, Yang FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion |
| title | FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion |
| title_full | FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion |
| title_fullStr | FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion |
| title_full_unstemmed | FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion |
| title_short | FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion |
| title_sort | fedgems federated learning of larger server models via selective knowledge fusion |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2110.11027 http://arxiv.org/abs/2110.11027 |
| work_keys_str_mv | AT chengsijie fedgemsfederatedlearningoflargerservermodelsviaselectiveknowledgefusion AT wujingwen fedgemsfederatedlearningoflargerservermodelsviaselectiveknowledgefusion AT xiaoyanghua fedgemsfederatedlearningoflargerservermodelsviaselectiveknowledgefusion AT liuyang fedgemsfederatedlearningoflargerservermodelsviaselectiveknowledgefusion AT liuyang fedgemsfederatedlearningoflargerservermodelsviaselectiveknowledgefusion |