tensorflowtensorflow-federatedfederated-learning

Federated Averaging and TensorFlow


I am a newbie in federated learning and just getting to know TensorFlow Federated TFF framework. I have some questions in my mind I would be really appreciated it if anybody can clarify them:

  1. Does Federated Averaging algorithm the only aggregation algorithm supported in TFF? and how it differs from Federated Stochastic Gradient Descent?
  2. Dose Federated Averaging require each client to be trained with the Neural Networks? or it is possible for local data to be trained with any machine learning algorithm?
  3. I have big data, and I am planning to partition my data into smaller datasets and simulated each part as one client? does this work in TFF? and does it consider horizontal or vertical federated learning?

Thanks in advance


Solution

  • The Federated Learning for Image Classification tutorial would be a good start to learn TFF: https://www.tensorflow.org/federated/tutorials/federated_learning_for_image_classification The "Preparing the input data" section is related to your third question. About vertical vs. horizontal: I know there are many types of federated learning defined in recent publications. Personally I would call what you described as cross-silo federated learning, see Section 2.2 in this paper for more information: https://arxiv.org/abs/1912.04977

    To answer your other questions:

    See the above tutorial for how to create an iterative_process with federated averaging while setting SGD learning rate on both the server and client side. You can also implement customized federated learning algorithms: https://www.tensorflow.org/federated/tutorials/building_your_own_federated_learning_algorithm (this tutorial might also answer your second question about customized local training?)