This article is a part of a sequence on privacy-preserving federated learning. The sequence is a joint effort by NIST and the UK government’s Responsible Technology Adoption Unit (RTA), previously recognized as the Centre for Data Ethics and Innovation. To explore further and access all the previously published articles, visit NIST’s Privacy Engineering Collaboration Space or RTA’s blog.  Â
In our prior two articles within this series, we delved into strategies for preserving input confidentiality in privacy-preserving federated learning, especially in the realm of horizontally and vertically segmented data. To create a comprehensive privacy-preserving federated learning system, these strategies need to be interwoven with an approach for output security, which restricts the extent to which insights can be derived about individuals in the training data post model training.
As highlighted in the latter part of our article on privacy intrusions in federated learning, trained models have the capacity to unintentionally reveal significant information about their training data – encompassing entire images and textual fragments.
Training utilizing Differential Privacy
The most robust known form of output safeguarding is differential privacy. Differential privacy embodies a defined privacy framework that is applicable in various contexts; for more detailed insights on this topic, check out NIST’s blog series, particularly the article on differentially private machine learning.Â
Approaches for differentially private machine learning inject random noise into the model during training to fortify defenses against privacy breaches. This random noise prevents the model from memorizing intricate details from the training data, thereby ensuring that the training data remains inaccessible from the model in the future. For instance, Carlini et al. demonstrated that sensitive training data like social security numbers could be extracted from trained language models, with training using differential privacy successfully mitigating this vulnerability.
Differential Privacy in Privacy-Protecting Federated Learning
In the context of centralized training where the training data is centralized on a server, the server can execute the training and integrate noise for differential privacy in a single step. In privacy-preserving federated learning, determining the entity responsible for adding the noise and the method of integration can be more intricate.
Credit:
NIST
For privacy-preserving federated learning on horizontally partitioned data, Kairouz et al. introduce a variant of the FedAvg strategy described in our fourth article. In this visualized approach, each participant engages in local training, then appends a small measure of random noise to their model update before consolidating it with the updates from other participants. If each participant incorporates noise accurately, the new consolidated model will entail enough noise to guarantee differential privacy. This methodology ensures output privacy, even in scenarios involving a mistrustful aggregator. The Scarlet Pets team applied a variant of this technique in their successful solution for the UK-US PETs Prize Challenges.
For vertically partitioned data, achieving differential privacy can be quite intricate. The noise essential for differential privacy cannot be integrated before entity alignment as doing so would disrupt the co-alignment of data attributes. Instead, the noise must be incorporated post entity alignment, either by a trustworthy participant or through methodologies like homomorphic encryption or multiparty computation.
Training Exceptionally Accurate Differentially Private Models
The random noise required for differential privacy can impact model accuracy. Generally, more noise results in higher privacy but lower accuracy – this balance between accuracy and privacy is commonly termed the privacy-utility exchange.Â
For specific machine learning models such as linear regression models, logistic regression models, and decision trees, navigating this balance is straightforward – the aforementioned approach often proves effective in training remarkably accurate models with differential privacy. In the UK-US PETs Prize Challenges, both the PPMLHuskies and Scarlet Pets teams leveraged similar techniques to train highly precise models with differential privacy.
For neural networks and deep learning, the model’s sheer magnitude itself complicates training with differential privacy – larger models necessitate more noise to attain privacy, consequently posing a significant accuracy reduction risk. While these types of models weren’t featured in the UK-US PETs Prize Challenges, they are increasingly crucial across all generative AI applications, including vast language models.Â
Recent findings indicate that models pre-trained on publicly accessible data (without differential privacy) and subsequently fine-tuned with differential privacy can achieve nearly identical accuracy levels to models trained sans differential privacy. For instance, Li et al. demonstrate that pre-trained language models fine-tuned with differential privacy can reach nearly equivalent accuracy compared to models devoid of differential privacy during training. These results underscore the plausibility of privacy-preserving federated learning that effectively balances privacy and utility in domains wherein publicly accessible data can be leveraged for pre-training – encompassing language and image recognition models.Â
This methodology doesn’t confer privacy protection for the public data used during pre-training, hence underscoring the importance of ensuring that the utilization of such data aligns with pertinent privacy and intellectual property rights (comprehensive legal and ethical facets concerning this domain are beyond the purview of this article series).
Upcoming Content
In our upcoming article, we’ll delve into the challenges of implementation when deploying privacy-preserving federated learning in practical settings.
About Author
