Safeguarding Trained Models in Privacy-Preserving Federated Learning

The last duo of articles within our series explored methods for securing input privacy in privacy-ensuring federated learning within the realm of horizontally and vertically divided data. To construct an all-inclusive privacy-protecting federated learning system, these methods necessitate integration with a strategy for output privacy, which restricts the extent to which insights about individuals in the training data can be extracted post-training.

Expounded upon in the second segment of our discourse regarding privacy invasions in federated learning, educated models possess the potential to inadvertently disclose substantial details from their training data—including entire images and text excerpts.

Instruction with Differential Privacy

The most potent form of output privacy hitherto identified is differential privacy. Differential privacy serves as a structured privacy model pertinent across various contexts; for additional insights, refer to NIST’s array of blogs on this subject for more elaborate information and, specifically, the articulation on privately differential machine learning in the disquisition accessible through this link. 

Methods for imparting differential privacy to machine learning involve the introduction of random noise to the model during the training process to secure against breaches of privacy. This incidental noise prevents the model from memorizing specifics of the training data, ensuring that the original training data remains indiscernible from the model. As an illustration, research by Carlini et al. demonstrated that sensitive training data such as social security numbers could be extrapolated from trained linguistic models, with training supported by differential privacy successfully inhibiting these attacks.

Differential Privacy for Privacy-Preserving Federated Learning

In consolidated training scenarios, where the training data is amassed on a central server, the server holds the capability to undertake the training process and infuse noise for differential privacy in a solitary operation. Conversely, in privacy-centered federated learning, discerning the entity responsible for incorporating the noise and the manner in which it should be infused can pose a more arduous challenge.

In the context of privacy-conserving federated learning on horizontally divided data, Kairouz et al. introduce a variation of the FedAvg methodology, as outlined in our fourth article. Under this strategy, visualized, each participant conducts localized training, followed by the inclusion of a slight measure of arbitrary noise to their model update prior to its aggregation with other participants’ updates. In the event each participant proficiently incorporates noise within their update, the newly aggregated model will possess adequate noise to ensure differential privacy. This approach delivers output privacy, even in cases involving a deceitful aggregator. This approach was utilized by the Scarlet Pets group in their triumphant endeavor for the UK-US PETs Prize Challenges.

In instances dealing with vertically segregated data, guaranteeing differential privacy can prove intricate. The requisite noise for differential privacy must not be introduced prior to entity alignment, as it would obstruct the correct alignment of data attributes. Instead, noise must be incorporated post-entity alignment, conducted by a reliable participant or utilizing methodologies such as homomorphic encryption or multiparty computation.

Training Exceptionally Accurate Differentially Private Models

The arbitrary noise needed for differential privacy has the potential to impact model precision. Greater noise typically augments privacy protection but depreciates accuracy. This compromise between accuracy and privacy is often referred to as the privacy-utility tradeoff. 

For certain types of machine learning models, including linear regression models, logistic regression models, and decision trees, navigating this tradeoff is relatively straightforward – the above-mentioned approach generally suffices to train remarkably precise models with differential privacy. During the UK-US PETs Prize Challenges, both the PPMLHuskies and Scarlet Pets teams leveraged akin techniques to train exceedingly accurate models with differential privacy.

For neural networks and deep learning frameworks, the considerable size of the model exacerbates the process of training with differential privacy – larger models necessitate additional noise to attain privacy, which can substantially diminish accuracy. While such models were not an element of the UK-US PETs Prize Challenges, their relevance is escalating across all iterations of generative AI, encompassing grandiose language models. 

Recent findings have indicated that models pre-conditioned on publicly accessible data (sans differential privacy) and subsequently tweaked with differential privacy can achieve near-identical accuracy levels as models trained without differential privacy. For example, Li et al. exhibit that pre-conditioned linguistic models can undergo fine-tuning with differential privacy while retaining almost analogous accuracy levels as models trained without differential privacy. These results imply that within domains accommodating publicly accessible data for pre-conditioning—comprising language and image recognition models—privacy-ensuring federated learning capable of uniting privacy and efficacy is viable. 

This methodology does not furnish privacy safeguards for the public data utilized during pre-conditioning, stressing the significance of ensuring that the utilization of this data aligns with pertinent privacy and intellectual property rights (considerations regarding the legal and ethical facets of this are out of the scope of this blog series).

Up Next

In the forthcoming publication, we will delve into the challenges encountered when applying privacy-oriented federated learning in practical scenarios.

About Author

AndyC

Andy Curtis is an award-winning security consultant, researcher and public speaker. He has been working in the computer security industry since the early 1990s, having been employed by state and federal government, leading healthcare and banking providers across three continents. He has given talks about computer security for some of the world’s largest companies, worked with law enforcement agencies on investigations into hacking groups, and is a regular voice on TV and radio explaining IT security threats.

See author's posts