DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning
Alex Stein
Abstract
Differentially private synthetic data generation is important when private datasets cannot be inspected directly, but existing approaches often trade off privacy boundaries against domain fidelity. This work introduces Differentially Private Reinforcement Fine-Tuning (DP-RFT), an online reinforcement learning method for training LLMs to generate synthetic text without direct eyes-on access to individual private examples. DP-RFT uses differentially private nearest-neighbor votes from an eyes-off private corpus as the reward signal for on-policy synthetic generations, then optimizes the model with PPO. The paper evaluates DP-RFT on long-form and domain-specific generation tasks including news, meetings, and medical abstracts, showing improved fidelity and downstream utility while preserving the private data boundary.
BibTeX
@misc{xu2026dprftlearninggeneratesynthetic,
title={DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning},
author={Fangyuan Xu and Sihao Chen and Zinan Lin and Taiwei Shi and Sydney Graham and Pei Zhou and Mengting Wan and Alex Stein and Virginia Estellers and Charles Chen and Morris Sharp and Richard Speyer and Tadas Baltrusaitis and Jennifer Neville and Eunsol Choi and Longqi Yang},
year={2026},
eprint={2602.18633},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2602.18633}
}