State-of-the-Art DP Text Synthesis Fails to Transfer Corpus Knowledge, Even at High Privacy Budgets
ContinuousBench is a new benchmark designed to rigorously test whether differentially private (DP) synthetic text actually transfers novel knowledge from sensitive corpora to trained models. Unlike existing benchmarks — which are nearly solvable without any corpus access — ContinuousBench generates quarterly QA tasks that are both unsolvable without the training corpus and verifiably learnable under DP constraints. Empirical results show that while non-private synthesis successfully transmits substantial corpus knowledge, state-of-the-art DP synthesis methods consistently fail to do so, even at the very lenient privacy budget of ε=100.