When synthetic data helps — a study shows the Wasserstein distance predicts augmentation reliability
This paper asks a simple question: when does adding synthetic examples from a generative model actually improve a classifier? The authors develop a theory and run experiments to show that the answer depends less on raw predictive scores and more on how closely the generator matches the true class-conditional distributions. They propose concrete measures to quantify that closeness and show how it controls downstream risk.
On the theory side, the team treats generative augmentation as mixing real and generated data distributions. They prove that the change in classification risk caused by augmentation is controlled by two factors: the augmentation strength (how much synthetic data is mixed in) and the class-conditional Wasserstein discrepancy. The Wasserstein discrepancy is a distance from optimal transport theory that measures how far the generated samples are from the real ones in feature space. The authors also give a finite-sample generalization bound that depends on Rademacher complexity — a standard way to measure hypothesis class capacity — making explicit a trade-off between model complexity, how much augmentation you use, and how faithful the generator is.
To test these ideas they run experiments on both binary and multiclass imbalanced classification tasks. They compare two conditional generative models: Conditional GAN (CGAN) and Conditional WGAN-GP (CWGAN-GP). Across datasets, CWGAN-GP consistently achieves lower Wasserstein discrepancies than CGAN, indicating it produces samples that are closer in distribution to the real class-conditional data. However, the study finds that lower Wasserstein discrepancy does not always translate into better classification accuracy. In many cases, simple classical oversampling methods (for example, replicating or interpolating minority examples) remain competitive.
Why this matters: many people judge synthetic data by downstream accuracy alone. This work argues that such a single number is incomplete. Instead, measuring distributional fidelity with metrics like the Wasserstein distance gives a principled way to assess whether generated data is likely to help. The theoretical bounds further clarify the trade-offs practitioners face: too much augmentation with a poor generator can increase risk, and more powerful classifiers can be either helped or harmed depending on generative fidelity.