Rectified Lpjepa Jointembedding Predictive Architectures With Sparse And Maximumentropy Representations

Yann LeCun

paper · 2026-02-01

Recommended

“We introduce Rectified Distribution Matching Regularization (RDMReg), a sliced two-sample distribution-matching loss that aligns representations to a Rectified Generalized Gaussian (RGG) distribution. RGG enables explicit control over expected \ell_0 norm through rectification, while preserving maximum-entropy up to rescaling under expected \ell_p norm constraints. Equipping JEPAs with RDMReg yields Rectified LpJEPA, which strictly generalizes prior Gaussian-based JEPAs.”