A Parameter-Free Wrapper Unlocks Normalization Equivariance for Any Neural Architecture
Wrapped Normalization Equivariance (WNE) is a parameter-free input-output wrapper that enforces normalization equivariance (NE) — the property f(ay + b𝟏) = af(y) + b𝟏 — on arbitrary backbones, including transformers with softmax attention and LayerNorm that prior NE methods cannot accommodate. The authors prove that every NE function admits this normalize-apply-denormalize factorization, making the wrapper an exact parameterization of the full NE function class. Applied to blind image denoising, WNE improves robustness under noise-level mismatch across both CNN and transformer architectures with no measurable GPU overhead, while layer-wise architectural NE baselines incur up to 1.6× runtime cost.