Rat

“Recent work RAT+, introduces a recurrence-augmented attention backbone that enables flexible dilated attention at inference time.”