MuonBP: Faster Muon via Block-Periodic Orthogonalization
Gradient orthogonalization is a simple strategy that shows great utility in speeding up gradient descent. The Muon optimizer (Jordan, Jin, et al., 2024) combines gradient orthogonalization with first-order momentum and achieves significant improvement in data efficiency over Adam/AdamW (Loshchilov a...
Kaydedildi:
| Asıl Yazarlar: | , , , , |
|---|---|
| Materyal Türü: | Makale |
| Baskı/Yayın Bilgisi: |
2025
|
| Konular: | |
| Online Erişim: | https://arxiv.org/abs/2510.16981 http://arxiv.org/abs/2510.16981 |
| Etiketler: |
Etiket eklenmemiş, İlk siz ekleyin!
|