New tutorial on [link] on Focal Modulation Networks: [link] Focal Modulation is a 1:1 replacement for Self-Attention, that has the considerable advantage of being highly interpretable. A must read :)