Video — Deep dive — Better Attention layers for Transformer models

#opensource #datascience #transformers #artificialintelligen

Video — Deep dive — Better Attention layers for Transformer models

The self-attention mechanism is at the core of transformer models. As amazing as it is, it requires a significant amount of computing and memory bandwidth, leading to scalability issues as models get more complex and context length increases.

In this video, we’ll quickly review the computation involved in the self-attention mechanism and its multi-head variant. Then, we’ll discuss newer attention implementations focused on compute and memory optimizations, namely Multi-Query Attention, Group-Query Attention, Sliding Window Attention, Flash Attention v1 and v2, and Paged Attention.

Hands-on debugging session: instrument, monitor, and fix

Join Lazar for a hands-on session where you’ll build it, break it, debug it, and fix it. You’ll set up Sentry, track errors, use Session Replay and Tracing, and leverage some good ol’ AI to find and fix issues fast.

RSVP here →