CS480/680 Lecture 19: Attention and Transformer Networks

Why HN readers recommended it

HN recommends it to newcomers as a clear explanation of attention, including the query/key/value intuition.

This is a summary from the source dataset, not an endorsement by Hacker News or the video creator.

Video & recommendation details

Creator / channel

Pascal Poupart

Topic

transformers and attention

Format

lecture

Level

intermediate

HN recommenders in dataset

3

HN mentions in dataset

4

Recommendation years in dataset

2023–2023

These counts and years describe the included Hacker News records; they are not YouTube views or the video’s publication date.

From the original HN conversations

Excerpts from the recorded comments. Follow each source for the full comment and surrounding discussion.

• I would also recommend Pascal Poupart's talk [1] on Attention and Transformers. Doesn't seem to be cited often, but its quite well done and seeing him work out some of the details using chalk and board is very reassuring. [1] [VIDEO]

• This might be dumbing this down very very much but it all boils down to having some sort of a special lookup table and you give it a "query" and a bunch of "keys" and the "value" is the most likely next word. As you …

• What made attention mechanism click to me was the video [LINK] by Sebastian Raschka. Another good video on attention that tries to give a bit more intuition on the query/key/value stuff is [VIDEO] In hindsight, what helped me most is understanding the dimensionality at each …