HackerTube

THE VIDEO SHELF · transformers and language models

Let's build GPT: from scratch, in code, spelled out.

Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.

Watch & explore the libraryYouTube ↗

Why HN readers recommended it

It is the go-to recommendation for learning how transformers and LLMs actually work, praised for its clarity and Karpathy's teaching.

This is a summary from the source dataset, not an endorsement by Hacker News or the video creator.

Video & recommendation details

Creator / channel
Andrej Karpathy
Topic
transformers and language models
Format
tutorial
Level
intermediate
HN recommenders in dataset
14
HN mentions in dataset
20
Recommendation years in dataset
2023–2025

These counts and years describe the included Hacker News records; they are not YouTube views or the video’s publication date.

From the original HN conversations

Excerpts from the recorded comments. Follow each source for the full comment and surrounding discussion.

  • Great compilation, would be great to see Vision Transformer (ViT) included. Andrej Karpathy's GPT video is a must have companion for this [VIDEO] I was going nuts trying to grok Key, Query,Position and Value until Andrej broke it down for me.

    yonz on Hacker News · 2023-01
    In The Transformer Family

  • Very impressive. Recently I watched this really amazing lecture on building GPT from scratch from Karpathy, I was blown away: [VIDEO]

    durdn on Hacker News · 2023-02
    In GPT in 60 Lines of NumPy

  • Andrej Karpathy, the director of AI at tesla, published an AMAZING video about how the current models are built which I think everyone in the tech space should watch. It is intuitive, easy to follow and quite frankly the best video I have seen on …

    wesammikhail on Hacker News · 2023-05
    In Google DeepMind CEO Says Some Form of AGI Possible in a Few Years

Summaries, classifications and recommendation records are from the HackerTube source dataset. Read about this library and its limitations.

Keep exploring