THE VIDEO SHELF · language modeling
The spelled-out intro to language modeling: building makemore
Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.
Why HN readers recommended it
HN recommends it as a long but worthwhile way to demystify how language models work technically.
This is a summary from the source dataset, not an endorsement by Hacker News or the video creator.
Video & recommendation details
- Creator / channel
- Andrej Karpathy
- Topic
- language modeling
- Format
- tutorial
- Level
- intro
- HN recommenders in dataset
- 2
- HN mentions in dataset
- 4
- Recommendation years in dataset
- 2024–2024
These counts and years describe the included Hacker News records; they are not YouTube views or the video’s publication date.
From the original HN conversations
Excerpts from the recorded comments. Follow each source for the full comment and surrounding discussion.
I resource I found invaluable in demystifying LLM's was Andrej Karpathy's Youtube series. In particular, his "Makemore" video built a basic LLM in the same vein as OP's. It's long (2h), but well worth the time if anyone is curious about the technical workings of …
islewis on Hacker News · 2024-02
In Markov Chains are the Original Language ModelsYou've made something interesting. Many people have already answered this here [1], so there's no point in me explaining it again. Dropping some pointers that will be helpful for you: - [LINK] - Highly recommend this one [VIDEO] - [LINK] - [LINK] 1: ex: [LINK]
georgehill on Hacker News · 2024-04
In Show HN: Next-token prediction in JavaScript
Summaries, classifications and recommendation records are from the HackerTube source dataset. Read about this library and its limitations.