HackerTube

THE VIDEO SHELF · gpt training pipeline

State of GPT | BRK216HFS

Andrej Karpathy walks through how GPT assistants are trained, from tokenization and pretraining to supervised finetuning and RLHF, and gives practical prompting strategies and a view of the tooling ecosystem.

Why HN readers recommended it

Recommended as the clearest overview of how LLM assistants are built and how to use them well.

This is a summary from the source dataset, not an endorsement by Hacker News or the video creator.

Video & recommendation details

Creator / channel
Microsoft Developer
Topic
gpt training pipeline
Format
talk
Level
intro
HN recommenders in dataset
6
HN mentions in dataset
11
Recommendation years in dataset
2023–2025

These counts and years describe the included Hacker News records; they are not YouTube views or the video’s publication date.

From the original HN conversations

Excerpts from the recorded comments. Follow each source for the full comment and surrounding discussion.

  • I would highly recommend watching - State of GPT video by Karpathy at Microsoft Build to get an idea of how LLM’s work. There are some really good prompting ideas. It changed the way I use GPT-4. Video: [VIDEO]

    sagarpatil on Hacker News · 2023-06
    In GPT Best Practices
  • I share many of your concerns and frustrations, although I suspect what you're asking for their consider a moat along the lines of a trade secret, rivaled only by the collection of performance improvement techniques they've amassed in 1000s-10,000s of training runs, 100s of engineers, …

    mcint on Hacker News · 2023-06
    In It’s infuriatingly hard to understand how closed models train on their input
  • I recognize there's plenty of catnip here when it comes to calling this "engineering" or not, however, whatever you want to call it (prompt fiddling?), the techniques are crucial if you want to achieve reasonably consistent output from current-state LLMs. As models improve concerns about …

    valgaze on Hacker News · 2023-06
    In A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

Summaries, classifications and recommendation records are from the HackerTube source dataset. Read about this library and its limitations.