THE VIDEO SHELF · reward modelling and rlhf
Training AI Without Writing A Reward Function, with Reward Modelling
Robert Miles explains reward modelling, in which an AI learns what humans want from their preference comparisons instead of a hand-written reward function. He walks through the 2017 paper on deep reinforcement learning from human preferences.
Why HN readers recommended it
Seen as the clearest introduction to the idea behind RLHF, and also borrowed by commenters as a metaphor for human motivation.
This is a summary from the source dataset, not an endorsement by Hacker News or the video creator.
Video & recommendation details
- Creator / channel
- Robert Miles AI Safety
- Topic
- reward modelling and rlhf
- Format
- explainer
- Level
- intermediate
- HN recommenders in dataset
- 3
- HN mentions in dataset
- 6
- Recommendation years in dataset
- 2019–2023
These counts and years describe the included Hacker News records; they are not YouTube views or the video’s publication date.
From the original HN conversations
Excerpts from the recorded comments. Follow each source for the full comment and surrounding discussion.
I would highly recommend [VIDEO] as a way of framing the problem. It seems like our options for ML are currently: self-play, dataset fitting, or human feedback. It's possible to solve any problem that can fit into one of those molds. (Unfortunately the opposite is …
sillysaurusx on Hacker News · 2019-12
In Autonomous DeLorean drives sideways to move forwardSpecifically about RLHF, I find this video by Rob Miles still the best presentation of the ingenious original 2017(!) paper: [VIDEO] RLHF is actually older than GPT-1, which came out in 2018. It didn't get applied to language models until 2022 with InstructGPT, an approach …
cubefox on Hacker News · 2023-07
In What We Know About LLMs (A Primer)ADHDer here. Please steal some of the motivation-techniques I've learnt over the years: 1. Find a friend who is a software engineer or data analyst and cares about you as a person. Talk about this with them. Humans evolved being able to talk things over …
afarrell on Hacker News · 2020-07
In Ask HN: How do I get out of the procrastination abyss?
Summaries, classifications and recommendation records are from the HackerTube source dataset. Read about this library and its limitations.