Why Would AI Want to do Bad Things? Instrumental Convergence

Why HN readers recommended it

HN commenters repeatedly recommend it as the clearest introduction to why smart AI might behave badly by default.

This is a summary from the source dataset, not an endorsement by Hacker News or the video creator.

Video & recommendation details

Creator / channel

Robert Miles AI Safety

Topic

ai safety instrumental convergence

Format

explainer

Level

intro

HN recommenders in dataset

6

HN mentions in dataset

8

Recommendation years in dataset

2019–2023

These counts and years describe the included Hacker News records; they are not YouTube views or the video’s publication date.

From the original HN conversations

Excerpts from the recorded comments. Follow each source for the full comment and surrounding discussion.

• ...and many would say that’s because us humans are bad at imagining optimizing agents without anthropomorphizing them. This is a reasonable, even typical suspicion that many people share! The best explanation I know of why it’s unfortunately wrong is by Robert Miles in a video, …

• The basic argument is trivial: it is plausible that future systems achieve superhuman capability; capable systems necessarily have instrumental goals; instrumental goals tend to converge; human preferences are unlikely to be preserved when other goals are heavily selected for unless intentionally preserved; we don't know …

• The paperclip maximizer is an extreme thought experiment to make a point about the the orthogonality thesis. If you think anyone actually takes it seriously as a real-world thing you never understood the example. It is used to guide discussion. The trolley problem is similarly …