HackerTube

THE VIDEO SHELF · incident response and postmortems

Debugging Under Fire: Keep your Head when Systems have Lost their Mind • Bryan Cantrill • GOTO 2017

Bryan Cantrill's GOTO Chicago 2017 talk on debugging production systems under pressure, centered on the incident where an operator's mistyped command rebooted an entire Joyent datacenter. It covers human factors, automation and architecture.

Why HN readers recommended it

A perennial HN recommendation for outage and postmortem threads, praised as funny, educational and one of the best postmortems anyone has seen.

This is a summary from the source dataset, not an endorsement by Hacker News or the video creator.

Video & recommendation details

Creator / channel
GOTO Conferences
Topic
incident response and postmortems
Format
talk
Level
intermediate
HN recommenders in dataset
8
HN mentions in dataset
17
Recommendation years in dataset
2018–2025

These counts and years describe the included Hacker News records; they are not YouTube views or the video’s publication date.

From the original HN conversations

Excerpts from the recorded comments. Follow each source for the full comment and surrounding discussion.

  • Perhaps not famous, but Bryan Cantrill, who gives my favorite talks, has an interesting and funny talk on one of the Joyent outages: [VIDEO]

    OmarAssadi on Hacker News · 2022-01
    In Famous outages along with deep postmortems?
  • Time to share one of my favorite talks (and speakers) ever - "Debugging Under Fire: Keep your Head when Systems have Lost their Mind" (Bryan Cantrill, GOTO 2017) [VIDEO]

    int0x2e on Hacker News · 2023-01
    In WAN router IP address change blamed for global Microsoft 365 outage
  • The best postmortem I've seen was actually a conference talk: "Debugging Under Fire: Keep your Head when Systems have Lost their Mind" by Bryan Cantrill (in the GOTO conference, 2017) Here's a YouTube video of it: [VIDEO] Here's the slides: [LINK] It goes into detail …

    KronisLV on Hacker News · 2023-03
    In Ask HN: What is the best postmortem you've seen?

Summaries, classifications and recommendation records are from the HackerTube source dataset. Read about this library and its limitations.