When the lab breaks

Every outage worth writing down: what broke, what actually caused it once the logs were read properly, and what changed so it doesn’t happen the same way twice. Published in full, including the ones where the cause turned out to be me.