jm + rollback + regexps   1

Details of the Cloudflare outage on July 2, 2019
Great writeup from jgc. Worth noting some important lessons:

* config changes should be rolled out carefully and gradually, just like code;

* particularly regexps, which are effectively code anyway;

* emergency-use rollback systems need to work, of course!;

* having emergency-only systems is a risk, too, since infrequently-used code paths are likely to atrophy and break without anyone noticing (as nsheridan said);

* /.*/ in a regexp is pretty much always bad news, and would have been worth a linter to catch before commit.
cloudflare  outages  regex  postmortems  regexps  deployment  rollback  via:jgc 
9 weeks ago by jm

Copy this bookmark:



description:


tags: