How One Line of Code Almost Blew Up the Internet
Kevin Fang Kevin Fang
151K subscribers
1,994,196 views
0

 Published On Feb 19, 2023

Sources:
https://blog.cloudflare.com/incident-...
https://blog.cloudflare.com/quantifyi...
https://bugs.chromium.org/p/project-z...
https://asamborski.github.io/cs558_s1...
https://www.colm.net/open-source/ragel/
"[CloudFlare] A Day at the CloudFlare Office"    • [CloudFlare] A Day at the CloudFlare ...  

Assumptions:
- The graph for "email obfuscation" vs. "bug occurrence" at 2:51. This was added to illustrate that the bug was being triggered by this feature. They did not have a convenient graph that told them when the bug was being triggered.
- The "crossroads" mentioned at 3:55 probably did not happen. Just to add drama/plot.
- Explanation of why fhold is called within the finishing action of script_consume_attr is my best guess 7:50
- The history behind the empty last buffer was never explained. But I assume that some existing Module A would originally feed data to the Ragel parser. Module A still existed, and still continued to output this empty last buffer, but now cf-html can stand between Module A and the existing Ragel parser. Here, cf-html would consume Module A's data + the empty last buffer with no issues, but it's output would no longer include the dummy buffer. This output can then be taken in by the Ragel parser.
- Whether or not Cloudflare modified the compiled C code is unknown/never mentioned. There must be a reason that Ragel chooses to use == for the buffer end check rather than ≥, and semantically, == makes more sense if it checks for the buffer end with every iteration, which should make buffer overrunning impossible.
- Technically in the strictest sense this is a "buffer over-read" as opposed to an "overflow" or "overrun" but the Wikipedia page for Cloudbleed says "overflow" so w/e
- Whether or not this bug going unnoticed/discovered by hackers first would've "blown up the internet" is arguable

Error corrections:
- 13:13, the correct number is 0.06% (what is shown), but I say 0.6%
- 13:28, the bug was possible since September (what is shown)

Chapters:
0:00 Exposition/useless story building stuff
0:50 Explanation of Cloudflare and CDNs
1:44 Implications of the bug
2:40 Mitigation timeline
4:46 Root cause
10:43 Lessons learned
12:41 Resolution

Music by LEMMiNO:
Nocturnal -    • LEMMiNO - Nocturnal (BGM)  
Encounters -    • LEMMiNO - Encounters (BGM)  
Cipher -    • LEMMiNO - Cipher (BGM)  

show more

Share/Embed