Cloudbleed

From HandWiki
Short description: Security bug affecting Cloudflare

Cloudbleed was a Cloudflare buffer overflow disclosed by Project Zero on February 17, 2017. Cloudflare's code disclosed the contents of memory that contained the private information of other customers, such as HTTP cookies, authentication tokens, HTTP POST bodies, and other sensitive data.[1] As a result, data from Cloudflare customers was leaked to all other Cloudflare customers that had access to server memory. This occurred, according to numbers provided by Cloudflare at the time, more than 18,000,000 times before the problem was corrected.[2][3] Some of the leaked data was cached by search engines.[3][4][5][6][7][8]

Discovery

The discovery was reported by Google's Project Zero team.[1] Tavis Ormandy[9] posted the issue on his team's issue tracker and said that he informed Cloudflare of the problem on February 17. In his own proof-of-concept attack he got a Cloudflare server to return "private messages from major dating sites, full messages from a well-known chat service, online password manager data, frames from adult video sites, hotel bookings. We're talking full https requests, client IP addresses, full responses, cookies, passwords, keys, data, everything."[1]

Similarities to Heartbleed

In its effects, Cloudbleed is comparable to the 2014 Heartbleed bug, in that it allowed unauthorized third parties to access data in the memory of programs running on web servers, including data which had been shielded while in transit by TLS.[10][11] Cloudbleed also likely impacted as many users as Heartbleed since it affected a content delivery network serving nearly two million websites.[4][11]

Tavis Ormandy, first to discover the vulnerability, immediately drew a comparison to Heartbleed, saying "it took every ounce of strength not to call this issue 'cloudbleed'" in his report.[1]

Reactions

Cloudflare

On Thursday, February 23, 2017, Cloudflare wrote a post noting that:[12]

The bug was serious because the leaked memory could contain private information and because it had been cached by search engines. We have also not discovered any evidence of malicious exploits of the bug or other reports of its existence.
The greatest period of impact was from February 13 and February 18 with around 1 in every 3,300,000 HTTP requests through Cloudflare potentially resulting in memory leakage (that’s about 0.00003% of requests).

Cloudflare acknowledged that the memory could have leaked as early as September 22, 2016. The company also stated that one of its own private keys, used for machine-to-machine encryption, has leaked.

It turned out that the underlying bug that caused the memory leak had been present in our Ragel-based parser for many years but no memory was leaked because of the way the internal NGINX buffers were used. Introducing cf-html subtly changed the buffering which enabled the leakage even though there were no problems in cf-html itself.[3]

John Graham-Cumming, Cloudflare CTO, noted that Cloudflare clients, such as Uber and OkCupid, weren't directly informed of the leaks due to the security risks involved in the situation. “There was no backdoor communication outside of Cloudflare — only with Google and other search engines,” he said.[6]

Graham-Cumming also said that "Unfortunately, it was the ancient piece of software that contained a latent security problem and that problem only showed up as we were in the process of migrating away from it." He added that his team has already begun testing their software for other possible issues.[7]

Google Project Zero team

Tavis Ormandy initially stated that he was "really impressed with Cloudflare's quick response, and how dedicated they are to cleaning up from this unfortunate issue."[1] However, when Ormandy pressed Cloudflare for additional information, "They gave several excuses that didn't make sense,"[13] before sending a draft that "severely downplays the risk to customers."[14]

Uber

Uber stated that the impact on its service was very limited.[10] An Uber spokesperson added "only a handful of session tokens were involved and have since been changed. Passwords were not exposed."[15]

OKCupid

OKCupid CEO Elie Seidman said: "CloudFlare alerted us last night of their bug and we've been looking into its impact on OkCupid members. Our initial investigation has revealed minimal, if any, exposure. If we determine that any of our users has been impacted we will promptly notify them and take action to protect them."[10][15]

Fitbit

Fitbit stated that they had investigated the incident and only found that a "handful of people were affected". They recommended that concerned customers should change their passwords and clear session tokens by revoking and re-adding the app to their account.[16]

1Password

In a blog post, Jeffery Goldberg stated that no data from 1Password would be at risk due to Cloudbleed, citing the service's use of Secure Remote Password protocol (SRP), in which the client and server prove their identity without sharing any secrets over the network. 1Password data is additionally encrypted using keys derived from the user's master password and a secret account code, which Goldberg claims would protect the credentials even if 1Password's own servers were breached. 1Password did not suggest users change their master password in response to a potential breach involving the bug.[17]

Remediation

Many major news outlets advised users of sites hosted by Cloudflare to change their passwords, as even accounts protected by multi-factor authentication could be at risk.[18][19][20][7][21] Passwords of mobile apps too could have been impacted.[22] Researchers at Arbor Networks, in an alert, suggested that "For most of us, the only truly safe response to this large-scale information leak is to update our passwords for the Web sites and app-related services we use every day...Pretty much all of them."[23]

Inc. Magazine cybersecurity columnist, Joseph Steinberg, however, advised people not to change their passwords, stating that "the current risk is much smaller than the price to be paid in increased 'cybersecurity fatigue' leading to much bigger problems in the future."[24]

References

  1. 1.0 1.1 1.2 1.3 1.4 "Issue 1139: cloudflare: Cloudflare Reverse Proxies are Dumping Uninitialized Memory". 19 February 2017. https://bugs.chromium.org/p/project-zero/issues/detail?id=1139. 
  2. "About Cloudflare". Cloudflare. https://www.cloudflare.com/about-overview/. "Every week, the average Internet user touches us more than 500 times." 
  3. 3.0 3.1 3.2 "Incident report on memory leak caused by Cloudflare parser bug". Cloudflare. 23 February 2017. https://blog.cloudflare.com/incident-report-on-memory-leak-caused-by-cloudflare-parser-bug/. "1 in every 3,300,000 HTTP requests through Cloudflare potentially resulted in memory leakage." 
  4. 4.0 4.1 Thomson, Iain (24 Feb 2017). "Cloudbleed: Big web brands leaked crypto keys, personal secrets thanks to Cloudflare bug". https://www.theregister.co.uk/2017/02/24/cloudbleed_buffer_overflow_bug_spaffs_personal_data/. 
  5. Burgess, Matt. "Cloudflare has been leaking private Uber, Fitbit and Ok Cupid details for months" (in en-GB). WIRED UK. https://www.wired.co.uk/article/cloudflare-cloudbleed-passwords-leaked-online. 
  6. 6.0 6.1 Conger, Kate (24 February 2017). "Major Cloudflare bug leaked sensitive data from customers' websites". https://techcrunch.com/2017/02/23/major-cloudflare-bug-leaked-sensitive-data-from-customers-websites/. 
  7. 7.0 7.1 7.2 "CloudFlare Leaked Sensitive Data Across the Internet For Months". http://fortune.com/2017/02/24/cloudflare-leak-bug-sensitive-information/. 
  8. Wagstaff, Jeremy (2017-02-24). "Bug causes personal data leak, but no sign of hackers exploiting: Cloudflare". https://www.reuters.com/article/us-cyber-cloudflare-idUKKBN1630RT. 
  9. Marc Rogers interviewed on the TV show Triangulation on the TWiT.tv network
  10. 10.0 10.1 10.2 Fox-Brewster, Thomas. "Google Just Discovered A Massive Web Leak... And You Might Want To Change All Your Passwords". Forbes. https://www.forbes.com/sites/thomasbrewster/2017/02/24/google-just-discovered-a-massive-web-leak-and-you-might-want-to-change-all-your-passwords/#134d3e43ca3e. 
  11. 11.0 11.1 Estes, Adam Clark. "Everything You Need to Know About Cloudbleed, the Latest Internet Security Disaster" (in en-US). Gizmodo. https://gizmodo.com/everything-you-need-to-know-about-cloudbleed-the-lates-1792710616. 
  12. "CloudBleed memory leak bug explained-why it all happened | PcSite" (in en-US). PcSite. 2017-02-25. https://pcsite.co.uk/cloudbleed-cloudflare-memory-leak-bug-explained/. 
  13. "1139 - project-zero - Project Zero - Monorail". https://bugs.chromium.org/p/project-zero/issues/detail?id=1139#c16. 
  14. "1139 - project-zero - Project Zero - Monorail". https://bugs.chromium.org/p/project-zero/issues/detail?id=1139#c19. 
  15. 15.0 15.1 Larson, Selena (2017-02-24). "Why you shouldn't freak out (yet) about the 'Cloudbleed' security leak". https://money.cnn.com/2017/02/24/technology/cloudflare-cloudbleed-security-vulnerability/index.html. 
  16. "Help article: How is Fitbit keeping my data secure in light of the Cloudflare security issue?". https://help.fitbit.com/articles/en_US/Help_article/2177. 
  17. "Three layers of encryption keeps you safe when SSL/TLS fails | 1Password" (in en). 2017-02-23. https://blog.1password.com/three-layers-of-encryption-keeps-you-safe-when-ssl/tls-fails/. 
  18. "Cloudbleed: How to deal with it". 2017-02-24. https://medium.com/@octal/cloudbleed-how-to-deal-with-it-150e907fd165. 
  19. "Cloudbleed Explained: Flaw Exposes Mountains of Private Data" (in en). Popular Mechanics. 2017-02-24. http://www.popularmechanics.com/technology/security/a25380/cloudbleed-explained/. 
  20. Constantin, Lucian. "Cloudflare bug exposed passwords, other sensitive data from websites" (in en). CIO. http://www.cio.com/article/3173858/security/cloudflare-bug-exposed-passwords-other-sensitive-data-from-websites.html. 
  21. Menegus, Bryan. "Change Your Passwords. Now." (in en-US). Gizmodo. https://gizmodo.com/cloudbleed-password-memory-leak-cloudflare-1792709635. 
  22. Weinstein, David (2017-02-24). "Cloudflare 'Cloudbleed' bug impact on mobile apps: Data sample of..." (in en-US). NowSecure. https://www.nowsecure.com/blog/2017/02/23/cloudflare-cloudbleed-bugs-impact-mobile-apps/. 
  23. "Dark Reading - Cloudflare Leaked Web Customer Data For Months" (in en). http://www.darkreading.com/attacks-breaches/cloudflare-leaked-web-customer-data-for-months/d/d-id/1328266?print=yes. 
  24. Joseph Steinberg (February 24, 2017). "Why You Can Ignore Calls To Change Your Passwords After Today's Massive Password Leak Announcement". Inc.. http://www.inc.com/joseph-steinberg/why-you-can-ignore-calls-to-change-your-passwords-after-todays-massive-password-.html. 

External links