However, Google Cloud Platform repelled security researchers’ attack
UPDATED Security researchers have harnessed the novel ‘H2C smuggling’ technique to achieve authentication, routing, and WAF bypasses on a number of leading cloud platforms.
The attack’s first in-the-wild scalps included routing and WAF bypasses in Microsoft Azure, and an authentication bypass in Cloudflare Access, although Google Cloud Platform emerged unscathed.
The technique’s architects, from security firm Bishop Fox, noted in a landmark write-up that load balancers such as AWS ALB/CLB, NGINX, and Apache Traffic Server, blocked H2C smuggling because they “won’t forward the required headers for a compliant H2C connection upgrade”, reads a blog post from security monitoring platform Assetnote.
BACKGROUND H2C smuggling named top web hacking technique of 2020
However, Bishop Fox had also noted that “not all backends were compliant, and we could test with the non-compliant Connection: Upgrade variant, where the HTTP2-Settings value is omitted from the Connection header,” according to Assetnote engineering lead Sean Yeoh.
Reengineering Bishop Fox’s h2cSmuggler tool accordingly, Assetnote researchers managed to find “multiple instances of off-the-shelf configured services that permitted H2C upgrades”, paving the way to authorization control bypasses “on interim reverse proxies”.
What are H2C smuggling attacks?
Unveiled in September 2020, HTTP/2 cleartext (H2C) smuggling “abuses H2C-unaware front-ends to create a tunnel to backend systems, enabling attackers to bypass frontend rewrite rules and exploit internal HTTP headers,” James Kettle, head of research at PortSwigger Web Security*, has said.
Kettle made the comments after the attack was revealed to be the top web hacking technique of 2020.
H2C, a deprecated protocol, upgrades a regular, transient plaintext HTTP connection to a persistent connection using the HTTP2 binary protocol. And when a HTTP request issued to a reverse proxy “includes a Connection: Upgrade header the proxy maintains the persistent connection, and scope for continuous communication, between the client and server”, explained Yeoh.
“Using H2C Smuggling, we can bypass [routing] rules a reverse proxy uses when processing requests such as path-based routing, authentication, or the WAF processing provided we can establish a H2C connection first.”
Microsoft Azure
Microsoft Azure presented “the most interesting use case for impact,” said Yeoh, because “the Azure Application Gateways offer the ability to attach the Azure WAF to the gateway.”
With the access gateway removing HTTP2-Settings from the Upgrade header but leaving the others “untouched”, the researchers were able to bypass routing rules.
But “more importantly, when the Azure WAF is configured, this provides a global WAF bypass provided your first request does not get blocked by the WAF and you can establish a H2C connection”.
Yeoh praised Microsoft for ensuring “a painless and smooth process” despite the difficulty of applying security fixes without disrupting the customer experience.
Cloudflare Access
Rules applied by Cloudflare Access, an authentication service enforced by Cloudflare’s load balancer, were bypassed because request proxying “modified the Upgrade header to exclude HTTP2-Settings” but retained the other headers.
Alerted via their bug bounty program, Cloudflare “were very responsive” in fixing the flaw, despite having to “balance customer expectations around servicing H2C connections”, said Yeoh.
Google Cloud Platform
Although Google’s load balancer permits configuration of basic routing rules, an attempted HTTP upgrade prompts the load balancer to strip “all Connection and HTTP2-Settings headers”, thus blocking a connection upgrade – and H2C smuggling attacks.
All other vulnerable cloud platforms denied Assetnote permission to disclose the details.
Lessons learned
To find these bypasses, researchers configured a server that upgraded both non-compliant and compliant H2C connections and found a load balancer configurable with routing rules or features.
Even though they used a non-compliant server, Yeoh pointed out that developers “may not understand the internals of their reverse proxies/internal services hosted behind the load balancer and hence may be vulnerable even if their load balancer is configured properly.”
That Jake Miller of Bishop Fox had surmised that major cloud providers would be invulnerable to H2C smuggling demonstrated “that even the best security researchers make [incorrect] assumptions about their research or may not have the time needed to find all affected parties”, concluded Yeoh.
Read more on the latest hacking techniques
“Consequently, even when research is made public there are often plenty of opportunities to extend and further the research.”
Assetnote’s investigation also demonstrates that security measures on the load balancer alone “can be insufficient when restricting access or securing your application”, the researcher added.
Nevertheless, he acknowledged the difficulty of keeping abreast of “these nuanced configuration issues, particularly across a large and fluid cloud attack surface”.
Asked what most be the most fruitful direction for further H2C smuggling research research, Yeoh told The Daily Swig that “there are a number of interesting avenues worth exploring”, in particular “H2C smuggling in the context of Kubernetes ingress and services”.
This article was updated on March 25 with the addition of further comments from Sean Yeoh of Assetnote.
*Disclosure: PortSwigger is The Daily Swig’s parent company
RELATED Google awards Uruguayan researcher $133,337 top prize in cloud security competition
