Device ‘breakage’ concerns persist days before Let’s Encrypt root cert expiry

Many devices and systems may not be ready for the switchover

Let’s Encrypt’s root certificate is expiring in little over a week, breaking a chain of trust that could result in widespread problems.

The certificate currently used by the non-profit certificate authority, the IdentTrust DST Root CA X3, will expire on September 30.

A new root cert is ready to come into operation but even so, the changeover may not run smoothly, both independent experts and Let’s Encrypt itself warn.

Catch up on the latest encryption-related news and analysis

The change potentially impacts anything that uses TLS/PKI (public key infrastructure) on everything from web servers to smartphones and internet-connected devices. Digital certificates that underpin HTTPS are issued by certificate authorities, with indexes updated through operating system updates.

On updated systems the changeover is transparent. But if these indexes have not been updated for some time, then affected systems will not recognize the new Let’s Encrypt root certificate and the chain of trust will be broken. There are precedents for this.

In May 2020, the AddTrust External CA Root expired leaving organisations such as Roku, Stripe, Spreedly, and many others with problems as a result.

Ch-ch-changes

Whether or not the Let’s Encrypt root cert change will sever the chain of trust that underpins secure connections from a device or server broadly depends on how up to date a technology is, although in practice there’s more factors involved than this simple benchmark alone.

Web security expert Scott Helme told The Daily Swig: “The device needs to either not check the expiration date of the root (so it will keep working regardless) or the device needs to have updated recently enough to get the new ISRG root. Any other device will fail.”

Servers must also be configured correctly. “They need to have an updated certificate chain to serve to clients,” Helme explained.

Research carried out so far by Helme and others suggests some breakage is all but certain, even though the extent of potential problems is far less clear cut. In a detailed technical blog post, Helme outlines the root certificate issue and points to potential faultlines where things might break.

Systems depending on OpenSSL 1.0.2, released in January 2015 and last updated in December 2019, is particularly at risk along with earlier libraries. Manual updates on these systems are required, as explained in an OpenSSL advisory.

Catch up with the latest internet infrastructure news

A range of older Windows, Mac, iOS, and other clients could run into trouble, as listed by Helme. Older versions of Microsoft’s IIS web server software may be at risk too, although this remains unproven.

“IIS looks like it has some manual work required for the transition to be smooth, but it’s not too much,” Helme explained. “The problem is people won’t know it needs doing until it breaks and they figure out why.”

Point break

In response to queries from The Daily Swig, Let’s Encrypt thanked Helme for his research and acknowledge the root certificate breakage issue was real, while suggesting the main area of concern lies with Android devices.

“In order to make the transition easier, primarily for Android clients, we got a new cross-sign[ature] that extends past the life of the expiring root,” Josh Aas, executive director of the Internet Security Research Group at Let’s Encrypt, explained.

Aas went on to offer specific advice:

Probably the most important things people can do in advance of the expiration are look at how many clients are using affected versions of OpenSSL (an issue Scott described nicely in his post) and older versions of Android. Encourage clients to upgrade where possible and if it's not possible look into whether serving a certificate chain with our new cross-sign makes sense.

Helme commented: “Anecdotally, I think Let’s Encrypt are used more for websites, and their typical client will be a web browser, which puts them in a better position here with less likely issues to come… That said, I can’t shake the feeling that at least one important thing, somewhere, will break.”

The researcher concluded: “The main concern will be slightly older devices/software and we can only take a guess at how widespread that is. In truth we will never know everything that will break until it breaks!”