What I don't get is how this wasn't thought up to be put into the SSL/TLS standard when it was built. How did we wind up with infinitely powered root certification authorities?
There is the part where the usage of X.509 certificates for DNS names on the Internet, as opposed to in the context of the (nonexistent) X.500 directory, is a gigantic hack[1]. This means the definition of “when it was built” is rather hazy. Also, the 1994 Netscape implementation literally accepted a CN=foo.com certificate when connecting to bar.org, so the state of SSL when it was built is not exactly a stellar reference.
Still, the name constraints extension, which restricts all certificates (transitively) issued from a given CA to a given DNS subtree, has been in the “Internet profile” of X.509 (PKIX) since the December 1996 draft[2]. The problem from this technical point of view is that very few implementations supported it until a couple of years ago[3].
Is this actually part of the standard? AFAIK TLS allows/demands to authenticate one or both sides via certificates, and defines a mechanism to delegate trust via certificate chains. It does not, AFAIK, define how trust is established, and my guess is that the standard authors realized that this is an infinitely complex topic that should not be intermixed with the technical and cryptographic side of the problem.
It just turns out that delegating trust to root CAs, CAs, and browser/OS vendors (the latter via built-in certificate lists) makes it easy for the end user.
TLS itself has no opinion about how certificates work. AFAIK it would be totally fine by the standard to put a JPEG photo of a your primary school certificate for 10m swimming where certificate goes in the protocol. If the other party is OK with your proof that you can swim to secure the connection, all is good.
Netscape invented all this stuff in the 1990s as SSL. Turns out you need a PKI to make it work, because of a tricky edge case which otherwise makes the whole thing worthless. So, they used the existing but little used X.509 PKI left over from the X.500 directory work, even though the Internet is not part of the envisioned global network X.500 is for. The X.509 PKI had a bunch of famous brand "trustworthy" companies minting certificates.
PKIX, an IETF working group to figure out how to force X.509 to be suitable for the Internet, adds stuff like SANs (Subject Alternative Names, a way to express Internet ideas about naming like IP addresses and DNS names) but that all happens after SSL 2.0 and SSL 3.0 and people start writing https URLs.
> It just turns out that delegating trust to root CAs, CAs, and browser/OS vendors (the latter via built-in certificate lists) makes it easy for the end user.
This flexibility is what allowed let’s encrypt to bootstrap, right?
The same reason that lies behind many problems in IT: a lot of the early (and now still foundational) protocols date back to times where the participants in the network were universities or other large entities (governments, large corporations) and trust was assumed between them.
Obviously, that broke down over time, and nefarious actors (both governmental and private) popped up.
That isn't the only reason. Another big one is that the problem itself is hard, so noone could come up with an easy solution. How would you establish trust in a world without root CAs?
Peer-to-peer? That would require you to constantly monitor the activity of those peers and update your lust of trusted certificates.
Singular trusted entities like browser or OS vendors? We already have that. Browser/OS certificate list are currently the entry point and have even higher priority than root CAs -- that is what the article is about.
Government bodies are an easy-to-go "trusted" entity because, while you may or may not actually trust them, you are forced to "trust" them in many aspects of daily life anyway. You are forced to rely on the fact that the police doesn't randomly knock on your door and arrest you. Governmnent bodies may (depending on your country) simply demand higher taxes than you owe them, and you have to fight a lawsuit to avert that. Government bodies can easily falsify evidence that makes you lose your house because it now says that somebody else is the owner (lawsuit again).
Basically, establishing trust in a matter of "I am actually talking to the legitimate server behind example.com and not a MITM" is easy, that could be solved by DNSSEC and cert pinning (although for legacy reasons, we went with CAs as the middle man).
But the really interesting thing, which is unsolved to date, is making sure that when you type "bank.com" in a browser, you have some sign that you are actually talking to Bank LLC and not some other entity.
In ye olde times, you didn't have to worry about someone else spoofing the domain name of "bank.com", e.g. as "b4nk.com" or "bänk.com", for one because access to registering domains was complex and expensive, and for one because criminals hadn't found out they could make money...
I feel like the underlying cause is that we give a significance to URLs that they should not have. When, in your example, I want to visit the website of Bank LLC, I have to know that its URL is bank.com (right now I usually know this through Google), then enter that and then have to deal with the problem you describe.
This would be better solved if the URL as a middle step did not exist can I could directly select Bank LLC. Right now we rely on google to shield us from domain name spoofing.
But then, unless we exchange keys with an entity beforehand (1), how can I even express that I want to visit that entity's website? Let alone solve the spoofing problem. We have to delegate trust to search engines, browser built-in lists, governments, company registries, whatever.
(1) possible, but only works if you know in advance that you want to communicate with a specific entity. Like before opening a bank account. I does not help when you don't even know which banks make an offer you are interested in.
The current PKI is geared toward ease-of-use and adoption.
It can also be turned completely around: only trust a single root certificate. This design is often used in client authentication: each client need to get its certificate signed by the one single CA that's trusted by the server.
Right but implementation wise we've never implemented as a default anything else. Domain suffix pinning is available as an extension, but that's all it is - very few TLS stacks support it and that's unlikely to ever change.