Tip: Check that your Offline Root CA is actually Offline, mmkay?

I spend a fair whack of time chatting PKI and certificates with customers, and tyre-kicking their environments as part of the Active Directory Certificate Services Assessment (or ADCSA - available via Premier Support).

Many customers have a fairly standard design, often deployed by a partner (it’s the “off the shelf plus customize” option), which includes an Offline Root CA, and one or more online Issuing CAs.

The Offline Root purely exists to sign Issuing CA certificates and publish a CRL occasionally, and is typically airgapped if it’s physical. The Issuing CAs are the ones which are typically connected to directly (or via a Web Service) by client computers.

What’s perhaps underemphasised in some designs is the inferred meaning of the word “Offline”. “Offline” means “no network cable”.

Pros of an Offline Root CA:

- Because it’s airgapped (i.e. has no network cable):

- You don’t have to service it (i.e. patch, service pack, update - except for reliability issues)

- You don’t have to closely manage its operational health – just boot it up once a quarter, copy a CRL to local storage, and shut it down

- It gets to use separate credentials from the rest of AD, so it’s isolated from credential attack

  - Also, remember, no network cable = limited network attack surface, right?

Cons of an Offline Root CA:

- Because it’s got no network cable:

- It’s harder to manage than an Online Root CA, which it becomes if you plug a network cable into it.

  - You need to use console access if it’s on a VM host (which, broadly, is an iffy idea at most organizations. Yes, probably including yours.)

  - You need to use virtual floppies, or real floppies, or probably more likely USB sticks to transfer files to and from it.

Okay, so you trade no network cable and hands-on management for greatly improved security, with the goal of keeping your root’s private key safe.

Airgapped is pretty safe: no network cable is a fairly heavy defence against the casual network-based attacker!

But…

But: at some point, due to error or oversight, many “Offline” Root CAs get attached to the network. Maybe because a new admin wants to RDP in for some reason, maybe because it’s more convenient when trying to publish the CRL. Whatever. Now, we can build a list of pros and cons for your Online Offline CA:

Pros and Cons of an Online Offline Root CA:

- It’s one of the most vulnerable hosts on the network, because it’s not patched, because it’s not part of any patch group or configured for WSUS, and you don’t need to patch Offline CAs, right?

- Does it have antivirus or firewall or even local policy applied? Probably not, because it’s been designed not to be network-attached. (Not that settings or AV or software defences beat an unpatched exploit, but let’s suggest they might help with some common attacks.)

- It’s easier to manage, because it’s network-accessible. So, yay that!

The Good News:

Hah! Just kidding! If you’ve read carefully this far and think there’s any good news, my communication skills have failed you. There’s not, and depending on your other security practices and the assurance level required for certs issued by this CA, you should think carefully about starting over with provable key provenance.

If your Offline (unpatched!) CA has been Online for any period of time, the Root’s private key provenance is unclear. Could the machine have been exploited, and the private key leaked, undetected? Unpatched security vulnerabilities make that more likely. Lack of close auditing reduces assurance. The primary security control in place – i.e. no network cable – was removed.

But how useful is a root private key, really? Well, an attacker can simply use it to mint any certificate which it’s likely your organization’s computers will trust, for any purpose, undetectably. The use of such a cert would be basically invisible to your organization without unbelievably close monitoring, the type of unbelievably close monitoring you probably don’t have if an Offline CA ends up Online for any length of time.

So what do I do?

Check now. Don’t wait for me to show up and look horrified/disappointed! Or worse, tell jokes. You wouldn’t like me when I’m funny.

If you’re running a physical computer as an Offline CA, it’s pretty straightforward. It’s hopefully in a safe, or in another secured location where you can either prove it’s airgapped or it’s very easy to infer that it is (nobody’s in the safe; computer is airgapped).

If you’re running on a virtualized OS, it’s murkier. Maybe someone adjusted the virtual network settings; maybe it’s attached to a fake network; maybe it’s unattached completely. Virtualized CAs have many other security implications which need attention if you’re serious about your PKI.

So quick test: If you can RDP to the CA from an internal network client, it’s not airgapped. And it’s not Offline. And if it’s not Offline, and it’s not being updated with all haste at the start of each month, odds are it’s been vulnerable to known exploits for a period of time already (without restrictive firewall policies – but if you think about an RDP vulnerability, maybe even with restrictive firewall policies in place…).

There’s an extensive white paper on Securing Public Key Infrastructure which helps talk through many important aspects of CA security.

But this is something I’ve seen in the wild, and it’s scary.

So as a final note: if you come across something called “XXQ-BLAH-CA01” in a VM console and it doesn’t seem to be network connected: Leave it unplugged!

(Also, please avoid deleting it. No, there’s no story there. Why would you ask?)