LINUX Certificate Enrollment and Automated Renewal Using NDES (Updated)


(APR 2018)  NDES continues to evolve (slowly) as everyone embraces the deprecation of SHA-1.  This post has been updated to reflect those changes and the impact on NDES.  Note the difference between the cryptography used in NDES / SCEP communications versus the cryptography in the certificates provided through NDES / SCEP.  For more detail on the reason for an update, check here.

We continue the customer conversation of managing heterogeneous environments... The conversations often revolve around “I know how to do this in (Windows / Linux)… how do I do this for both”.  In most cases, it is a matter of understanding risks, benefits and capabilities of the available technology.  Today we cover issuing and renewal of certificates for Linux systems issued from a Windows based Certificate Authority.

Certificates are needed for many reasons.  The example use case presented here would be to provide secure communications for an infrastructure agent like System Center Configuration Manager or to secure a web site running on Apache.  In those scenarios the certificate expiring will cause a service outage.  The solution would be to have the deployed certificates renewed (replaced) prior to expiration automatically.

Many Windows administrators are familiar with the native capability to auto-enroll domain joined systems with an enterprise Certificate Authority (CA). In simple terms, the process involves implementing an Active Directory integrated CA, configuring templates and enabling group policy to automatically enroll clients. (See: https://technet.microsoft.com/en-us/library/cc731522.aspx )

The reality is that implementing a proper CA hierarchy can be a lengthy task depending upon your organization’s size, security needs, requirements and policies.  (See: https://technet.microsoft.com/en-us/library/cc754057.aspx )  On a complexity scale (easy to difficult), it would be a moderate task to implement primarily because of the planning and design that should go into it.

This article is based on the existence of an already implemented Enterprise CA with Windows clients already auto-enrolled.

With your Windows clients taken care of the next question would be, “how do I do this with my Linux clients?” Auto-enrollment of domain joined Windows clients is made easier because the clients can already securely authenticate and be authorized by the enterprise CA. Non-domain joined systems like Linux clients and network devices require intervention to tell the enterprise CA, “I trust this device and authorize it to be enrolled”. To solve this, we will discuss two methods to obtain certificates; manual enrollment and automated using solutions like NDES.

MANUAL ENROLLMENT

Certificates can be obtained manually from a CA; often through web enrollment. The administrator authenticates to the CA on behalf of the device or system and generates (or submits) a certificate request.  While manual enrollment works perfectly, it does not allow to scale to large numbers of systems or address management of the certificates when they near expiration.

AUTOMATED ENROLLMENT USING NETWORK DEVICE ENROLLMENT SERVICE

The Network Device Enrollment Service (NDES) allows software on routers and other network devices running without domain credentials to obtain certificates based on the Simple Certificate Enrollment Protocol (SCEP) (See https://technet.microsoft.com/en-us/library/hh831498(v=ws.11).aspx )

The Network Device Enrollment Service performs the following functions:

  • Generates and provides one-time enrollment passwords to administrators.
  • Submits SCEP enrollment requests to the CA.
  • Retrieves enrolled certificates from the CA and forwards them to the network device.

SIMPLE CERTIFICATE ENROLLMENT PROTOCOL

(APR 2018) SCEP, as a protocol, remains in draft with the IETF.  For a number of years it garnered little attention.  The industry continues to heavily embrace its use though. The SCEP draft has been updated more frequently over the past few years with the most recent update as of MAR 2018 (see https://tools.ietf.org/html/draft-gutmann-scep-10 )  This of course makes SCEP security a moving target for all vendors and the open source community.  Hardware vendors for network devices will need to update firmware and software vendors as well as the open source community will need to update code to accommodate the changes.

SECURITY ADVISIORY

In 2012 a security advisory was posted regarding the vulnerability of SCEP. (See http://www.kb.cert.org/vuls/id/971035 ) The advisory states that Simple Certificate Enrollment Protocol (SCEP) does not strongly authenticate certificate requests made by users or devices. SCEP was designed for use in closed environments and not for the purpose of mobile device management. This solution may not meet your security requirements and warrant a thorough review. The use case example is for protected Linux systems, not fully untrusted mobile devices. There are a number of mitigations may allow this solution to meet your security requirements.

The above advisory is also subject to change as the spec is updated and the changes begin to be implemented.

IMPLEMENTATION OVERVIEW

The scenario outlined here should be executed in a lab allowing you to build a functional solution that you can then tune to meet your organization requirements.  The solution has been tested with number of Linux distributions.

The lab scenario setup for this article utilizes the following platforms:

  • Windows Server 2012 R2 Enterprise Certificate Authority in  a two tier hierarchy (offline root, online enterprise issuing subordinate) (APR 18) Tested with Windows Server 2016 in a two tier hierarchy.  The CA hierarchy should be configured to issue SHA2 certificates (SHA256 or higher). For this lab, the subordinate issuing authority is configured to issue SHA384 certificates.
  • CentOS 6.8 - 7.4

Within the LINUX environment, execute the actions with elevated privileges (via sudo or su). Ensure the hostname of your Linux system is the fully qualified domain name of the host. Example: linux1.contoso.corp The FQDN will be used in the certificate submission as the DNS name field for the certificate.

IMPLEMENTATION STEPS

  1. SETUP NDES FOR ACTIVE DIRECTORY CERTIFICATE SERVICES

The first step in the process is to enable your environment to support NDES. Follow the implementation guidance in the NDES WIKI http://social.technet.microsoft.com/wiki/contents/articles/9063.network-device-enrollment-service-ndes-in-active-directory-certificate-services-ad-cs.aspx

 

  1. CONFIRM THE DEFAULT TEMPLATES EXIST

Once NDES is deployed, confirm that the IPSEC Intermediate (Offline) template is deployed. The out-of-box template cryptography settings will likely be 1024 bit SHA 1. (APR 2018) The minimum key size should be updated to 2048.  The request hash can remain at SHA1 for testing or be moved to a higher value if all clients and network devices serviced by this NDES server support it.  For the purpose of the example it should be moved to SHA256

Note: Once your test environment is functional, you can go back and create new templates that meet the organizational needs.  Review the NDES wiki for the section on updating the registry to point to the new template name.

 

  1. INSTALL THE SSCEP CLIENT

The Fedora hosted DogTag project is a collection of Open Source Certificate Authority technologies. One of the components is Simple SCEP (SSCEP). SSCEP can be found in the Extra Packages for Linux (EPEL) or can be pulled from GIT (see https://github.com/certnanny/sscep ).  To install SSCEP from EPEL, first add the epel repository and then install the client.  On the Linux system, execute the following:

yum install epel-release
yum install sscep

 

  1. INSTALL THE CA CHAIN ON THE LINUX CLIENT

The Linux endpoint requires the CA hierarchy chain to being SCEP enrollment. In the following syntax example:

  • Replace “ContosoCA” with a simple name that reflects your CA (to indicate where the file came from).
  • Replace “NDES.CONTOSO.CORP” with the FQDN that reflects the CA running NDES’s fully qualified domain name.

Execute the following command:

sscep getca -F sha1 -c /etc/pki/ca-trust/source/anchors/contosoCA.crt -u http://ndes.contoso.corp/certsrv/mscep/mscep.dll/pkiclient.exe?

Note the use of SHA 1.   This is specifying use for the fingerprint algorithm.  For more detail on why the SHA1 can be used with NDES / SCEP, see section 8.8 of the current draft (See https://tools.ietf.org/html/draft-gutmann-scep-10).

In a two tier hierarchy, the command will pull down four certificate files.

  • contosoCA.crt-0                               The certificate for the issuing subordinate CA root Registration Authority (RA)
  • contosoCA.crt-1                                The certificate for the subordinate CA root CEP Encryption (CE)
  • contosoCA.crt-2                               The certificate for the CA root.
  • contosoCA.crt-3                               The certificate for the issuing subordinate CA root.

For an understanding of the LINUX system PKI file structure, see https://www.mankier.com/8/update-ca-trust   If you are running CentOS 6, you may have to enable your CA trust store to be updated by running:

update-ca-trust enable

Next, execute the following command:

update-ca-trust extract

This will add the downloaded CA chain to the Linux system’s trusted CA list.

 

  1. GENERATE A CSR FOR THE INITIAL ENROLLMENT

Log into the issuing CA server website with the required credentials of a certificate administrator. The example URL is http://ndes.contoso.corp/certsrv/mscep_admin

Obtain the NDES enrollment challenge password. In the NDES WIKI example, the password is 302034237AE9A1D1 (See the picture) . By default, the enrollment challenge password is for one time use and has a time expiration. This can be changed to allow longer time periods and multi-device use. Those changes however may not meet your organization security requirements.

On the LINUX client, run the command mkrequest to generate a Certificate Signing Request (CSR). Insert the issued enrollment challenge password. Example:

mkrequest -dns $(hostname) 302034237AE9A1D1

The mkrequest script will generate two files by default. The two files are the CSR and the certificate private key. (local.csr and local.key).

 

  1. EXECUTE THE NDES INITIAL ENROLLMENT

The next step is to execute the actual NDES enrollment using the SCEP client. An example enrollment would be:

sscep enroll -E 3des -S sha1 -c /etc/pki/ca-trust/source/anchors/contosoCA.crt-0 -e /etc/pki/ca-trust/source/anchors/contosoCA.crt-1 -k local.key -r local.csr -l $(hostname).crt -u 'http://ndes.contoso.corp/certsrv/mscep/mscep.dll/pkiclient.exe?' -d -v

This command will execute a certificate enrollment using the CSR, the certificate private key and the one time enrollment password.  Again, note the -E and -S switches.  -E is the PKCS#7 encryption algorithm, and the -S is the signature algorithm.  Again, refer to section 8.8 of the current draft.  This is the encryption used with NDES, not the encryption settings for the certificate that the client will receive.  That is specified in the template and will be SHA256 or higher (based on your template).

 

CERTIFICATE RENEWAL

Certificate renewal is a very similar process. However instead of using an enrollment challenge password, the enrollment uses the private key of the existing certificate and the valid existing certificate as the authorization credential.

  1. GENERATE A CSR FOR THE RENEWAL

For a renewal, no password is required. You will be using a combination of the old key, new key, old certificate and a new CSR. If you are executing this command in the same directory containing the original private key, it must be renamed so that it is not overwritten. Execute the mkrequest script including just the DNS name for the certificate. Example:

mkrequest -dns $(hostname)

 

  1. EXECUTE THE RENEWAL (REPLACEMENT CERTIFICATE)

For a renewal, you will need the existing certificate, existing private key, the new CSR and the new private key.

sscep enroll -E 3des -S sha1 -c /etc/pki/ca-trust/source/anchors/contosoCA.crt-0 -e /etc/pki/ca-trust/source/anchors/contosoCA.crt-1 -k local.key -K existinglocal.key -O existing.crt -r local.csr -l new.crt -u 'http://ndes.contoso.corp/certsrv/mscep/mscep.dll/pkiclient.exe?' -d -v

AUTOMATING THE SOLUTION

The primary objective is to automate certificate enrollment and renewal. This can be accomplished a number of ways depending on what existing capabilities your organization may already have. The automation can be:

  • A simple CRON job executing a script
  • PowerShell Desired State Configuration (DSC) for Linux
  • A Puppet module

The following shell script demonstrates what the workflow for initial enrollment and renewal would look like.
(Updated APR 2018)

 

#!/bin/bash
# Linux Certificate Enrollment Using NDES and SCEP
##################################################################
# This script is provided as an example for illustration only,
# without warranty either expressed or implied, including, but not
# limited to, the implied warranties of merchantability and/or
# fitness for a particular purpose.
##################################################################
#
# Insert the NDES Enrollment Password here.
ndeskey=ABC123DEF456ABC789DEF012ABC
# Insert the NDES server FQDN
ndesserver=ndes.contoso.corp
# CA Filename
cafilename=contoso
# Number of days ahead of cert expiration to renew
warning_days=45
###############################################################################
#EDIT ONLY THE VALUES ABOVE
###############################################################################
shopt -s -o nounset
# Create a log file directory
if [ ! -d /var/log/pki/ndes ]; then
    mkdir -p /var/log/pki/ndes/
fi
DTG=$(date +%Y%m%d%H%M)
LOGFILE=/var/log/pki/ndes/ndes-enrollment$DTG.log
if [ ! -f $LOGFILE ]; then
    touch $LOGFILE
fi
###############################################################################
# LOG FUNCTION TO WRITE LOCAL LOG FILE AND TO /VAR/LOG/MESSAGES
###############################################################################
writelog() {
    echo ${*} 2>&1 >> $LOGFILE
    if [ -f /bin/logger ] 
    then 
       logger ${*}
    fi 
}
###############################################################################
writelog  "sscep: Checking for required packages"
if rpm -qa | grep epel-release  2>&1 > /dev/null;
        then
        writelog "sscep: epel-release is installed."
else
    yum -y install epel-release 2>&1 >> $LOGFILE
fi
if rpm -qa | grep sscep  2>&1 > /dev/null;
        then
        writelog "sscep: sscep is installed."
else
    yum -y install sscep 2>&1 >> $LOGFILE
fi

MKREQUESTSCRIPT=/usr/bin/mkrequest
sed -i 's/KEYBITS=1024/KEYBITS=2048/g' $MKREQUESTSCRIPT
if ! grep -q "string_mask = nombstr" $MKREQUESTSCRIPT ; then
       sed -i '/\[ req \]/a string_mask = nombstr' $MKREQUESTSCRIPT
fi

if [ ! -f /etc/pki/tls/certs/$(hostname).crt ] ; then
        writelog "sscep: $(hostname) does not have an existing certificate.  Executing initial enrollment."
        writelog "sscep: Requesting CA certificate chain as x509 .crt files."
        sscep getca -F sha1 -c /etc/pki/ca-trust/source/anchors/${cafilename}CA.crt -u http://${ndesserver}/certsrv/mscep/mscep.dll/pkiclient.exe? 2>&1 >> $LOGFILE
        writelog "sscep: Adding the CA chain to the host CA trusted chain."
        update-ca-trust enable 2>&1 >> $LOGFILE
        update-ca-trust extract 2>&1 >> $LOGFILE
        writelog "sscep: Generating CSR."
        mkrequest -dns $(hostname) $ndeskey 2>&1 >> $LOGFILE
        writelog "sscep: Moving CSR and new private key." 
        mv local.key /etc/pki/tls/private/$(hostname).key
        mv local.csr /etc/pki/tls/private/$(hostname).csr
        writelog "sscep: Executing enrollment using NDES key."
        sscep enroll -E 3des -S sha1 -c /etc/pki/ca-trust/source/anchors/${cafilename}CA.crt-0 -e /etc/pki/ca-trust/source/anchors/${cafilename}CA.crt-1 -k /etc/pki/tls/private/$(hostname).key -r /etc/pki/tls/private/local.csr -l /etc/pki/tls/private/$(hostname).crt -u http://${ndesserver}/certsrv/mscep/mscep.dll/pkiclient.exe? -d -v 2>&1 >> $LOGFILE
        writelog "sscep: Checking generated certificate status."
        CERTSTATUS=$(openssl verify /etc/pki/tls/private/$(hostname).crt | cut -d: -f2)
        if [ $CERTSTATUS == "OK" ]; then
                writelog "sscep: Certificate status is: " $CERTSTATUS
                writelog "sscep: Copying new certificate to /etc/pki/tls/certs/$(hostname).crt" 
                cp /etc/pki/tls/private/$(hostname).crt /etc/pki/tls/certs/$(hostname).crt 2>&1 >> $LOGFILE
        else
                writelog "sscep: The certificate is not valid per $CERTSTATUS"
        fi
else
        output=$(openssl x509 -in /etc/pki/tls/certs/$(hostname).crt -noout -subject -dates 2>/dev/null)
        cert=$(echo $output | sed 's/.*CN=\(.*\).*not.*/\1/g')
        start_date=$(echo $output | sed 's/.*notBefore=\(.*\).*not.*/\1/g')
        end_date=$(echo $output | sed 's/.*notAfter=\(.*\)$/\1/g')
        start_epoch=$(date +%s -d "$start_date")
        end_epoch=$(date +%s -d "$end_date")
        epoch_now=$(date +%s)
        if [ "$start_epoch" -gt "$epoch_now" ]; then
                writelog "sscep: Certificate for [$cert] is not yet valid"
                writelog $output
        fi
        seconds_to_expire=$(($end_epoch - $epoch_now))
        days_to_expire=$(($seconds_to_expire / 86400))
        writelog "sscep: Days to expiry: ($days_to_expire)"
        warning_seconds=$((86400 * $warning_days))
        if [ "$seconds_to_expire" -lt 0 ]; then
                writelog "sscep: Certificate [$cert] has expired.  Remove the certificate and rerun to start a new enrollment."
        else
                if [ "$seconds_to_expire" -lt "$warning_seconds" ]; then
                        writelog "sscep: Certificate [$cert] is soon to expire ($seconds_to_expire seconds)"
                        writelog "sscep: Existing certificate found for $(hostname).  Executing re-enrollment."
                        writelog "sscep: Backing up current private key."
                        mv -f /etc/pki/tls/private/$(hostname).crt /etc/pki/tls/private/$(hostname).crt.bak
                        mv -f /etc/pki/tls/private/$(hostname).key /etc/pki/tls/private/$(hostname).key.bak
                        writelog "sscep: Generating CSR."
                        mkrequest -dns $(hostname) 2>&1 >> $LOGFILE
                        writelog "sscep: Moving CSR and new private key."
                        mv -f local.key /etc/pki/tls/private/$(hostname).key
                        mv -f local.csr /etc/pki/tls/private/$(hostname).csr
                        writelog "sscep: Executing enrollment using existing certificate."
                        sscep enroll -E 3des -S sha1 -c /etc/pki/ca-trust/source/anchors/${cafilename}CA.crt-0 -e /etc/pki/ca-trust/source/anchors/${cafilename}CA.crt-1 -k /etc/pki/tls/private/$(hostname).key -K /etc/pki/tls/private/$(hostname).key.bak -O /etc/pki/tls/private/$(hostname).crt.bak -r /etc/pki/tls/private/$(hostname).csr -l /etc/pki/tls/private/$(hostname).crt -u http://${ndesserver}/certsrv/mscep/mscep.dll/pkiclient.exe? -d -v  2>&1 >> $LOGFILE
                        writelog "sscep: Checking generated certificate status."
                        CERTSTATUS=$(openssl verify /etc/pki/tls/private/$(hostname).crt | cut -d: -f2)
                        if [ $CERTSTATUS == "OK" ]; then
                                writelog "sscep: Certificate status is: " $CERTSTATUS
                                writelog "sscep: Copying new certificate to /etc/pki/tls/private/$(hostname).crt"
                                cp -f /etc/pki/tls/private/$(hostname).crt /etc/pki/tls/certs/$(hostname).crt
                        else
                                writelog "sscep: The certificate is not valid per $CERTSTATUS"
                                writelog "sscep: Restoring the backup existing certificate and keys."
                                mv -f /etc/pki/tls/private/$(hostname).crt.bak /etc/pki/tls/private/$(hostname).crt
                                mv -f /etc/pki/tls/private/$(hostname).key.bak /etc/pki/tls/private/$(hostname).key
                        fi
                fi
        fi
fi

A useful workflow would include:

  • Installation of the SSCEP client; if not already installed.
  • Check to see if a certificate exists; if not, execute initial enrollment.
  • If a certificate exists, check its expiration date. If it is within the defined renewal period, renew the certificate.
  • The logic would have to handle rotation of the private keys and matching certificates used for each subsequent renewal.

TUNING THE LAB TO MEET YOUR REQUIREMENTS

(APR 2018) At this time, SHA1 has been deprecated and the expected minimum key length is 2048. Hash algorithms should be SHA2 (SHA-256) or higher.

Changing the certificate length is easy. The /usr/bin/mkrequest file is actually a script. The default value of for KEYBITS=1024 can be changed to KEYBITS=2048. On the Windows CA, the CA template used for NDES would need to be updated to be set to 2048 bit as well.  The sample script above modifies the KEYBITS value during execution to update the value automatically.

In my previous version of this post, I outlined some of the cryptographic limitations of the SSCEP client.  Many of those have now been resolved as the clients have received a number of revision updates.

 

ALTERNATE OPTIONS

If the above solution does not meet your requirements, you might consider some of the other certificate solutions that are beyond the scope of this article. There are third party LINUX agents / add-ons that allow Linux clients to emulate Windows client capabilities to leverage group policy and certificate auto-enrollment. (For example, see http://www.centrify.com)

 

Q&A

Q:        My environment has hundreds of Linux systems that need certificates. How can I do this with each one requiring a unique enrollment challenge password?

A:        This is a conversation with your organization security official. NDES can be configured to provide a multi-use enrollment challenge password. Additionally the use time limit can be extended. Your organization has to evaluate the risk of that key being compromised and certificates being obtained by systems you do not trust.

 

Q:        Can I use this solution to deploy certificates to my mobile devices?

A:        NDES was designed for closed (internal trusted protected) network devices. Mobile devices should be managed with a Mobile Device Management (MDM) solution like Microsoft Intune. https://www.microsoft.com/en-us/cloud-platform/mobile-device-management

 

Q:        My organization determined that using the enrollment challenge password is not acceptable and I have not found another cost effective solution. What can I do to decrease the security risk?

A:        There are a number of options. One simple option would be to manually issue / install the initial certificate onto the Linux system but continue to configure it to use NDES for renewal. This ensures that you have installed certificates on fully trusted machines and the renewals will occur using a valid existing certificate and matching unique private key.

 

Q:        How do I test enrollment and renewal in the lab? I don’t want to wait a year or two until the certificate expires.

A:        In the lab, configure your deployed certificate template used by NDES to have a shorter validity period (4 hours for example).

 

Q:        Have I looked at CERTMONGER as an alternative to the SSCEP client?

A:        See https://blogs.technet.microsoft.com/jeffbutte/2018/04/05/linux-certificate-enrollment-and-automated-renewal-using-ndes-v2/

 

Q:        I am getting the following error:

sscep: pkistatus: FAILURE
sscep: finding attribute failInfo
sscep: allocating 1 bytes for attribute
sscep: reason: Transaction not permitted or supported
sscep: illegal size of payload

On the issuing CA, I see Event 29 in the Application event log.

The password in the certificate request cannot be verified. It may have been used already. Obtain a new password to submit with this request.

A:      This issue is most frequently caused by the version of OpenSSL that the system is running.  Since the creation of sscep, OpenSSL has undergone some significant revisions.  Recent security relevant updates to OpenSSL no longer allow UTF-8 fields added to the CSR.  Additional fields  like the SCEP password must be sent in PrintableString.  The resolution is to edit the /usr/bin/mkrequest file and add ‘string_mask = nombstr’ in the [req] section (Contributed by Todd Mote)

 

Comments (20)

  1. Anthony de Lagarde says:

    Jeff,

    Great content and article. Thank you for posting

  2. Todd Mote says:

    Holy cow. I’ve been trying to come up with something like this for 2 years. I wrote powershell to execute all the cert operations on windows then ftp all the certs to each Linux server for the sccm client to use.. they are expiring now and renewal was looking like a bear. almost got this working. my new NDES server gets a cert issued using the right template, but doesn’t put the Linux hostname i supply as the issued common name so the enroll fails on the Linux machine because it can’t find a name match:
    sscep: decrypting inner PKCS#7
    sscep: PKCS#7 payload size: 1249 bytes
    write_local_cert(): found 1 cert(s)
    sscep: found certificate with
    subject: ‘/CN=ndes.server.com’
    issuer: /DC=com/DC=server/CN=My-CA
    request_subject: ‘/CN=mylinux.server.com’
    X509_NAME_cmp() workaround: strcmp request subject (/CN=mylinux.server.com) to cert subject (/CN=ndes.server.com)
    sscep: cannot find requested certificate

    any ideas? i’m so close!!!

    1. Todd,
      You are close. Are you doing initial request or a renewal? It would appear that you have a mix up in your sscep enroll statement for a renewal. The error you are getting is that the CN in your CSR does not match the CN in the certificate. Can you post post your enrollment line? I suspect that you are pointing to the wrong CRT values in one of the -c -e -k -K -O -r -l options (likely -O). I also noticed in my example, there was an extra -k (fixed now). I should also update the post with examples of the output (which might help). What you should be seeing is:

      sscep: found certificate with
      subject: ‘/CN=mylinux.server.com’
      issuer: /DC=com/DC=server/CN=My-CA
      request_subject: ‘/CN=mylinux.server.com’
      CN’s of request and certificate matched!
      sscep: writing cert

      1. Todd Mote says:

        Initial request. yea, i can see the certs being issued on the CA but all of them have the wrong issued common name. So, I get 3 CA responses when i sscep getca. I have one root CA, so i got two RA certs, one usage digital signature, the other key encipherment, then the cert for the CA that has digital signature, certificate sign, CRL Sign. -0, -1, -2 respectively. my initial enroll looked like this(i just followed your example):

        # sscep enroll -c /etc/pki/ca-trust/source/anchors/ADTEST.crt-0 -e /etc/pki/ca-trust/source/anchors/ADTEST.crt-1 -k local.key -r local.csr -l myserver.crt -u http://ndes.server.com/certsrv/mscep/mscep.dll/pkiclient.exe? -d -v

        your example CA had 4 certs in it and your enroll only had the first two used so i followed suit. I’m not renewing so -K and -O shouldn’t come into play yet. i played around putting the other CA certs in different places, but none get me as far as the way it is above. i looked at the makerequest script, but i can’t see anything obvious in there that might do it. i may need to reinstall NDES again maybe and double check all of that is correct too?

        1. OK… all looks good so far. (In fact you are helping me find items to correct in the post. My order of the certs description was incorrect.) You are correct that since you have a single online root CA, you will only get 3 certs and you are using the correct RA and CE certs. My example shows 4 because I have an offline root.

          contosoCA.crt-0 The certificate for the issuing subordinate CA root Registration Authority (RA)
          contosoCA.crt-1 The certificate for the subordinate CA root CEP Encryption (CE)
          contosoCA.crt-2 The certificate for the CA root.
          contosoCA.crt-3 The certificate for the issuing subordinate CA root.

          In your case, you don’t get crt-3. Everything else looks good.
          For others reading this post, you can see the details by copying the certs over to a windows system and examining the certs or by using openssl to see the details (openssl x509 -in 'InsertCertHere' -noout -subject).

          Next, let’s examine the CSR to see where the incorrect CN / client FQDN is coming from. Run the command:
          openssl req -text -noout -verify -in local.csr
          The subject and DNS entries should be “mylinux.server.com” If that is not the case, then check your linux system’s hostname and make sure it is set as the FQDN.
          If the CSR is good, then we need to check the NDES configuration. Confirm what template you are using by checking the keys under HKLM\Software\Microsoft\Cryptography\MSCEP
          Select Manage Templates. Open up the template and check the Subject Name tab. Ensure that “Supply in the request” is selected. If it is set to “Build from Active Directory”, then make the change, remove the published template and republish.
          For further testing the NDES configuration, check out https://blogs.technet.microsoft.com/configmgrdogs/2015/08/24/so-you-want-to-test-your-ndesscep-certificate-enrollment/

          1. Todd Mote says:

            Jeff

            Thanks so much. I got it. DNS names were correct in the request, so that could only leave the CA or NDES. It turned out I was trying to use the same cert template that I have on my CA for SCCM clients as prescribed by their guidance for autoenrollment… on windows.. So it was set to ‘Build from AD’ and set to use ‘DNS name’ as the format. My brain saw ‘DNS name’ in that pull down and equated that with what it was provided in the request. i didn’t read the radio button closely enough. doh! so i duplicated that template, changed the subject name tab, redid my request and enrolled. and viola! certificate! thanks so much for responding and helping me get through it!

            now that i have the proof of concept, I’ve got the much harder job of convincing the powers that be that it’s a good idea! and how to change NDES to use the same password and how to secure that. we’ve got about 400 Linux machines to get new certs on. copy and pasting from the NDES admin portal doesn’t sound scalable. Do you have any thoughts on that?

          2. Glad to hear that you have achieved success.
            Here would be my recommended approach towards the discussions regarding mass deployment. Manually obtaining 400 (or more) one-time enrollment passwords is not reasonably manageable or sustainable and goes against the concept of automation. Increasing an organization’s risk exposure is also not desirable. Ideally the organization’s security resources would agree to the temporary use of a multi-use, extended time password for a fixed period of time (your initial deployment period).
            Using automation (script via CRON, DSC or Puppet module), you would enroll all 400 machines. After enrollment, change the NDES server back to using a single use enrollment password with a reasonable time limit.
            Your enrolled LINUX systems will execute the script or module periodically using their existing valid cert to authenticate going forward and no longer need the enrollment password. (In my example script, the renewal only happens when the existing cert is 45 days out from expiring. The script could be set to run daily on each system and would only re-enroll only when it is in the expiration window that you define.)
            After that, you would only need single-use, one time passwords for new LINUX OS deployments. You achieve your mass deployment; security is not unreasonably affected and your go-forward process remains automated and manageable.

  3. Alex Bush says:

    Great article, unbelievably useful!

    With regards to the SCEP client and SHA2 support, have you looked at using certmonger for the client that issues requests? (https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System-Level_Authentication_Guide/certmonger-scep.html)
    If so, what was your experience as it seems the more obvious choice? Just looking for ways to simplify the process.

    1. Alex,
      I had read over it during my initial research for a solution but have not actually worked with it. At the time (thinking back…), there seemed to be more issues with certmonger (more bugs than with sscep) and that SSCEP had been around a little longer. (The higher count of issues in certmonger through its development are very likely do to the deeper integration that they are trying to achieve.) Another reason for going with SSCEP was likely S (for “simple”) in SSCEP. The client was simple and I found decent documentation.
      Certmonger does appear to solve a number of the SSCEP shortcomings. I’ll have to pull it into the lab and see how works out. It looks like you have given me a topic for “LINUX Certificate Enrollment and Automated Renewal Using NDES Chapter 2”
      Thanks!

      1. Alex Bush says:

        Hi Jeff,

        I managed to get certmonger working with no issues. Below are the only Linux client configurations needed:

        [root@cd ~]# yum list certmonger | grep cert
        certmonger.x86_64 0.78.4-3.el7 @base
        [root@cd ~]# getcert add-scep-ca -c ad -u http://${ndesserver}/certsrv/mscep/mscep.dll
        [root@cd ~]# getcert request -c ad -k /etc/pki/node.key -f /etc/pki/node.cer -N cn=”`hostname -f`” -L ${ndeskey}
        [root@cd ~]# getcert list | tail -18
        Request ID ‘20170308104812’:
        status: MONITORING
        stuck: no
        key pair storage: type=FILE,location=’/etc/pki/node.key’
        certificate: type=FILE,location=’/etc/pki/node.cer’
        signing request thumbprint (MD5): BF59F452 CDE6269A 59594F3F 80D8EBCC
        signing request thumbprint (SHA1): 60A14C31 7988E024 ABF88702 52ACBE7D 8586C4EB
        CA: ad
        issuer: CN=xxxxxxxxx,DC=xxxxxxxxx,DC=xxxxxxxx,DC=xxxxxxxxx
        subject: CN=xxxxxxxxxxxxxxxxxx
        expires: 2019-03-08 10:38:13 UTC
        key usage: digitalSignature,keyEncipherment
        eku: iso.org.dod.internet.security.mechanisms.8.2.2
        certificate template/profile: IPSECIntermediateOffline
        pre-save command:
        post-save command:
        track: yes
        auto-renew: yes

        So it is definitely worth looking at this option.

        Thanks,

        Alex

        1. YeskeJA says:

          How automated were you able to get with it? Were you able to use Jeff’s script with your certmonger implementation?

          Is there any way to automated the NDES enrollment challenge password creation/sync with the Linux endpoints/certmonger?

          Will the NDES setup support an HTTPS Enrollment Site/Web Page?

      2. Ayaz Khan says:

        Hi Alex,

        Thanks a lot for posting such a good article. I have been looking for ADCS PKI integration with Linux for a long time but was not able to find any information, specifically for auto enrollment capability.

        Are you able to test out certmonger solution? As explained by Alex?
        Also your article talks about Linux boxes that are not joined to domain, what about Linux box that are joined to domain?

        Is there a way we can leverage Linux certmonger capabilities using Kerberos authentication interface? If not can we use SSCEP with Kerberos for domain joined boxes?

        Lastly can we utilize either certmonger or SSCEP certificate auto enrollment on Linux boxes and certutil function on windows for java keytool.

        Basically Java based apps use java keytool for TLS carts and there is no auto enrollment capability available to auto enroll / renew certs in keytool using Microsoft ADCS PKI.

        Any guidance to address above areas? My next problem is to address Dockers certificate management integration with ADCS…

        Please let me if there are any suggestions
        Thanks
        Ayaz

        1. Ayaz,
          Certmonger works great (as Alex indicated). I am working on posting version 2.0 of this article with a Certmonger based solution (which requires me to find more free time to finish it up 🙂
          At this time, I have not found a configuration that will allow me to leverage Kerberos authentication from a Linux client to a Windows based CA. Until that time, if that is a requirement, you may consider adding a FreeIPA Certificate Authority as a subordinate to the Windows CA. The idea is that if you want a fully integrated environment, you would:
          • Domain join your systems (using SSSD or Centrify).
          • Add a FreeIPA CA as a subordinate issuing authority to your PKI hierarchy (http://www.freeipa.org/page/PKI)
          • Configure your Linux clients to use Kerberos with Certmonger to authenticate the request for certificates.
          Regarding your question for integration with java keytool, I am not aware of a direct solution. I would recommend you look at automation. That could be done through a script, Desired State Configuration (DSC), Puppet or other mechanism. The automation would track the existing certificate issues to the system using SCEP or Windows Autoenrollment. When the certificate renews, automation would leverage keytool to import the certificate into the keystore and then go back to waiting for a certificate change.
          As far as Docker goes, it is on my roadmap…
          Thanks,
          Jeff

  4. Scott Tendering says:

    Very nice, Mr. Butte! 🙂

  5. Todd Mote says:

    Hi Jeff!

    Since February we were able to stand up an NDES server and working with some smart Linux folks we were able to create an RPM for the SCCM client that autoenrolls using SSCEP prior to install. It’s been working great! However today I discovered something that I can’t seem to track down. Last successful install using the RPM was Tuesday last week. (yes, it was a Patch Tuesday) the script in the rpm that we’ve been using since February to get certificates on Linux has stopped working. Using openssl to generate the csr’s and keys, and all that works fine, it breaks now when it tries to actually enroll for the certificate using the CA’s and openssl generated request. Here’s the last bit from sscep:

    sscep: finding attribute pkiStatus
    sscep: allocating 1 bytes for attribute
    sscep: pkistatus: FAILURE
    sscep: finding attribute failInfo
    sscep: allocating 1 bytes for attribute
    sscep: reason: Transaction not permitted or supported
    sscep: illegal size of payload

    I’ve arrived at something had to have changed on the NDES server with the way it processes requests after patching last week because the rpm hasn’t changed, it worked on Tuesday morning and hasn’t since the NDES server patched on Sunday. I was able to get a cert issued from Windows using certreq, but surprisingly only after including ‘requestType=SCEP’ in the inf. without it, all certreq did was create a csr. openssl doesn’t know that attribute and fails to generate a valid csr if i try to include it. Any ideas?

  6. Todd Mote says:

    As it turns out, the solution was due to a patch, just not to Windows. OpenSSL updated on our systems past 1.0.1f and the csr needed to have ‘string_mask = nombstr’ in the [req] section. It turns out the challenge password passed to openssl was getting encoded in UTF8 and NDES was choking on that. adding the section to not encode it returned our process to action!

    1. Todd Mote says:

      I just realized that my previous post doesn’t look like it was posted. The original script we’ve been using since February stopped working last week. We were getting returned from openssl and NDES that the csr was the wrong size and it fails with an invalid operation error.

  7. Andy says:

    Thanks for this great article. It is very useful, along with all these comments

    Incidentally, if anybody wants to try sscep easily on Debian/Ubuntu, PacketFence has it packaged in their Debian repository. See https://packetfence.org/support/faq/article/how-to-install-packetfence-on-debian.html
    It does not include mkrequest, but that is a just a shell script that can be easily downloaded from the sscep git repo

  8. Adam Millgate says:

    Hi Jeff,

    Great article – did you get anywhere with the version 2.0 you mentioned with the certmonger based solution?

    Many Thanks!

Skip to main content