Use the new PowerShell cmdlet ConvertFrom-String to parse KLIST Kerberos ticket output


Tired of hacking away at RegEx and string functions to parse text? This post is for you!

ConvertFrom-String

imageIn yesterday’s post we reviewed a simple example of the new PowerShell 5.x Convert-String cmdlet. Today we are going to study a complex example with ConvertFrom-String. This cmdlet was first documented here in a PowerShell team blog post. To make a long story short, the Microsoft Research team invented some fuzzy logic for parsing large quantities of text using sample data. The technology learns and adapts as it parses. If the output does not look correct, you simply add more examples demonstrating the variation that was not matched.

The trouble with KLIST

I was doing some Kerberos research lately, and I wanted to parse the output of KLIST to be more PowerShell friendly. KLIST displays current Kerberos tickets on a machine, but it is flat text. We will use ConvertFrom-String to turn it into beautiful, sortable, filterable object data like this in Out-Gridview:

image

First, let’s take a look at a sample of KLIST output:

Current LogonId is 0:0x3f337

Cached Tickets: (12)

#0>	Client: Administrator @ CONTOSO.COM
	Server: krbtgt/CONTOSO.COM @ CONTOSO.COM
	KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
	Ticket Flags 0x60a10000 -> forwardable forwarded renewable pre_authent name_canonicalize 
	Start Time: 8/29/2016 16:06:22 (local)
	End Time:   8/30/2016 2:06:21 (local)
	Renew Time: 9/5/2016 16:06:21 (local)
	Session Key Type: AES-256-CTS-HMAC-SHA1-96
	Cache Flags: 0x2 -> DELEGATION 
	Kdc Called: 2012R2-DC.contoso.com

#1>	Client: Administrator @ CONTOSO.COM
	Server: krbtgt/CONTOSO.COM @ CONTOSO.COM
	KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
	Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize 
	Start Time: 8/29/2016 16:06:21 (local)
	End Time:   8/30/2016 2:06:21 (local)
	Renew Time: 9/5/2016 16:06:21 (local)
	Session Key Type: AES-256-CTS-HMAC-SHA1-96
	Cache Flags: 0x1 -> PRIMARY 
	Kdc Called: 2012R2-DC

#2>	Client: Administrator @ CONTOSO.COM
	Server: host/2012R2-MS.contoso.com @ CONTOSO.COM
	KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
	Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize 
	Start Time: 8/29/2016 16:12:25 (local)
	End Time:   8/30/2016 2:06:21 (local)
	Renew Time: 9/5/2016 16:06:21 (local)
	Session Key Type: AES-256-CTS-HMAC-SHA1-96
	Cache Flags: 0 
	Kdc Called: 2012R2-DC.contoso.com
...
...

One line of code!

The data contains multiple objects with multiple properties. Unfortunately that is all plain text. To parse this using regular expressions or string functions would be time-consuming and error-prone. Here is the ConvertFrom-String syntax used to produce the screenshot at the top of the article:

klist | ConvertFrom-String -TemplateFile .\template.txt | Out-GridView

Template magic

Wow! Are you kidding?! Nope. That’s it. But… wait… what’s in that template.txt file? Ah. That is the magic. Let’s take a look:

Current LogonId is 0:0xb4ffc69

Cached Tickets: (4)

#{[int]ID*:0}>	Client: {Client:Administrator @ CONTOSO.COM}
	Server: {Server:krbtgt/CONTOSO.COM @ CONTOSO.COM}
	KerbTicket Encryption Type: {KerbTicketEncryptionType:AES-256-CTS-HMAC-SHA1-96}
	Ticket Flags {TicketFlags:0x60a10000} -> {TicketFlagsEnum:forwardable forwarded renewable pre_authent name_canonicalize} 
	Start Time: {[datetime]StartTime:8/29/2016 16:06:22} (local)
	End Time:   {[datetime]EndTime:8/30/2016 2:06:21} (local)
	Renew Time: {[datetime]RenewTime:9/5/2016 16:06:21} (local)
	Session Key Type: {SessionKeyType:AES-256-CTS-HMAC-SHA1-96}
	Cache Flags: {CacheFlags:0x1} -> {CacheFlagsEnum:PRIMARY} 
	Kdc Called: {KdcCalled:2012R2-DC.contoso.com}

#{[int]ID*:1}>	Client: {Client:Administrator @ CORP.CONTOSO.COM}
	Server: {Server:krbtgt/CONTOSO.COM @ CONTOSO.COM}
	KerbTicket Encryption Type: {KerbTicketEncryptionType:AES-256-CTS-HMAC-SHA1-96}
	Ticket Flags {TicketFlags:0x40e10000} -> {TicketFlagsEnum:forwardable renewable initial pre_authent name_canonicalize} 
	Start Time: {[datetime]StartTime:8/29/2016 16:06:21} (local)
	End Time:   {[datetime]EndTime:8/30/2016 2:06:21} (local)
	Renew Time: {[datetime]RenewTime:9/5/2016 16:06:21} (local)
	Session Key Type: {SessionKeyType:AES-256-CTS-HMAC-SHA1-96}
	Cache Flags: {CacheFlags:0x2} -> {CacheFlagsEnum:DELEGATION} 
	Kdc Called: {KdcCalled:2012R2-DC.contoso.com}

#{[int]ID*:2}>	Client: {Client:Administrator @ CORP.NA.ALPINESKIHOUSE.COM}
	Server: {Server:host/2012R2-MS.contoso.com @ CONTOSO.COM}
	KerbTicket Encryption Type: {KerbTicketEncryptionType:AES-256-CTS-HMAC-SHA1-96}
	Ticket Flags {TicketFlags:0x40a10000} -> {TicketFlagsEnum:forwardable renewable pre_authent name_canonicalize} 
	Start Time: {[datetime]StartTime:8/29/2016 1:12:25} (local)
	End Time:   {[datetime]EndTime:8/30/2016 2:06:21} (local)
	Renew Time: {[datetime]RenewTime:9/5/2016 1:06:21} (local)
	Session Key Type: {SessionKeyType:AES-256-CTS-HMAC-SHA1-96}
	Cache Flags: {CacheFlags:0} 
	Kdc Called: {KdcCalled:2012R2-DC}

#{[int]ID*:3}>	Client: {Client:Administrator @ CONTOSO.COM}
	Server: {Server:RPCSS/2012R2-MS.contoso.com @ CONTOSO.COM}
	KerbTicket Encryption Type: {KerbTicketEncryptionType:RSADSI RC4-HMAC(NT)}
	Ticket Flags {TicketFlags:0x40a10000} -> {TicketFlagsEnum:forwardable renewable pre_authent name_canonicalize} 
	Start Time: {[datetime]StartTime:12/29/2016 16:12:25} (local)
	End Time:   {[datetime]EndTime:12/30/2016 12:06:21} (local)
	Renew Time: {[datetime]RenewTime:12/5/2016 16:06:21} (local)
	Session Key Type: {SessionKeyType:RSADSI RC4-HMAC(NT)}
	Cache Flags: {CacheFlags:0} 
	Kdc Called: {KdcCalled:2012R2-DC.contoso.com}

To create the parsing template we begin by capturing the KLIST output to a text file like this:

KLIST > template.txt

Let’s break down the formatting in the template file:

  • Basic property parsing looks like this: {[datatype]PropertyName:DATA}. To extract meaningful data from flat text, you surround sample output data values with this syntax.
  • The datatype is optional, if you want to use strings for everything. I chose to use [int] and [datetime] for intelligent sorting and filtering of property data.
  • For multi-object parsing in a file like this, the first property on the object gets the asterisk * after the property name. In this case I chose ID to indicate a new record.

For each data instance in the template file we add some formatting to identify the properties we want to extract. We do not need to keep every instance of the certificates in the list, only enough to demonstrate different sample values. Study the example template above to find these differences:

  • Cache Flags
  • Cache Flags Enum (does not appear in every record)
  • KerbTicket Encryption Type
  • Session Key Type
  • Ticket Flags
  • Dates (single-digit vs. double-digit days/months)
  • Server short name vs. FQDN (fully qualified domain name)
  • FQDNs of varying dotted patterns
  • Etc.

Mash the easy button

Does that template.txt look difficult? It might. But it is actually not that bad, and it is MUCH easier than trying to write a RegEx or string functions to parse this. Actually, the fuzzy logic behind the cmdlet does exactly that. It studies each sample to identify differences (string length, spaces, capitalization, etc.) and then dynamically generates parsing code. Some consider this to be like AI (artificial intelligence) or ML (machine learning).

The template above took a couple hours of tweaking and experimenting. It works 95% of the time. Your mileage may vary. For example, you may need to tweak the template for international date formats or different type values.

Your turn…

I decided to keep this blog post short and leave other things for you to discover with the cmdlet. I have shown you one way to use it. There is more! Get-Help is your friend.

Now take this and go parse some of your own flat text use cases. Use the comments below to share your challenges and victories. Enjoy!

Comments (5)

  1. Turbomcp says:

    Awesome
    Thanks

  2. So … I need to look up srv records on a machine which isn’t new enough to have the PowerShell DNS cmdlets,. 10 Minutes after re-reading this I have an NSLookup parser

    The first line is odd because (a) I have multiple lines of input for NSlookup in a single string and (b) NSlookup generates errors which I want to discard (hence capturing the result in a variable, and sending errors and output to null)

    Invoke-Command -ScriptBlock {“set type=srv`r`n_autoDiscover._tcp.ao.foo.com`r`n_autoDiscover._tcp.Contoso.com`r`n_autoDiscover._tcp.Contoso.lu`r`n_autoDiscover._tcp.at.foo.com`r`n_autoDiscover._tcp.az1.foo.com`r`nquit” |
    nslookup } -OutVariable Result 2>&1 | out-null

    Now I have my result in $Result.
    $Result | ConvertFrom-String -TemplateFile .\NslookupSrvTemplate.txt | ft -AutoSize

    and Ta da!

    The template above was all I needed to see how to mark up a successful result. Trial and error says it needs two results to work properly.

  3. Gixxer says:

    Great post, will try this asap to see it in action. Thx.

  4. J Dunlap says:

    This is about 25x as slow as using grep and awk to pull just the values you need, in my measurements… are speed improvements in the works?

    PS ~> (0..10 | %{(measure-command {get-klist}).TotalMilliseconds} | measure -Average).Average
    3955.98753636364
    PS ~> (0..10 | %{(measure-command { klist 2>&1 | grep -m1 krbtgt | awk ‘{print $3, $4}’}).TotalMilliseconds} | measure -Average).Average
    155.061109090909

Skip to main content