Post Decryption of TLS/SSL traffic


The Network Monitor Decryption expert was a very popular tool, so we decided to integrate it into Message Analyzer. To accompany this blog, I’ve also created this video.

Today you can capture the traffic before it is encrypted using the Unencrypted HTTPS scenario from the Quick Trace menu. This requires you install the Fiddler Core components, which is how we enable this scenario. However, there are some limitations and scenarios that it doesn’t cover, like non HTTP traffic. And not all clients and capturing scenarios work with Fiddler. So another other option is to decrypt after the fact.

Post decryption has its challenges too. You need the private cert and password, which might not always be easily available. However, one nice mitigation is that the person who does have access to the certificate can always save the results, filtered down, and send only that data he wants you to see. Also, we haven’t hooked up every parser yet, however for some cases it’s a simple change and something we’ll extend moving forward.

Steps to Decrypt

The process to decrypt is pretty simple. The idea is that you supply the certificate(s) (pfx) and password(s) using the File->Options->Decryption dialog. We’ll remember the path to the certificate for future session, but you’ll have to remember to make sure the check mark is set, and the password entered if you want to use them.

clip_image002

Next you open the trace. If you’ve already opened the trace, then you need to force it to reparse. This can be done by hitting the Edit button from the Session section. Then, Select Full Edit, and hit the Apply button.

clip_image003

clip_image005

At this point the message have been decrypted. But you can get more help to understand what was decrypted, what failed, and potentially some help as to why it failed by opening the Decryption Tool from the Tool Windows button on the ribbon.

clip_image006

This will open a decryption window which shows you the results for all successful and failed decrypted TLS/SSL sessions. Currently the failure cases need to be improved. We have some we know about already, but with your help we can improve the error reported.

clip_image008

By selecting a row in the decryption tool, it will select all the associate messages in the Analysis Grid, and any other opened visualizer. For instance you can see message 63 get selected, among others that are hidden from view. And looking in the Message Stack view, you can see TLS with HTTP above, proving that decryption has occurred.

clip_image010

Analysis Grid Tip

When using decryption it is helpful to organize the data so that all TLS messages in the same conversations are together. You can do this by selecting the Network Conversations Tree and Process ID from the View Layout in the Ribbon. This will organize the data using groupings.

 clip_image011

By default it includes DataSource, which helps sort data from different remote machines or log files. However we only have one source so we can remove it. We can also remove the process ID, only because we are going to expand all the groups, so one less group to deal with.

clip_image013

Now we expand all groups, left to right, for Network and Transport by right clicking each group. This will make all the messages visible, which is important because it won’t show as selected if it’s not visible.

clip_image014

Now when you select a row from the decryption tool window, we see all the related traffic adjacent to each other. For instance I can click on the first decryption message for the failed case, which jumps to that section of the trace. Now it’s easy to see that the first related message in this trace is TLS payload, which is not expected.

clip_image016Compared to a successful trace, we can see that we are missing the TLS handshake, which is necessary to decrypt the traffic as it has the Cert info, keys and such. This is why this TLS conversation failed to decrypt.

clip_image018

Saving the Trace

If you want to share the data, you can save the trace as a MATP, which standards for Message Analyzer Trace Parsed. This is the only way to preserve the unencrypted data. Exporting as a .cap file will lose the parsing, so that is not an option. You can see this is a convenient way to transport a trace from a customer, who can access the certificate but can’t share it with you.

Supported Configurations

TLS/SSL has many versions and combinations of Cipher suites and protocols. This table summarizes what we support today. Feel free to send us your most important combinations using the feedback mechanism.

Cipher suite

Supported Protocols

Supported transports

Supported Application Protocols for decryption by default

TLS_RSA_WITH_AES_128_CBC_SHA256,

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS 1.2

TCP and UDP

Over TCP : HTTP, LDAP,RDP,SOAP and ADWS

Over UDP : RDP Protocols

TLS_RSA_WITH_NULL_MD5,

TLS_RSA_WITH_NULL_SHA,

TLS_RSA_WITH_RC4_128_MD5,

TLS_RSA_WITH_RC4_128_SHA

TLS 1.2,

TLS 1.1,

TLS 1.0,

SSL 3.0

TCP and UDP supported for TLS 1.0 to TLS 1.3

Only TCP for SSL 3.0

Over TCP : HTTP, LDAP,RDP,SOAP and ADWS Over UDP : RDP Protocols

TLS_RSA_WITH_3DES_EDE_CBC_SHA,

TLS_RSA_WITH_AES_128_CBC_SHA,

TLS_RSA_WITH_AES_256_CBC_SHA

TLS 1.2,

TLS 1.1,

TLS 1.0

TCP and UDP

Over TCP : HTTP, LDAP,RDP,SOAP and ADWS Over UDP : RDP Protocols

SSL_RSA_WITH_RC4_128_MD5

SSL3.0

TCP

Over TCP : HTTP, LDAP,RDP,SOAP and ADWS

Conclusion

Post decryption can be a useful tool in your arsenal. Nothing is more frustrating than data you can’t see. As mentioned before, there will be many error cases that we’ll have to catch specifically. So feel free to ping us on the forums so we can gather the information necessary to fix the problem moving forward. And of course the feedback (clip_image019) mechanism is a great simple way to let us know how to improve Message Analyzer.

Comments (31)

  1. Paul E Long says:

    You need the private server Certs to do post decryption. I can't be sure, but I don't think that will work in your scenario unless you own the server too. Do you own the server piece?

    Another option is to try the HTTPS scenario which can capture the traffic unencrypted for IE traffic. Is that an option?

  2. Paul E Long says:

    You need the server side cert, as it has the secrete required to decrypt the traffic. There's no away around that as that is how the TLS/SSL protocol was designed.

    And yes, the HTTPS scenario is the one that users Fiddler. Granted it requires a 3rd party component, but it's the only solution for now.

  3. Paul E Long says:

    We only support the cipher suites listed in the blog. But servers can be configured to use a range of different cipher suites. Assuming this message is showing up in the Decryption tool window, then it seems a limitation of the tool. If that's the case,
    you can either try to configure the server to use only cipher suites we can decrypt, or send us the cipher suite and perhaps it's one we can try to extend if possible.

    If the error is coming up when you add the Certificate, let me know as I'll have to investigate further.

    Paul

  4. Paul E Long says:

    With Message Analyzer you can have much of what the fiddler UI gives you. In fact, our default mode is to defragment on only show you the upper layers. I've added some links to some other blogs that explain how to switch to the TCP layer. But for instance,
    if you filter on HTTP, and then select the "HTTP" View Layout, you can focus on the HTTP layer. You can also click on a payload, and see the rendered data (Text/Picture/XML) in the Field Data tool window. The TimeElapsed column tells you the performance of
    each transaction, and the ReponseTime column can be added to understand the server performance.

    BTW, we also have the Unencrypted HTTP trace scenario (File->Quick Trace->HTTP), which uses the Fiddler API (you install separately) to capture data just like fiddler does.

    http://blogs.technet.com/b/messageanalyzer/archive/2013/09/25/message-analyzer-why-so-different-from-network-monitor.aspx
    http://blogs.technet.com/b/messageanalyzer/archive/2013/08/14/viewpoints-osi-model-and-apstndp.aspx

  5. Austin McCollum says:

    Great post! Very timely too, as I've been having discussions and demos with my customers on this technique.

    As you mentioned, the only way to preserve the decrypted frames is to save the trace as an MATP file since .cap file will lose the decryption. I noticed similar behavior with Wireshark. I find that I'm still using the Fiddler UI to analyze unencrypted HTTP
    traffic though. Maybe I need to spend more time in Message Analyzer, but I'd love to have the best of both worlds, analyzing WebIO protocol level information next to TCP layers in Message Analyzer and then switching to Fiddler for the HTTP portions. Is there
    a way with FiddlerCore to save the HTTP decrypted packets as an .saz? or other format that could be imported into Fiddler? In parallel, I'm investigating whether it's possible to use the private key of a custom cert in Fiddler to decrypt the packets.

    In any case, thanks for all the hard work on Message Analyzer! Each new release is becoming exponentially easier to use with the improved performance and blog posts like this to demonstrate its power.

    -austinmc

  6. Austin McCollum says:

    UPDATE: looks like there is a way to decrypt HTTP portion of the traffic in Fiddler with a custom cert.
    http://www.telerik.com/forums/valid-official-certificate

    This will likely be how I operate in the two worlds until I can be more efficient in Message Analyzer for HTTP traffic analysis.

  7. Paul E Long says:

    I checked and it turns out the DH based keys are harder to support. The encryption key generated independently with the private key contained in the certificate. The user would have to input a key for each individual session. I think the best option is
    to limit your cipher suites on the server or client if you need to troubleshoot a problem and decrypt the traffic.

  8. Paul E Long says:

    If you upgraded, we retain your layout, so you might not see the exact same thing. To reset the layout you can delete ("C:UsersYOURLOGONNAMElAppDataRoamingMicrosoftMessageAnalyzerapp.DockLayoutConfiguration.cfg") and restart. If there's something
    else missing, tell me what you can't find.

    As for your issue, I think you might a have found a bug. Can you contact me through the blog (Email Blog Author) so we can start an email conversation and then you could send me the trace?

    Paul

  9. Paul E Long says:

    Yes, I think changing to TLS_RSA_WITH_AES_128_CBC_SHA should let you decrypt the traffic using the private server certificate.

    Paul

  10. Paul E Long says:

    When TLS negotiates it's server hello contains the cipher set that is chosen from a list provided in the client hello. You can change the client side and/or server side to find one that is supported. I've copied one from an example I have from the details
    window (which is TSV separated if you want to paste back into excel to get a better view).

    The relevant line is:
    cipher_suite TLS_RSA_WITH_AES_128_CBC_SHA(47)

    Name Value
    records [Handshake: [Server Hello],ChangeCipherSpec,Handshake(Encrypted)]
    [0] Handshake: [Server Hello]
    type handshake(22)
    version TLS 1.0
    length 81
    fragment [Server Hello]
    [0] Server Hello
    msg_type server_hello(2)
    length 77
    body ServerHello{server_version=TLS 1.0,random=Random{gmt_unix_time=1284137370,random_bytes=binary[46,189,117,252,170,30,24,33,114,227,14,251,183,231,183,215,121,66,170,241,173,249,35,100,208,196,219,139]},session_id=SessionID{length_in_bytes=32,session_id=binary[21,32,0,0,201,82,252,229,136,201,225,146,143,51,204,15,162,17,123,182,25,56,108,220,203,233,79,113,114,75,14,191]},cipher_suite=47,compression_method=0,extensions_length_in_bytes=5,extensions=[extension_type:
    renegotiation_info]}
    server_version TLS 1.0
    random Random{gmt_unix_time=1284137370,random_bytes=binary[46,189,117,252,170,30,24,33,114,227,14,251,183,231,183,215,121,66,170,241,173,249,35,100,208,196,219,139]}
    session_id SessionID{length_in_bytes=32,session_id=binary[21,32,0,0,201,82,252,229,136,201,225,146,143,51,204,15,162,17,123,182,25,56,108,220,203,233,79,113,114,75,14,191]}
    cipher_suite TLS_RSA_WITH_AES_128_CBC_SHA(47)
    compression_method NULL(0)

  11. WizardLizard says:

    I have updated MA to v 1.1. I do not see the same UI that you show in this blog or the video, so I'm not sure what version you are using. I am trying to look at FTPS traffic to attempt to explain to our network guys what the firewall is doing to block
    the data connection of FTP from being established.

    First off, as soon as the AUTH TLS command is recognized all the packet trace comes up with these errors:

    ParsingtActor: OpnGenerated.FTP_actor_FTPOverTCP+FTPOverTCP
    Exception: Index was outside the bounds of the array.
    Hash Code: aabb07fce303cb7b569192831d7f2ca0
    Call Stack:
    at Microsoft.Opn.Runtime.Values.ArrayValue`1.CheckIndex(Int32 index)
    at OpnGenerated.FTP_CG.Request3NoParam_BinaryCodecCompiler.__Decode(StreamValue sv, Object[] args)
    at OpnGenerated.FTP_CG.CommandToServer_BinaryCodecCompiler.__Decode(StreamValue sv, Object[] args)
    at OpnGenerated.FTP.CommandToServer.__BinaryDecoder(StreamValue p0, Object[] p1)
    at OpnGenerated.FTP_actor_FTPOverTCP.FTPOverTCP.__OnAcceptsVirtualDataSegment_2(MessageEventArgs __args)
    at Microsoft.Opn.Runtime.Actors.MessageEvent.Execute(MessageEventHandler handler, MessageEventArgs args)

    I was assuming this was due to encryption. Upon supplying the cert and trying take a guess at how the version of MA I am using differs from the one you demonstrate, I see no decryption.

  12. WizardLizard says:

    I actually just installed the 1.1 version on the FTPS server where I captured the session, so my layout of the UI is just default I'd think. I emailed you if you want the session trace.

  13. WizardLizard says:

    What does it mean when it says 'The cipher suite is unsupported'? I was tracing a short session between a web appliance and my endpoint using Chrome. This web appliance is using a CA-signed cert which I had converted to a .pfx file from a .crt from the
    CA.

  14. WizardLizard says:

    128 bit; TLS 1.2; AES 128 CBC; SHA1; ECDHE_RSA
    🙂

  15. Natan1 says:

    Hi, Pls can you advice how to decrypt RTCP packet data for MS Lync call.

  16. Tim Robinson says:

    I tried but failed using this to decrypt the traffic generated when saving files to my OneDrive for Business. Perhaps you can tell me what I'm doing wrong. Here's what I did:

    1. Exported all TRCA certs to one pfx file and then added that files to MA. I wasn't sure which one to use and this seem like the fastest way to get this working. I'm guessing this is the problem.

    2. Ran a trace using the Local Network Interface, and then saved a Word document to my OneDrive for Business.

    The capture generated 29 TLS messages, though the Decryption tool only reports back 17 encrypted messages, none of which could be decrypted.

  17. Tim Robinson says:

    No, I don't own the server piece. By the HTTPS scenario do you mean the one that uses the Fiddler web proxy? If so, I did that, and it works fine. But I'm trying to stick with a Microsoft-only scenario for now. So there's no way to decrypt the messages
    even when it's coming from my own machine? If not, I'll have to see about obtaining a test server.

  18. Anonymous says:

    With the release of Message Analyzer 1.2 , I thought it’s a good time to discuss the vocabulary. I’ve

  19. jayan says:

    Type TimeStamp ProtocolVersion SourcePort DestinationPort SourceAddress DestinationAddress Decrypted Message Count Undecrypted Message Count Message

    Info 4/16/2015 11:05:42 PM N/A 51889 443 2002:5A00:3C:0:0:0:5A00:3C 2002:5A00:4:0:0:0:5A00:4 0 1 The cipher suite is unsupported

  20. jayan says:

    I am getting "cipher suite is unsupported" and I see TLS 1.0i nthe below details window output . Is that not supported or am I missing something

    Name Value Bit Offset Bit Length Type
    [0] Handshake: [Client Key Exchange] 0 856 TLS.RecordLayer
    records [Handshake: [Client Key Exchange],ChangeCipherSpec,Handshake(Encrypted)] 0 1328 ArrayValue`1

    type handshake(22) 0 8 ContentType
    version TLS 1.0 8 16 TLS.ProtocolVersion
    major 3 8 8 Byte
    minor 1 16 8 Byte
    length 102 24 16 UInt16
    fragment [Client Key Exchange] 40 816 ArrayValue`1
    [0] Client Key Exchange TLS.Handshake
    msg_type client_key_exchange(16) 40 8 HandshakeType
    length 98 48 24 UInt32
    body binary[97,4,198,12,55,116,15,211,144,40,137,36,162,164,130,78,58,91,196,130,106,145,54,98,49,21,192,44,206,159,189,40,46,131,11,61,25,42,22,179,135,41,197,174,235,185,218,46,219,140,212,248,223,127,92,252,97,121,203,155,0,247,106,247,135,214,183,65,73,63,94,150,80,29,136,250,30,134,176,1,191,3,97,12,71,82,226,94,222,14,79,80,93,226,243,46,170,171]
    72 784 BinaryValue
    [1] ChangeCipherSpec 856 48 TLS.RecordLayer
    type change_cipher_spec(20) 856 8 ContentType
    version TLS 1.0 864 16 TLS.ProtocolVersion
    major 3 864 8 Byte
    minor 1 872 8 Byte
    length 1 880 16 UInt16
    fragment ChangeCipherSpec{type=1} 896 8 TLS.ChangeCipherSpec
    type change_cipher_spec(1) 896 8 ChangeCipherSpec_Type
    [2] Handshake(Encrypted) 904 424 TLS.RecordLayer
    type handshake(22) 904 8 ContentType
    version TLS 1.0 912 16 TLS.ProtocolVersion
    major 3 912 8 Byte
    minor 1 920 8 Byte
    length 48 928 16 UInt16
    fragment binary[61,84,60,119,227,187,241,220,3,250,224,252,214,208,88,21,42,112,201,211,183,186,135,232,170,133,135,61,131,26,88,139,119,167,9,138,206,169,184,127,254,186,139,248,249,47,14,16] 944 384 BinaryValue

  21. Jayan says:

    Name Value Bit Offset Bit Length Type
    compression_method NULL(0) 624 8 CompressionMethod
    records [Handshake: [Server Hello, Certificate, Certificate Status, Server Key Exchange, Server Hello Done]] 0 33584 ArrayValue`1

    [0] Handshake: [Server Hello, Certificate, Certificate Status, Server Key Exchange, Server Hello Done] 0 33584 TLS.RecordLayer

    type handshake(22) 0 8 ContentType
    version TLS 1.2 8 16 TLS.ProtocolVersion
    major 3 8 8 Byte
    minor 3 16 8 Byte
    length 4193 24 16 UInt16
    fragment [Server Hello,Certificate,Certificate Status,Server Key Exchange,Server Hello Done] 40 33544 ArrayValue`1

    [0] Server Hello TLS.Handshake
    msg_type server_hello(2) 40 8 HandshakeType
    length 81 48 24 UInt32
    body ServerHello{server_version=TLS 1.2,random=Random{gmt_unix_time=1428942709,random_bytes=binary[139,235,202,214,208,107,134,210,51,76,103,248,194,68,139,150,141,244,250,107,57,221,50,107,187,191,206,63]},session_id=SessionID{length_in_bytes=32,session_id=binary[170,51,0,0,157,91,8,61,55,243,52,125,83,147,37,136,232,23,166,127,99,118,181,44,209,238,120,99,48,135,228,45]},cipher_suite=49192,compression_method=0,extensions_length_in_bytes=9,extensions=[extension_type:
    status_request,extension_type: renegotiation_info]} 72 648 TLS.ServerHello
    server_version TLS 1.2 72 16 TLS.ProtocolVersion
    major 3 72 8 Byte
    minor 3 80 8 Byte
    random Random{gmt_unix_time=1428942709,random_bytes=binary[139,235,202,214,208,107,134,210,51,76,103,248,194,68,139,150,141,244,250,107,57,221,50,107,187,191,206,63]} 88 256 TLS.Random

    session_id SessionID{length_in_bytes=32,session_id=binary[170,51,0,0,157,91,8,61,55,243,52,125,83,147,37,136,232,23,166,127,99,118,181,44,209,238,120,99,48,135,228,45]} 344 264 TLS.SessionID

    cipher_suite TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(49192) 608 16 IANA.CipherSuite

    extensions_length_in_bytes 9 632 16 UInt16
    extensions [extension_type: status_request,extension_type: renegotiation_info] 648 72 ArrayValue`1

    [1] Certificate TLS.Handshake
    [2] Certificate Status TLS.Handshake
    [3] Server Key Exchange TLS.Handshake
    [4] Server Hello Done TLS.Handshake

    Thanks for responding Paul
    So here I see cipher_suite TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(49192) so should I change it to cipher_suite TLS_RSA_WITH_AES_128_CBC_SHA(47) on client and server ?

  22. Jayan says:

    Thanks Paul 🙂

  23. Konnan says:

    Is it complicated to change the supported cipher suite ?

    We are currently using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(49192) and I get the infamous "The cipher suite is unsupported".

  24. Norlando says:

    I think we're going to need support for more advanced cipher suites like TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256.
    DH-based suites are becoming more preferred. It would be great if you could support all the common ones used with TLS 1.2.

  25. Paul E Long says:

    From what we understand, this cipher technique is designed so that it can't be decrypted. The encryption key is independently created in memory and not persisted. There's more information here:
    http://en.wikipedia.org/wiki/Perfect_forward_secrecy. I would like to be wrong, but it doesn't seem possible to decrypt after the fact. Instead, we'll have to rely on specific providers. For
    instance, I recently found out we can use the LDAP client provider to log decrypted packets in the clear.

  26. Alun Jones says:

    A team here spent several hours with Wireshark and Message Analyzer, trying to figure out why they were unable to decrypt a stream. Particularly irritating was when they extracted part of the stream and saved it to a pcap file, they couldn't decrypt that
    partial stream, despite having been able to decrypt it when part of the larger capture.

    The Decryption window in Message Analyzer said "Certificate not found", which was clearly ludicrous, as we had the certificate sitting right there in the Options / Decryption window.

    I got to it and eventually figured out what was happening, and how the error message, while correct, was completely unhelpful.

    The key here is that the segment we trimmed out was a session resumption, not a full initial handshake. Session resume relies on previously-agreed keys and doesn't carry anything that would allow a protocol analyser to decrypt the traffic.

    You can recognize a resumed session by the fact that the ClientHello contains a valid session_id component, and the ServerHello isn't followed by a Certificate message. So, that's why the message says "Certificate not found" - this is an indication, not that
    it can't find the certificate file, but that it can't find the Certificate message in the SSL handshake.

    It'd be really nice if next versions of MSMA could change this error to be more helpful - something like "Certificate message not in handshake - cannot decrypt resumed session"

  27. Paul E Long says:

    We do have an open bug in our database to address this issue and identify more error cases. At this point it's just a question of priority. But we'll take this post as a +1 🙂

    Interestingly enough, I was just working on some patterns that identify this condition. So we'll have some more ways to get even more detail about TLS sessions.

  28. Andrey Kovalev says:

    TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA is required for RDP. I got "The cipher suite is unsupported" trying decode RDP with all updated 2012R2 servers.

  29. Paul E Long says:

    It is not possible to decrypt the ECDHE (elliptical) types of cipher suites. The only solution I know today is to configure the server to use a cipher suite we support.

  30. Andrey Kovalev says:

    How can I disable TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA for RDP? I already disabled TLS 1.2 and TLS 1.1. Now I got TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA with TLS 1.0.

  31. Andrey Kovalev says:

    Yesss, it works! Thank you.

    =====
    1. At a command prompt, enter gpedit.msc. The Group Policy Object Editor appears.
    2. Expand Computer Configuration, Administrative Templates, Network, and then click SSL Configuration Settings.
    3. Under SSL Configuration Settings, click the SSL Cipher Suite Order setting.
    4. In the SSL Cipher Suite Order pane, scroll to the bottom of the pane.
    5. Follow the instructions labeled How to modify this setting.

    It is necessary to restart the computer after modifying this setting for the changes to take effect.
    =====

    I remove all ciphers before TLS_RSA_WITH_AES_128_CBC_SHA, reboot and now Message Analyzer decrypt RDP.

Skip to main content