Does your win 8.1 /2012 R2/win10 logon hang after a password change?

********** UPDATE **********

This is now fixed in the following updates:

For Windows 8.1, 2012 R2, 2012 install:

For Windows 10 TH2 build 1511 install:

******************************

 

Hi, Linda Taylor here, Senior Escalation Engineer from the Directory Services team in the UK.

I have been working on this issue which seems to be affecting many of you globally on windows 8.1, 2012 R2 and windows 10, so I thought it would be a good idea to explain the issue and workarounds while we continue to work on a proper fix here.

 

The symptoms are such that after a password change, logon hangs forever on the welcome screen:

clip_image002

 

How annoying….

 

The underlying issue is a deadlock between several components including DPAPI and the redirector.

 

Why does it happen?

So far we have seen this issue in the following circumstances (always after a password change/reset which is done somewhere other than the users machine – i.e. on the DC or in a portal)

1. If the user has a home drive which maps to a DFS like path for example: \\contoso.com\homefolders\user1

OR

2. If The following GPO is applied:

Computer configuration\administrative templates Windows Components\File Explorer\ “Set a default associations configuration file”

And the XML file is stored on a DFS based path. For example \\contoso.com\netlogon

OR

3. If you use GPP to map drives during logon to a DFS path like \\contoso.com\someShare

This issue happens due to a deadlock between DPAPI, Credential manager and the Redirector (RDR).

 

It goes like this…

1. When the user logs on, the profile service tries to map network home folder to \\contoso.com\…

2. To do this, we need to have a call created in RDR, and this requires a SMB session setup to dcname.contoso.com

3. The SMB session setup requires a security blob created to authenticate with the target server, which is the DC.

4. To create the security blob, Kerberos will check saved credentials by calling DPAPI.

5. DPAPI cannot decode the saved credential because the master key is not available because the user’s password is reset on DC, so it will need to query the DC for a master key. This requires a named pipe call to \\dcname.contoso.com\IPC$\protected_storage

6. To connect to this named pipe, RDR found it is the same as previous call in#2 (same fqdn DC name \\dcname.contoso.com) so now session setup is queued…

7. The Kerberos thread will hang forever, and the profile service will hang forever until a reboot.

8. After reboot, the user still cannot logon with the same symptom. (note: a different user CAN log on).

 

The problem occurs on client computers with Windows 8.1, Windows 10 and also Windows Server 2012 R2 (for example RDS scenario).

The problem occurs most frequently after an admin password reset which has occurred elsewhere (not the on the client computer to which the logon is happening) but it can also occur when the password change is not recent if the user is logging onto a machine where the cached credentials are old and they have changed their password on some other machine some time ago.

 

 

So, what can you do to get out of this problem?

There are several options for working around the issue:

1. If you have the mentioned policy – move the XML to some file share which is not DFS based and is not on a DC.

2. Assuming no-one wants to change home drive paths because there are many users and it’s a hassle, the other option is to disable Credential manager and clear the cached credentials.

When there are no entries for credential manager, there are no reasons to access the DC to refresh the keys. Hence the dead-lock.

This could be done by disabling this policy:

https://technet.microsoft.com/en-us/library/jj852185.aspx

3. Dig your way out by connecting to the machine remotely and deleting the entry under C:\Users\<username>\AppData\Roaming\Microsoft\Protect\[Problem Users SID].

Noteif there is a roaming profile you can also delete this entry on the profile server and it works. This is because the profile is downloaded before the drive mapping takes place.

4. Another (not so great) option is to boot the PC without any network cable and log on with the old password. Then connect it back.

Some history:

This issue also has a long history and there are other variant’s of this deadlock which were fixed before. See below list of related fixes:

 

Related previous Fixes for Windows 8.1 an Windows 2012 R2:

Windows 8.1 & 2012 R2 KB#

 

X64 file versions for dpapisrv.dll and lsasrv.dll

 

https://support.microsoft.com/en-us/kb/3101183You can’t log on to a domain-joined computer in Windows 8.1 or Windows Server 2012 R2”

 

Released in October 2015.

 

Note: this is the latest but does not resolve this variant of the issue.

 

DPAPISRV.dll 6.3.9600.18088.

 

LSASRV.dll version 6.3.9600.18088.

 

3038562              Cannot access DPAPI data after an administrator resets your password on a Windows Server 2012 R2-based domain controller

 

http://support.microsoft.com/kb/3038562/EN-US

 

Dpapisrv.dll

 

6.3.9600.17707

 

Lsasrv.dll

 

6.3.9600.17415

 

 

The pre-requisite to 3101183 and 3038562   is April update:

 

2919355              Windows RT 8.1, Windows 8.1, and Windows Server 2012 R2 Update: April 2014

 

http://support.microsoft.com/kb/2919355/EN-US

 

This contains win8.1 version of KB2927267 https://support.microsoft.com/en-us/kb/2927

 

267You cannot log on to Windows after the admin changes your password”

 

Updates LSASRV.dll 6.3.9600.17042

 

There are no previous fixes for Windows 10.

 

There are some related past fixes for similar deadlocks in windows 8 also:

Windows 8 KB# X64 file versions for dpapisrv.dll and lsasrv.dll
3084956              You can‘t log on to a domain-joined computer in Windows 8 or Windows Server 2012

http://support.microsoft.com/kb/3084956/EN-US– Released September.

(NOTE: This is the windows 8 equivalent of KB3101183.)

X64 file versions:

DPAPISRV.dll 6.2.9200.21645 and LSASRV.dll  Version 6.2.9200.21582

KB3049843 https://support.microsoft.com/en-us/kb/3049843 – You cannot access DPAPI data after an administrator resets your password on a Windows Server 2012-based domain controller Dpapisrv.dll         6.2.9200.21442
2927267: https://support.microsoft.com/en-us/kb/2927267 Lsasrv.dll 6.2.9200.20931

Finally, Microsoft is actively working on fixes for Windows 8.1 / WS 2012 R2 and Windows 10 TH2 for the current issue described above. I will share the release dates and article #’s once known. If you experience this issue please ensure you have all of the above fixes in place and use the workarounds noted above and keep an eye out on updates to this blog.

Linda Taylor.