Userenv 1054 events as a result of time-stamp counter drift on Windows Server 2003 guests running in Hyper-V

Hello everyone, Brian here.

I was working with a colleague on an issue in which the customer was receiving the following Userenv event in their application event logs:

(Event ID 1054)
Windows cannot obtain the domain controller name for your computer network. (An unexpected network error occurred.). Group Policy processing aborted.

The customer explained to us that if he removes one of the Hyper-V virtual processors from his Windows Server 2003 Guest, the issue goes away. Based on this statement we asked the customer to gather a userenv log while forcing a group policy refresh with the additional virtual processor enabled, and this what we saw during the initial ping test before we process group policy:

USERENV(15c.858) 15:55:09:080 PingComputer: First time: 2069
USERENV(15c.858) 15:55:09:080 PingComputer: Second time: 2069
USERENV(15c.858) 15:55:09:080 PingComputer: First and second times match.
USERENV(15c.858) 15:55:09:080 PingComputer: First time: 2069
USERENV(15c.858) 15:55:09:080 PingComputer: Second time: -2069
USERENV(15c.858) 15:55:09:080 PingComputer: First time: -2069
USERENV(15c.858) 15:55:09:080 PingComputer: Second time: 0

We have a knowledgebase article that pertains to this issue on servers that uses dual-core or multiprocessor AMD Opteron processors:

938448 A Windows Server 2003-based server may experience time-stamp counter drift if the server uses dual-core AMD Opteron processors or multiprocessor AMD Opteron processors;EN-US;938448

Now in the case of our customer they were not running an AMD Processor server so they felt this resolution did not apply to them. Even though the article did not apply to the type of processor in their servers, the behavior was identical so we applied the resolution outlined in the knowledgebase article and this resolved the customer’s issue. I am in the process of having a knowledgebase article created to specifically address this issue with Windows Server 2003 virtual machines running in Hyper-V.

So we did a little digging and found the following blog post from Tony Voellm, who is a Principal Software Test Engineer in the Windows Kernel development team:

Negative ping times in Windows VM’s – whats up?

The following is from the above blog post:

If you see negative ping times in multiprocessor W2k3 guest OSes you might consider setting the /usepmtimer in the boot.ini file.

The root issue comes about from the Win32 QueryPerformanceCounter function.  By default it uses a time source called the TSC.  This is a CPU time source that essentially counts CPU cycles.  The TSC for each (virtual) processor can be different so there is no guarantee that reading TSC on one processor has anything to do with reading TSC on another processor.  This means back to back reads of TSC on different VP’s can actually go backwards. Hyper-V guarantees that TSC will not go backwards on a single VP.

So here the problem with negative ping times is the time source is using QueryPerformanceCounter which is using TSC.  By using the /usepmtimer boot.ini flag you change the time source for QueryPerformanceCounter from TSC to the PM timer which is a global time source.

I wanted to bring these together as one may read Tony’s blog post and not understand how this would relate to what they see with regards to group policy application and the errors that may occur as a result of this behavior with Hyper-V.

-Brian ‘Fast and Furious’ Singleton