User Profile Synchronization Service stuck in 'Starting' state and no error in ULS |SharePoint2010|
It was pretty interesting case that the UPS was stuck in starting stage but there were no error messages in ULS and not even the 'User Profiles' category.
Done many steps at least to get something in the ULS regarding this issue like flushing the config cache, Checking the permission level, deleting the one-time jobs etc..
While thinking logically, it seems something wrong with the timer job who broken in some point for some reason. To confirm that ran following powershell script..
$farm = Get-SPFarm
$disabledTimers = $farm.TimerService.Instances | where {$_.Status -ne "Online"}
if ($disabledTimers -ne $null)
{
foreach ($timer in $disabledTimers)
{
Write-Host "Timer service instance on server " $timer.Server.Name " is not Online. Current status:" $timer.Status
#Write-Host "Attempting to set the status of the service instance to online"
#$timer.Status = [Microsoft.SharePoint.Administration.SPObjectStatus]::Online
#$timer.Update()
}
}
else
{
Write-Host "All Timer Service Instances in the farm are online! No problems found"
}
The result of above script was >> "Timer service instance on server SERVERNAMEis not Online. Current status: Disabled"
Following script we used to start the timer service instance,
$farm = Get-SPFarm
$disabledTimers = $farm.TimerService.Instances | where {$_.Status -ne "Online"}
if ($disabledTimers -ne $null)
{
foreach ($timer in $disabledTimers)
{
Write-Host "Timer service instance on server " $timer.Server.Name " is not Online. Current status:" $timer.Status
Write-Host "Attempting to set the status of the service instance to online"
$timer.Status = [Microsoft.SharePoint.Administration.SPObjectStatus]::Online
$timer.Update()
}
}
else
{
Write-Host "All Timer Service Instances in the farm are online! No problems found"
}
Once the timer service instance is up, restarted the timer service and tried to start the user profile synchronization service after deleting timer jobs which are stuck to run one time. But this time also we wasn't able to make it and made a progress on working the functionality to write user profile logs to SharePoint ULS.
Later while checking the permission levels based on the file system, I found that the the 'NETWORK SERVICE' read per mission is missing on '14' hive. After clearing this permission issue again tried starting the UPS and found in ULS that the UPA web service was getting stopped with '503 Service Unavailable'. When checked the Application and found the web service is in stopped state for the identity as farm account with following error in system even viewer.
Description:
The identity of application pool 'GUIID' is invalid. The user name or password that is specified for the identity may be incorrect, or the user may not have batch logon rights. If the identity is not corrected, the application pool will be disabled when the application pool receives its first request. If batch logon rights are causing the problem, the identity in the IIS configuration store must be changed after rights have been granted before Windows Process Activation Service (WAS) can retry the logon. If the identity remains invalid after the first request for the application pool is processed, the application pool will be disabled. The data field contains the error number.
Event Xml:
<Event xmlns="https://schemas.microsoft.com/win/2004/08/events/event">
<System>
The user agreed that he reset the farm account password due to some inaccessibility issues. So we re-enter the password for failed app pool identity and the User Profile Synchronization Service started successfully.
Hope this is helpful !