Hello, my name is Michael C. Bazarewsky, and I’m a Public Sector Secure Infrastructure PFE, spending most of my time working on Azure-related engagements.
Today, I wanted to present two quick notes about some issues I saw while helping a customer perform a parallel installation of Azure AD Connect, to upgrade from an older DirSync installation. These are not necessarily issues you’ll see – in fact, they are very unusual, in my experience. They were both issues I had never seen, and I’ve helped customers with this many, many times.
Before beginning, I should reference the official documentation, located here. The documentation explains how to do the export and import process, so I’m not going to repeat that here. Instead, I’m just going to discuss the two specific issues that happened in this case.
The first issue occurred during the export. The export appeared to take a very long time – much longer than normal – before finally failing, indicating that the FIM Synchronization Service failed to restart, with error code %%-2145188858. Sure enough, when we looked at the Services Control Panel, the FIM Synchronization Service entry showed in was stuck in a “Starting” state. I know from experience this sometimes is a SQL Server connection issue. In this case, the customer was running a local SQL Server Express instance (as is very common.) So, I asked the customer to restart the SQL Server instance in the Services Control Panel. This caused the FIM Synchronization Service to stop with a failure. However, it restarted successfully when attempted to in the Services Control Panel. Emboldened by this, I then had the customer attempt the export again. This time, the export was successful. I’m still not completely sure what was happening there – something strange in the local SQL Server Express state, I suspect – but as we were doing a parallel install, and thus this server was not long for this world anyway – the customer was willing to accept “cosmic rays” as a tongue-in-cheek explanation and move on.
The second issue was even more subtle. The configuration process on import appeared to move steadily, until the final verification and configuration was being performed. When the verification of the Azure Active Directory connection was performed, we received the following error:
Unexpected exception thrown. Action: PingProvisioningServiceEndPoint, Exception: An error occurred. Error Code: 6. Error Description: Your credentials are not authorized to access Windows Azure Active Directory. Please check your Administrator credentials and try again. Tracking ID: [guid]
What was very frustrating about this error is that when you are doing a parallel installation, you are forced to use the existing Azure Active Directory account from the working installation. Further, the configuration wizard verifies the password is accurate on the page you enter it. This means that we knew the username and password had to be correct! Some quick Bing’ing suggested that this could be an issue with incorrect local credentials. However, we knew that not to be the case here. We also knew there was not a proxy in the on-premises environment to interfere with communications.
On a hunch, I asked the customer to sign in to the Office 365 Portal at https://portal.office.com as a Global Admin, to verify the directory synchronization user was in fact a Global Admin also. This is easy enough to check in the user list in the (new) Office 365 Admin Portal:
- Select Users.
- Select Active users.
- Locate the Directory Sync user in the list, and click it.
- In the Roles section, confirm the user is a Global administrator.
That told us the account was a valid account, with valid permissions. So, what could possibly be the problem?
On another hunch, I asked the customer to sign in to the Office 365 Portal as the Directory Sync user. When we did, that’s when something interesting happened. As some of you reading this know, the Office 365 Portal is very aggressive about ensuring all Global Admin users have a fallback phone number and e-mail address, in the case of misplaced credentials. It turns out, if your synchronization account is old enough, predating these aggressive requirements, then it may be in a state where although your current synchronization is working, the new synchronization cannot be configured, because the account won’t be allowed in! This was clear when the customer signed-in, as requested, and encountered the “dreaded” “update your admin contact info” redirect:
The customer entered appropriate information and verified it with the codes sent to each destination. Once they did that, they hit Retry in the Azure AD Connect wizard, and they were successfully able to move forward.
I hope you find this helpful if you have either of these situations occur.
— Michael C. Bazarewsky, PubSec Secure Infrastructure PFE