OpsMgr 2007: Error running report: "Message: Loading reporting hierarchy failed.” “Access is denied"

Hi Everyone, my name is Prakash and I recently worked an interesting case that I wanted to share with you here.  The issue was that whenever you tried to run a report in System Center Operations Manager 2007 you would receive this error:

Error message when you try to run a report in System Center Operations Manager 2007: "Message: Loading reporting hierarchy failed.” “Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))”    

This turned out to be caused by how the particular domain environment was constructed and I thought it might be helpful to blog this just in case you run into something similar:

========

Consider the scenario where we have a resource domain that is part of forest ‘A’ and we have all of the SCOM components and SCOM services account belonging to it. We also have another domain (the Resource domain) that is part of forest ‘B’ which has the SCOM report operators account in it.  There's only a one-way trust between the Resource domain and Account domain where the Resource domain trusts the Account domain.

In this scenario, if we try to run the report using a user which belongs to the Resource domain, the reporting fails with the following error.

Date: DD/MM/YYYY hh:hh:ss
Application: System Center Operations Manager 2007
Application Version: 6.0.6278.0
Severity: Error
Message: Loading reporting hierarchy failed.
System.Web.Services.Protocols.SoapException: An internal error occurred on the report server. See the error log for more details. ---> An internal error occurred on the report server. See the error log for more details. ---> Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
  at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Microsoft.EnterpriseManagement.Mom.Internal.UI.Reporting.ReportingService.ReportingService2005.ListChildren(String Item, Boolean Recursive)
   at Microsoft.EnterpriseManagement.Mom.Internal.UI.Reporting.ManagementGroupReportFolder.GetSubfolders(Boolean includeHidden)
   at Microsoft.EnterpriseManagement.Mom.Internal.UI.Reporting.WunderBar.ReportingPage.LoadReportingSubtree(TreeNode node, ManagementGroupReportFolder folder)
   at Microsoft.EnterpriseManagement.Mom.Internal.UI.Reporting.WunderBar.ReportingPage.LoadReportingTree(ManagementGroupReportFolder folder)
   at Microsoft.EnterpriseManagement.Mom.Internal.UI.Reporting.WunderBar.ReportingPage.LoadReportingTreeJob(Object sender, ConsoleJobEventArgs args)

Also we will have the below event Details generated in the Operations Manager event log on RMS server.

Event ID: 26319
Source: OpsMgr SDK Service
Description: An exception was thrown while processing GetUserRolesForOperationAndUser for session id uuid:38834c07-855b-47b9-9425-2297b283cd90;id=166.
Exception Message: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
Full Exception: System.UnauthorizedAccessException: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))
   at Microsoft.Interop.Security.AzRoles.IAzApplication2.InitializeClientContextFromStringSid(String SidString, Int32 lOptions, Object varReserved)
   at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AzManHelper.GetScopedRoleAssignmentsForUser(IList`1 roleNames, String userName)
   at Microsoft.EnterpriseManagement.Mom.Sdk.Authorization.AuthManager.GetUserRolesForOperationAndUser(Guid operationId, String userName)
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccess.GetUserRolesForOperationAndUser(Guid operationId, String userName)
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccessTieringWrapper.GetUserRolesForOperationAndUser(Guid operationId, String userName)
   at Microsoft.EnterpriseManagement.Mom.ServiceDataLayer.SdkDataAccessExceptionTracingWrapper.GetUserRolesForOperationAndUser(Guid operationId, String userName)
For more information, see Help and Support Center at https://go.microsoft.com/fwlink/events.asp

This problem only arises in the case of Reporting because in the reporting scenario the security context is initialized from a SID.

This restriction is an AzMan restriction and more details can be found at the MSDN link below under InitializeClientContextFromStringSid:

https://msdn.microsoft.com/en-us/library/aa377364(VS.85).aspx

So what's the resolution?  Well at the core this behavior is by design with System Center Operations Manager reporting although we can use the work around for this issue with the restriction mentioned below:

Workaround:

In our scenario, InitializeClientContextFromStringSid was called in the context of a user who belongs to the Resource domain. The SID that was passed belongs to a user from a trusted domain (the Account domain). Since there is only a one-way trust we get the Access Denied error.

As a work around, change the SDK and Config accounts to use a domain user from the Account domain instead of from the Resource domain.  You should also add the SCOM service account from the Account domain to the Windows Authorization Access Group" of both domains and then restart the SDK and Config service.

Restriction:

If you have an architecture where you have one main domain (domain1) with several trusted domains (one way trust, with say domain2, domain3 and domain4), then you can only pass SIDs that belong to users of the main domain (domain1) and ONE of the trusted domains. If you pass SIDs of users from domain3 and domain4 it will fail with access denied.

Note : To change the credentials for the OpsMgr SDK Service and for the OpsMgr Config Service in Microsoft System Center Operations Manager 2007 please follow the steps mentioned in KB936220 - https://support.microsoft.com/?kbid=936220 .

Thanks & Regards ,

Prakashan A K | Support Escalation Engineer