Exchange 2010: Error 0x721 - A security package specific error occurred
Recently I was presented with an interesting case regarding the inability to mount databases. The history preceding the event was fairly unremarkable and only noted after running patch maintenance on the server and rebooting. Post reboot every time the customer attempted to mount a public folder database, the following active manager error occurred:
Couldn't mount the database that you specified. Specified database: Public Folder Store NAME; Error code: An Active Manager operation failed. Error: The database action failed. Error: Operation failed with message: Error 0x721 (A security package specific error occurred) from cli_AmMountDatabaseDirectEx [Database: Public Folder Store NAME, Serve r: server.child.domain.local].
+ CategoryInfo : InvalidOperation: (Public Folder Store NAME:ADObjectId) [Mount-Database], InvalidOperationException
+ FullyQualifiedErrorId : F34E87D0,Microsoft.Exchange.Management.SystemConfigurationTasks.MountDatabase
Also when reviewing the application log the following event was noted:
Log Name: Application
Source: MSExchange Configuration Cmdlet - Remote Management
Date: 11/7/2010 9:46:17 AM
Event ID: 4
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: SERVER.child.domain.local
Description:
(PID 6364, Thread 43) Task Mount-Database writing error when processing record of index 0. Error: System.InvalidOperationException: Couldn't mount the database that you specified. Specified database: Public Folder Store NAME; Error code: An Active Manager operation failed. Error: The database action failed. Error: Operation failed with message: Error 0x721 (A security package specific error occurred) from cli_AmMountDatabaseDirectEx [Database: Public Folder Store NAME, Server: SERVER.child.domain.local]. ---> Microsoft.Exchange.Cluster.Replay.AmDbActionWrapperException: An Active Manager operation failed. Error: The database action failed. Error: Operation failed with message: Error 0x721 (A security package specific error occurred) from cli_AmMountDatabaseDirectEx ---> Microsoft.Exchange.Data.Storage.AmOperationFailedException: An Active Manager operation failed. Error: Operation failed with message: Error 0x721 (A security package specific error occurred) from cli_AmMountDatabaseDirectEx ---> Microsoft.Exchange.Rpc.RpcException: Error 0x721 (A security package specific error occurred) from cli_AmMountDatabaseDirectEx
at ThrowRpcException(Int32 rpcStatus, String message)
at Microsoft.Exchange.Rpc.RpcClientBase.ThrowRpcException(Int32 rpcStatus, String routineName)
at Microsoft.Exchange.Rpc.ActiveManager.AmRpcClient.MountDatabaseDirectEx(Guid guid, AmMountArg arg)
at Microsoft.Exchange.Data.Storage.ActiveManager.AmRpcClientHelper.<>c__DisplayClass26.<MountDatabaseDirectEx>b__25(String )
at Microsoft.Exchange.Data.Storage.ActiveManager.AmRpcClientHelper.<>c__DisplayClass4e.<RunRpcOperationWithAuth>b__4c()
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRetryableOperation(String serverName, RpcClientOperation rpcOperation)
--- End of inner exception stack trace ---
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientHandleRpcException(RpcException ex, String serverName)
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRetryableOperation(String serverName, RpcClientOperation rpcOperation)
at Microsoft.Exchange.Data.Storage.ActiveManager.AmRpcClientHelper.RunRpcOperationWithAuth(AmRpcOperationHint rpcOperationHint, String serverName, String databaseName, NetworkCredential networkCredential, Nullable`1 timeoutMs, AmRpcClient& rpcClient, InternalRpcOperation rpcOperation)
at Microsoft.Exchange.Data.Storage.ActiveManager.AmRpcClientHelper.MountDatabaseDirectEx(String serverToRpc, Guid dbGuid, AmMountArg mountArg)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmDbAction.MountDatabaseDirect(AmServerName serverName, AmServerName lastMountedServerName, Guid dbGuid, MountFlags flags, AmDbActionCode actionCode)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmDbPamAction.RunMountDatabaseDirect(AmServerName serverToMount, MountFlags mountFlags, Boolean fLossyMountEnabled)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmDbPamAction.<>c__DisplayClass3.<AttemptMountOnServer>b__1(Object , EventArgs )
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmHelper.HandleKnownExceptions(EventHandler ev)
--- End of inner exception stack trace (Microsoft.Exchange.Data.Storage.AmOperationFailedException) ---
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmDbOperation.Wait(TimeSpan timeout)
at Microsoft.Exchange.Cluster.ActiveManagerServer.ActiveManagerCore.MountDatabase(Guid mdbGuid, MountFlags flags, DatabaseMountDialOverride mountDialOverride, AmDbActionCode actionCode)
at Microsoft.Exchange.Cluster.ActiveManagerServer.AmRpcServer.<>c__DisplayClass4.<MountDatabase>b__3()
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.RunRpcServerOperation(String databaseName, RpcServerOperation rpcOperation)
--- End of stack trace on server (SERVER.child.domain.local) ---
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRethrowIfFailed(String databaseName, String serverName, RpcErrorExceptionInfo errorInfo)
at Microsoft.Exchange.Data.Storage.ActiveManager.AmRpcClientHelper.RunDatabaseRpcWithReferral(AmRpcOperationHint rpcOperationHint, Database database, String targetServer, AmRpcClient& rpcClient, InternalRpcOperation rpcOperation)
at Microsoft.Exchange.Data.Storage.ActiveManager.AmRpcClientHelper.MountDatabase(Database database, Int32 flags, Int32 mountDialOverride)
at Microsoft.Exchange.Management.SystemConfigurationTasks.MountDatabase.InternalProcessRecord()
--- End of inner exception stack trace ---
The error message and event unto themselves are not very telling as to what the issue was. The important part of the event, which is not unique to Exchange and has been seen with other shell commands, is the security package error:
# for hex 0x721 / decimal 1825
RPC_S_SEC_PKG_ERROR winerror.h
# A security package specific error occurred.
After some investigation we were able to determine that the active directory forest where Exchange was installed contained a multiple domain structure. In this case we searched the entire directory, and found that there were two ENABLED machine accounts with the same name residing in two different domain naming contexts in the same forest. After identifying the machine account that was not being used (in this case the one in a child domain where Exchange servers were not installed) and deleting it – our mount commands proceeded successfully with no issues noted.