App-V: On Operational Troubleshooting of the V5 Client

 Update 12/6/2014: The App-V 5 logs have been consolidated and are organized differently as of App-V 5 SP3 - but the event and event formats are still the same. Please read more about it here: https://technet.microsoft.com/en-us/library/dn858700.aspx#BKMK_event_logs_moved

Whenever I am doing customer engagements, I find it always valuable to ensure that the customer has a plan in place for ongoing support. Having my origins in support myself, I can relate to the customers concerns with impact to their helpdesk. Especially when a new product has been introduced in their ecosystem.

In the world of App-V, we divide up troubleshooting problems in two ways: 1) troubleshooting the operational client engine and 2) troubleshooting the application itself. IT departments relegate level 1 operational troubleshooting to their helpdesk team. This requires training your helpdesk with FAQs, CubeNotes, flowcharts, etc. to resolve basic issues and to at the very least, gather the necessary data to help higher level escalation teams isolate the problem.

Training or Retraining your HelpDesk on App-V 5

The App-V Client will be a very different experience for your help desk in terms of toolkit. Conceptually, they will be doing a lot of the same thing but the way they do it will be different.

Package Repair

Repairing a package will still be needed, but instead of using the client UI (which is deprecated in App-V 5.0 SP2) or SFTMIME, you can use the Repair-AppVClientPackage to reset user and or package state. You get can get specifics on this by reading Ryan Cobb’s blog post on the subject here:

https://blogs.technet.com/b/appv/archive/2013/10/28/how-to-quickly-repair-published-app-v-packages.aspx
 

Also NOTE: I have seen information out there stating that you can also simply delete files directly in the native locations. That is true but this command is much cleaner and useful for the help desk professional. You should also never, ever, ever, ever, ever, ever, ever delete the VFS directories manually as this will also create problems by potentially breaking the user mode VFS and copy-on-write mappings for the application. If you suspect issues with those VFS directories, use the Repair-AppVClientPackage command instead.

Connection Group Cleanup

The same options available for an individual package are also available for a connection group using the Repair-AppVClientConnectionGroup command. In fact, you should be advised that if you repair the connection group instead of the individual packages when you are experiencing issues where you would normally repair the application, this will purge out all of the settings for all applications within the connection group. If there are applications currently in use and it is preventing you from running these repair operations you will need stop all of the package processes. Stop-AppVClientPackage will stop all virtual processes in a package and Stop-AppVClientConnectionGroup will stop all virtual processes in a group.

App-V Client Logs

The SFTLOG.TXT is gone. So are all of the other text-based logs. Knowing how to collect good log data is essential to every help desk professional especially if escalations will be involved. The good news is everything is now logged to event logs. You can view these by opening up the event viewer and navigating to Applications and Services Logs – Microsoft – AppV – Client. The default view will show the three basic event logs: Admin, Operational, and Virtual Applications.
  
 

If you would like to view the additional logs that can enabled, navigate to the view menu and select “Show Analytic and Debug Logs.” In many cases, you may need to take advantage of these.

 

As you will see, there are a tremendous amount of additional logs. Most of these are not enabled by default. You will have to expand the folder, right-click the log and select “Enable Log” to enable it. Be careful turning on a lot of these as it can potentially affect performance.

Which Log do I Use?

Well now that will depend on the type of issue that is occurring be it an error upon publishing, streaming, launching, client service start-up, etc. often, the basic logs will give you what you need to know to fix the issue, but when it doesn’t, it is time to investigate further and these additional logs can come in very handy. The question is, which log or logs should I enable? The actual error message or error code may quickly provide some insight.

The App-V 5 Error Code

Like I did in V4, I am not necessarily concerned with the entire error code. I usually only have to pay attention to the last 10 digits when looking at V5 errors ONCE I have the error in the correct Hex format. Often time, in the event logs, the error code is actually in decimal format. One of the first things you will want to do is get the error in its correct format. For example, if you see the following error in the event log:

64 notification failed with error 5746720685053444123, Package <GUID>, version <GUID>, pid
<PID>, ve id 0

The first thing I do is convert the decimal error over to hex.

 

5746720685053444123  translated to hex is 0x4FC074040000001B.

If you break it down first separating the last ten digits (04-000000001B) you diagnose this further by resolving the first two digits to the component using the table below:

 

Hex Code

Category

Where you might further Info

01

Integration

Client-Integration Log

02

Orchestration

Client-Orchestration Log

03

Client Service

Client-Service Log

04

Virtualization Manager

Client-VirtualizationManager Log or Client-Vemgr Log

05

Shared Components

Admin and Operational Logs

06

Catalog

Client-Catalog

07

Publishing

Client-Publishing Log, Client-PackageConfig Log, or  FileSystemmetadataLibrary Log

08

Client Common

Admin and Operational Logs

09

Policy

PolicyLibrary Log

0A

Virtualization Subsystem

May need to diagnose using the individual virtual subsystem debug logs.

0B

Subsystem Controller

Client SubsystemController Log

0C

Streaming Manager

Any of the five streaming debug logs

0E

Client Host

 

0F

Client Configuration

Registry/PowerShell

10

Scripting

Client Scripting Log

11

Client User Interface

Client-StreamingUX Log

12

Sequencer

Only on Sequencer

13

Reporting

Client Reporting Log

14

Manifest

ManifestLibrary Log

 

 

 

 

Then the last 8 digits will usually just be a standard Windows HRESULT code. In the example above, I know the issue is somewhere in the virtualization manager and the last eight digits are 0x0000001B. The HRESULT alone does not really lead me anywhere so I would likely need to dive further into the Client-VirtualizationManager debug log.

Sometimes, you will get really lucky. For example, if the error is 0xFC03E0480070003. Well, again we know it is an issue with the virtualization manager but the HRESULT 0x80070003 is well-known - ERROR_PATH_NOT_FOUND. You know where I would then go next – Process Monitor of course!

         
  Update 12/6/2014: The App-V 5 logs have been consolidated and are organized differently as of App-V 5 SP3 - but the event and event formats are still the same. Please read more about it here: https://technet.microsoft.com/en-us/library/dn858700.aspx#BKMK_event_logs_moved