Happy New Year! I have seen several blogs and some entries on the forums with bits and pieces of information related to the issue and solution that I will present below, but there wasn't a blog outlining this specific issue. So I wanted to put one together for those who have struggled with the issue described below.
I was working with a customer this week who had an issue with their application catalog. They were using the new software center and noticed that the color scheme in the new software center had reverted back to the default blue color. As you may or may not know, the new software center still depends on the actual application catalog and application catalog web service to port deployments and color scheme into the new Win32 software center application. So, we knew there was an issue of sorts that needed to be addressed.
The first thing you want to do when troubleshooting is check your logs and component status in the Configuration Manager console. Checking the components in the console, we saw both components were green. In order to corroborate this, we checked the application catalog logs, portlctl.log and awebctl.log and there were no errors. So we needed to dig a little deeper. In the SMS_CCM folder where the application catalog roles are installed on their respective server or servers, there are two directories that contain additional logs that can help pinpoint an error. The directories are CMApplicationCatalog\Logs and CMApplicationCatalogSvc\Logs respectively. In the CMApplicationCatalog\Logs location is the ServicePortalWebsite.log. In the CMApplicationCatalogSvc\Logs location is the ServicePortalWebService.log. In those logs, we immediately found clues to what the issue was.
We looked at the ServicePortalWebService.log first as the web service is the foundation for the actual application catalog website to run correctly. In this log, we saw tons of red entries for this error message:
- The service certificate is not provided; this could cause errors when web sites attempt to communicate with the web service. The certificate thumbprint in the web.config did not match any certificate in the SMS cert store.
- FindCertificate - No matching certs found
So immediately, we now know we have a certificate issue in IIS. We scrolled up in the log to see when the error started and the error had started on 1/12/2018. We then looked in the certificate store of the server that hosted the application catalog roles and low and behold, there was a certificate that had an expiration date of 1/12/2019. What had happened was the certificate auto-renewed and the SSL binding in IIS had been lost with the certificate that expired/auto-renewed. So we bound the certificate back to the website and ran IISRESET. We watched the logs and got a little impatient so we opened Configuration Manager Service Manager and bounced the application catalog component services to expedite. Problem solved right? Well the error went away in the ServicePortalWebService.log as it found the certificate. But when we tried to access the application catalog URL, we still got the error. Next, we looked in the ServicePortalWebSite.log and saw the following error:
- The client certificate is not provided; this could cause errors when the web site attempts to communicate with the web service. The certificate thumbprint in the web.config did not match any cert in the SMS cert store.
So even though we had bound the certificate, reset IIS, and bounced the component services, the application catalog was still complaining about the certificate. The next thing we did was open up the SMS_CCM\CMApplicationCatalog web.config file to try to find the thumbprint of the cert. Sure enough, the thumbprint of the cert in the web.config file did not match the thumbprint of the cert that was renewed and that we bound back to the website in IIS. For whatever reason, this value was not updating in IIS via Configuration Manager. NOTE: At this point, had we uninstalled and reinstalled the application catalog roles, it would have updated the web.config file with the thumbprint. However, we decided to update the thumbprint manually. Once we did that and saved the web.config file, everything worked as intended.
In all, it took about 40 minutes to get this all worked out. However for those who are not familiar with the application catalog and how it works or where all the logs are located, this would frustrate you to no end searching the web on how to fix this issue. I hope anyone who comes across this gets some use out of it.
Until next time!