HTTP404 while using IIS URL Rewrite to present a different WebApp

  • Article

I recently supported a customer who was implementing a reverse proxy in IIS using URL rewrite to present an SSRS report from a backend server via an external-facing IIS machine. They had developed the rewrite rule in their Dev environment and it worked perfectly. They moved it over to Prod and modified it accordingly; however, in Prod the page was returning an HTTP 404 File Not Found error.

Summary

A 404 while trying to implement a reverse proxy where all URLs are correct and the resources exist could indicate that ARR is not installed and/or enabled, causing IIS to attempt to satisfy the entire request all on its own.

Failed Request Tracing (FREB) proved invaluable to troubleshooting this issue.

Details

We first investigated the URL rewrite rule to confirm it was built correctly - it was. We then collected failed request (FREB) traces from the working and non-working scenarios. We saw from the FREB in both scenarios the URL rewrite module correctly matched the pattern and the correct backend URL to hit. Testing the generated SSRS report URL directly produced a working report. Conclusion: our problem was not the URL rewrite module. So I continued digging into the FREB traces to see how the request was treated. Below are what events took place after the URL rewrite module finished:

working trace:

nonworking trace:

You'll notice a different set of events immediately following the URL_REWRITE_END event. Most important is the HANDLER_CHANGED event. This indicates the request processing pipeline handed-off processing of the request from PageHandlerFactory-Integrated-4.0 to the ApplicationRequestRoutingHandler in the working scenario. In the nonworking scenario, the PageHandlerFactory[...] does not hand-off the request and continues to process it, eventually moving through the Windows Authentication module and on to the AspNet pipeline. Further down the line (not shown in the screenshot) I see all the normal request processing events as if IIS was trying to process the entire request itself. This explains why a 404 is being returned: the server is processing the request and attempts to present the SSRS report as if it is hosted on itself, instead of reaching out to the SSRS server. Even though IIS is changing the URL in the rewrite module, it's acting as if that URL is hitting just another site located on itself.

Implementing a reverse proxy in IIS that utilizes a separate backend server for requests requires Application Request Routing (ARR) on the proxy server. This module takes care of creating the request with the necessary headers, sending it to the downstream server (SSRS on top of IIS in this case), and capturing the response for further processing and flushing to the user. This is what we see in the working trace. To further cement this, we see the gap in time (54.208 to 54.536) between the URL_CHANGED event and the GENERAL_NOT_SEND_CUSTOM_ERROR event, whereas there is no gap of time whatsoever in the nonworking trace. The descriptions of the events indicate the request was successfully serviced by the downstream IIS machine and a response was received, which is then processed.

At this point I provided this information to the customer and they found that ARR was installed on the IIS proxy server, but not enabled in the IIS configuration. Once they enabled it, the issue resolved and the site started working as intended.

Below is a graphical representation of how this scenario works:

Resources

https://www.iis.net/learn/extensions/url-rewrite-module/reverse-proxy-with-url-rewrite-v2-and-application-request-routing
https://www.iis.net/learn/extensions/planning-for-arr/using-the-application-request-routing-module
https://www.iis.net/learn/extensions/url-rewrite-module/using-the-url-rewrite-module
https://www.iis.net/learn/troubleshoot/using-failed-request-tracing/troubleshoot-with-failed-request-tracing