With the Beta release of Global Service Monitor (GSM) we added one of the most requested features: ability to execute multistep web tests with authentication.
This is a big step forward compared with single URL tests we had in CTP, because it allows monitoring health of end-to-end user scenarios. Imagine your application sells some goods: say books or food (or groupons 🙂 ). Your users need go to the home page, login, possibly perform a search, place the item in the shopping cart, and check out.
Visual Studio Ultimate provides capability to record all those steps while they are being performed in a browser, and play them back. During the playback, various parameters about the test execution are being measured and compared for correctness with the criteria specified in the recorded test. For example, you can verify that each response is faster than certain time limit; you can check whether a certain string or a certain tag is present in the response, or if response is a redirect, you can verify the URL it redirects to. In essence, Visual Studio web test provides many knobs to fine-tune the criteria which indicate the success of your end-to-end scenario. Please see this walkthrough for more details about recording and configuring Visual Studio web tests.
Please keep in mind that GSM imposes certain restrictions on each Visual Studio web test to ensure that the test can co-exist well with other web tests which are running on the service. Specifically, we enforce the following limits in Beta release (these conditions are subject to change in the future):
- The web test file size is less than 100 KB.
- Number of steps in the test cannot be more than 100.
- The test overall is faster than 30 seconds.
- There are no loop statements, plugins, or references to other tests.
- Each subscription cannot have more than 3 tests per location, or 45 tests total (including all the locations).
Developers of the application will probably create web tests to verify performance and correctness of the end-user scenarios while they are preparing to roll out the application to production. These tests can be shared with the Operations Manager admin to use for monitoring of the application health. Operations Manager needs to have Global Service Monitor management packs installed and the service configured in order to use those tests in production.
Global Service Monitor can distribute the tests to the specified locations around the globe: currently we have 16 locations. The results of the test execution are analyzed for failure, and the pass/fail results of the tests are presented in the Summary Dashboard of the Global Service Monitor. Please note that you can configure more than one test, if your application supports multiple end-user scenarios. The results of all the configured tests will be rolled up into the health state of the application, which is displayed on the map (Summary) dashboard.
If something goes wrong in production and the tests start failing, Operations Manager will bring up an alert. Alerting behavior in GSM can be configured to alert on each web test individually, or to roll up all the tests and alert once for the whole application if at least one of the tests failed. You will get a reason why the test failed in the Alert Context. For example, if a string match validation rule fails, you may see a description similar to this:
Often, in order to fix the problem with the application, input from the developers is required.
With the TFS synchronization functionality available in OpsMgr 2012 SP1 Beta, we are completing the loop of DevOps communication: now the Operations Manager admin can assign the alert back to the developer for debugging and fixing the problem. Enabling running web tests recorded in Visual Studio and ability to assign OpsMgr alerts back to engineering makes DevOps integration and communication more effective and more informative. Both developers and operation admins can use the tools they have always been using and simultaneously get access to the full context of the issue. TFS synchronization provides two-way updates between OpsMgr alerts and operational issues in the TFS database, so that both parties could remain on the same page regarding the progress of the investigation. Such agile communication may help reducing MTTR.
We hope that ability to run web tests recorded in the Visual Studio from GSM locations will help companies to leverage existing dev assets (recorded tests) for the monitoring purposes. Multistep tests with authentication will enable in-depth monitoring of main end-user scenarios to ensure customer satisfaction, and ultimately will help running a successful business!