Being a Architectural Hero

Architecture is great because it is so easy to look like a hero. I talked about earlier in this blog about looking like a hero with a poorly architected application but there are lots of other ways that bad architectural decisions can make you look like a hero.

In particular uninformed consultants are a rich seam of ego boosting; I have to say that sometimes they are assisted by our product groups. To give you an example in SQL Server under the admin tab we used to have a button labelled “Boost Performance”. This boosted the performance of the SQL sever by increasing its thread priority, a great idea if the was nothing else on the system.

In a production system however it was a very bad idea because there is another user of the CPU on a server of course, it’s the operating system! When the database came under heavy load, such as when there were a lot of users, then the OS started to get starved of CPU. Alas when there are a lot of users it means that the networks stacks (clearly part of the OS) are also heavily loaded. In the resulting battle SQL Server, with its “boosted performance” won, took all the CPU so the network started to fail.

So a typical assignment for the caped architect would start with a frantic phone call from a customer saying that their network kept crashing on load. I always like those calls because it is typically an architectural problem and indeed you can normally tell where the issue is by the number of users online when the crash occurs (and someone tells you “Windows doesn’t scale”… gurr!).

For instance SQL Sever will support about 5 to 10 users with the application taking a Database lock (amazing but true). So if someone has a system crashing with 10 users the first question you ask is what locks they are taking out (to which the normal response is “what’s a lock”!). One day I will write out a list of the architectural issue associated with a particular level of scalability.

With boost performance set on then typically the system will crash with about 60 users, so if the system used to work and suddenly starts to crash with network errors at sixty users then the first question to ask is “have you had a consultant in recently”. The normal response is “Yes, we had a chap in last week to tune the system and it was much faster, but he worked on the database and the problem is in the network”

I know straightaway I am about to make someone very happy which is a nice feeling and sometimes leads to a free lunch! I would just ask the person to go into the SQL Administration panel and see if the “boost” button was on and if it was to turn it off. They would come back 10 minutes later, amazed and incredulous, to say that it was on, they had switched it off and now everything was working perfectly! I could then bask in the admiration of my technical abilities. Too easy!

I used to call the “boost” button the “Hero” button because it always made me look like a Hero. Alas in later versions of SQL Server Microsoft hid the button so people who didn’t understand what it did can’t find it; so another opportunity to shine was lost!