Let assume that we have our performance testing environment and in that we have a web farm with 40 servers and portfolio of application hosted on those servers.20 of them are the web servers and rest are web services servers. We also have front end load balancers which balances the load based on least connection algorithm.Then we also have the SSO which is being implemented by Site minder and this site minder protects all the application hosted on this performance environment.
Now lets assume that for some reasons one of the our application is having some issues with load balancer or lets say that with Site minder servers. For some reasons requests are either not getting spread across all the servers or for some reasons cookies is either getting lost in some of the web servers in the server farm. So as the performance engineer, how would you determine which server in the farm is giving an issue or how would you determine on which web server your request is going ?. I believe the first thing people do, is look at the application logs or web servers logs one by one and see which server the request is being processed.Yes this is an correct solution ,however this consumes lot of time and effort to go through each servers and check individually.Sometimes it so happens that your logs files are in several MB and you can understand the pain it takes to open hundred MB log files.Some times it happens that request is served by only one web server and we check all the servers logs only to find out that later that those servers do not have any requests coming to it.There are many such things we come across while troubleshooting in web farms.
Now as a performance engineer we can find out which server is serving the request in the web farm by looking at the Miscellaneous headers of the response. These are the additional Custom headers which are normally configured to provide additional information to know as which servers are serving the request in the web farm or anything which we are looking for like asp.net version or jdk version etc. For IIS based web farms, below are some headers,
If you install the fiddler , you can see all these headers in the response header section of the response.X-Server is the header which will tell you which sever in the farm is serving the request.The value associated with this header is the server which is serving the request.The fiddler trace will look something like below
If you look at the value of X-Server in above picture , its having value web303 which means that response has be served from web303 box and this is IIS web server box by name WEB303.Also please understand that all these servers naming convention depends on your environment.So you need to check with your server admistrator for your server names.
I would also like highlight that not all web farms environment will have this setup for custom additional headers,if case if this setup is not available in your web farm,you can request your server admin to add custom these header.This is normally done using URL Rewrite Outbound Rules which can be done for most of the web servers available in the market like IIS and Apache.This process also do not have much processing overheads.
We can also have the scripted solution for this in our all Load testing scripts . Below is the LoadRunner code which will capture the value from the server response headers.
All you need to do it to place this code at the top of the first request and output the X-Server value in the log. Using this approach , I am sure you are going to save lot of time in troubleshooting your webservers in your web farm environment.
If you still have better ideas for web farms, go ahead and mail me at kiran at vasanti.org