StudentNET stress testing - it's not just about running the scripts ...

We have done some preliminary stress tests to see how StudentNET responds to high levels of usage. To start us off we created very simple test scripts that simulated a user logging on, loading the framework and logging off.

Creating test scripts using Visual Studio Team Foundation is a piece of cake, so at first we didn't stumble upon any problems. Within a few minutes of launching the wizard, the script was created in C# and we were ready to set up the first load test.

For an adequate load test we needed to have at least 4 agents that would be sending requests to our web servers; a controller that would be distributing the test load among the agents; a test client that executes the script. Now, considering we do not have a testing environment, let alone a testing lab, we had to make do with PCs of colleagues that were on holiday. So getting together 6 PCs was difficult enough.

Set up

Setting up the testing grid with staff PCs proved to be quite tricky. At first we had problems connecting the client (runs the scripts) to the controller (distributes the load). A few IPCONFIGS later we discovered that the two machines were on completely different IP ranges and were unable to resolve the machine name. Even though we were able to connect via IP, Visual Studio Team Foundation did not work, seemingly because the controller tried to contact the client with its name, which again did not resolve.

Secondly, running Virtual PC 2007 on our staff PCs meant that all 6 machines were too weak to generate a load that we would have wanted to test. It created a bottle neck of 300 requests to our web servers at any one time before the agents started to fail.

So we reduced the load of concurrent users to be able to run a few 5 hour tests to see whether the application was deteriorating in any way. Although this significantly reduced the load on our webfarm and resulted in minimal load balancing, we were able to fetch a few results that helped us improve the application.

Next steps

We're going back to the framework to improve its load and also to fix fundamental errors that cropped up during stress tests.

We are also going to look at the possibility of using a Virtual Environment to run 10-12 agents to increase the stress on our web servers.