Load Test Using Windows Azure and Visual Studio 2013
In a perfect world I would expect to be able to run a load test or a stress test using a cloud provider without being forced to change any line of code from my tests.
This article will show you how to do this using the new cloud service. It will teach a person without knowledge of cloud to run a load test with 50.000 users for 24 hours with minimum costs, without being compelled to purchase and configure 10 or 20 nodes.
What is a load test?
What is a load test in fact? An expression that I think fits when we need to answer this question would be:”Are you ready to be that popular?”. The main scope of this kind of test is to define and validate the maximum load that a system can take without affecting performance. Even though the developer says that the system can handle 1000 users at the same time without problems, the load test can prove the contrary even when it comes to simple scenarios.
There is a tiny difference between a load test and a stress test. Many times the load test finishes with a stress test that has the purpose to observe system behaviour in the moment when the load increases above the maximum supported capacity.
What does the market offer us now?
At this moment it is full of different solutions for automated testing and for running load tests. Products like Selenium, LoadStorm or Neoload dominate the market. There are many others and I don’t think 10 pages would be enough to list them all.
If we use an on-premise solution then we need to be prepared to resolve things like the acquisition and reservation of the resources for the load-test. It will be very hard to buy 10 servers in a small company just to run the load-test. As for a big company, even if it has the financial resources, it will take a while until these resources will be available and sometimes it might be too late. All these factors aside, the configuration of each machine and deploy can prove to be the labyrinth from which we won’t be able to find a way out.
What is the Microsoft proposal?
Microsoft has developed an ingenious solution for this extremely mature market. If they already have a powerful and robust infrastructure for cloud why shouldn’t they use it to run load tests using Azure. Even though they are not the first ones offering this kind of solution, Microsoft has an advantage.
They allow you to run load tests using Azure without using a new configuration. All you have to do is have an account on Visual Studio Team Foundation Service which will be used for the receipt.
Before a load test using Microsoft could only be carried out through Load Test Ring. It would have to be formed from a controller that controls the tests- Load Test Controller and one or more agents on which our tests run. It uses an architecture that is based on the master-slave principle.
The new version of Visual Studio 2013 provides us with a new option- instead of running our tests in our own Load Test Ring we can run them on Azure, without a new configuration. We don’t have to deploy any virtual machine or configure different services.
Load Test Web Service
Load Test Web Service is the new Azure service that helps us. Through this service, Visual Studio 2013 loads our tests on the cloud. Behind this service is a pool of test agents that is used to run our tests. This happens behind the scenes and we don’t have to do anything for it to run. All the results from our test together with other performance counters are available to us.
Which are the main characteristics?
Before looking over different functionalities that are supported I suggest we take a look at the most important options that are supported now. We have the possibility to define UI tests and more. Beside the fact that we can record our UI tests we can also write our custom tests and hit different endpoints.
The tested endpoint doesn’t have to be only HTTP or HTTPS. We can also test a WCF or REST endpoint and also a web service. The only condition is that the endpoint must be accessible via internet. Visual Studio 2013 brings a project template- Web Performance and Load Test Project. This type of project will allow us to define and run load tests on cloud.
How do we create a new load test?
The easiest way to create a load test for a web application is to create one or more Web Performance Tests. We can create UI tests which are very easy to create and automate. Using the UI recorder that comes with Visual Studio 2013 we can create a Web Performance Test in just a few seconds.
This type of test can be modified at any time and if we want we can generate code for the test. This method helps the ones who wish to modify the code because it can be done very easily. Of course functionalities such as automatic detection of the page’s dynamic elements or the extraction of different constants as parameters is supported out of the box.
We can generate different sources for each test, like database, CSV or XML file. You can also use the tests written in Selenium. Each test can call another test and we can reuse the logic that we already have for testing. Until now we saw how we can create a test that can be used for the load test. It is time for us to see what a load test allows us to do.
Which are the main functionalities?
The first thing we need to do in order to create a load test is to THINK. We can define different profiles and time intervals through which we should be able to simulate a real user. For example we can simulate a delay of X seconds after each test. We have different available profiles which we can use.
We can use different methods to simulate a load test. We have the possibility to run a test with a specific number of users or we can define the number of users to increase at every step. Of course that most of the times we will have to run different scenarios at the same time. That is why we can select the tests that we want to run and in what proportion, but also the way that these tests have to run. For example we can specify how many times a test should run in a time interval or the number of users that have to run a specific test at every moment while the load test is running.
We can simulate different browsers that the clients might have and different types of connections. The most interesting thing is that we have the possibility to add and access not just performance counters from the clients but also the ones from the servers on which our application is running. This way we can monitor and see the counters from both the client and the servers.
How do we run a load test in cloud?
Until now we saw the main options that we have available to run a load test, but we didn’t see how to run this type of test. All we have to do to run the load test on cloud is to open Local.testsettings and select “Run tests using Visual Studio Team Foundation Service” in the General tab. This is the only thing we have to do to run the test on cloud. Of course we also have to be logged into Visual Studio with an account that is connected to the Visual Studio Team Foundation Service.
What is the price?
The service is on preview for now. Each user has 2000 virtual minutes per month that they can use for the load tests. There aren’t any fixed prices. If you need more minutes for load tests all you have to do is to enter the early adoption program without additional costs. You will have 200.000 virtual minutes per month that you can use. Accepting the request takes very little time (in my case around 6 hours).
The first time I heard about virtual minutes I wondered what these are. A virtual minute is the duration of the load test multiplied with the number of users.
Once we run the tests we will have access to all the results including performance counters, failed tests, error messages and different diagrams. All the results can be exported into an Excel file that will automatically contain the diagrams that you can show to the clients.
I invite you to try the new cloud service for load tests. I was pleasantly surprised by this service and I think it will make our life easier.