Load Testing Web Services with Grinder

by Jim Alateras

Since finishing the first version of the Work Manager web service I have been looking at load test tools. During my search I revisited a load test framework called Grinder.


I first came across Grinder a year or so ago but never got beyond installing it on my machine. The latest version, which is still in beta, has a lot of new features, including support for the Jython scripting engine.
The latest version also has a console, which collects and displays timing information generated by the load tests.


First impressions have been very positive indeed. I was able to install and run a simple Jython-scripted example in less that 1 hour. After one day with Jython and Grinder I was able to leverage some existing JUnit test cases to do some ad hoc load testing on my web service.


Initial contact with Jython has also been very positive and I intend on learning more about this scripting language. I found the Introduction to Jython Part 1 and Part 2 tutorials, from IBM devWorks, very useful.


Below is a simple Jython test script, which can be executed by Grinder. (Check the Grinder Tutorial for more details)



from net.grinder.script import Test;
from net.grinder.script.Grinder import grinder;
from com.comware.wm.test.rest.load import RestWorkManagerSession;

log = grinder.logger.output;
test = Test(1, "Create, Accept and Complete");
class TestRunner:
def __call__(self):
session = test.wrap(RestWorkManagerSession());
session.getMyWorkList();


This test basically leverages an existing Java class called RestWorkManagerSession (partially shown below) and makes a call to the Work Manager web service to retrieve my work list.



public class RestWorkManagerSession extends AbstractRestTestCase {
...
public WorkItems getMyWorkList() {
String myWorkListUrl = getLocationUri() + _sessionId +
"/workLists/myWorkList";
try {
return getWorkList(myWorkListUrl);

} catch (Exception exception) {
// convert to a runtime exception

throw new RuntimeException("getMyWorkList",
exception);
}
}
}


Grinder uses a property file to parametize particular aspects of a load test. In the property file you can configure the number of processes, threads and iterations for a instance of grinder. You can also configure sleep time between each iteration, which is useful for simulating web service transaction rates.



#
# Example grinder.properties
#
grinder.processes=1
grinder.threads=2
grinder.runs=10

grinder.jvm.classpath=build/classes;build/test
grinder.jvm.arguments=-Dpython.home=/applications/jython-2.1

grinder.useConsole=false
grinder.consolePort=6372

grinder.logDirectory=logs
grinder.numberOfOldLogs=2
grinder.logProcessStreams=false

grinder.initialSleepTime=500
grinder.sleepTimeFactor=0.01
grinder.sleepTimeVariation=0.005

grinder.script=src/test/load/grinder/CreateAcceptCompleteScenario.py


To run grinder you simply pass it the property file


grinder.bat grinder.properties


The next step is to determine whether I can configure grinder to execute the following load test


  • 10 users concurrently retrieving work lists every 5 seconds and

  • 5 users concurrently creating work items every 10 seconds and

  • 3 users accepting and completing work items every 15 seconds




Four years ago a collegue and I created openexec , which was supposed to do just that. This is what the configuration file looked like. (There is even some documentation).



<?xml version="1.0"?>
<OpenExecConfiguration>
<Scheduler />
<Logger />

<Users>
<User userName="chris" />
<User userName="jima" maxConcurrentThreads="1" />
<User userName="jimm" maxConcurrentThreads="2" />
</Users>

<Scenarios>
<Scenario className="SleepExecutable" scenarioName="sleep100">
<Properties>
<Property name="sleep.time" value="100" />
</Properties>
</Scenario>
<Scenario className="SleepExecutable" scenarioName="sleep1000">
<Properties>
<Property name="sleep.time" value="1000" />
</Properties>
</Scenario>
<Scenario className="SleepExecutable" scenarioName="sleep10000">
<Properties>
<Property name="sleep.time" value="10000" />
</Properties>
</Scenario>
</Scenarios>

<UserScenarios>
<UserScenario interval="500" delay="0" serialSceduling="false"
executionCount="20" userName="chris" scenarioName="sleep1000"/>
<UserScenario interval="500" delay="15000" serialSceduling="false"
executionCount="20" userName="jima" scenarioName="sleep1000"/>
<UserScenario interval="500" delay="40000" serialSceduling="false"
executionCount="20" userName="jimm" scenarioName="sleep10000"/>
</UserScenarios>
</OpenExecConfiguration>


...the CVS logs show that it hasn't been touched for 4 years :-(