HTTP random client driver – existing solutions review

I’m searching a solution (or just a library) that will allow to deliver random driver for HTTP-based applications. Ideally the tool will cover randomly state space of UI with random input to every HTML control found on page.

Firstly, lets review options that do not support JavaScript, only forms + HTTP functionality:

No JavaScript support


webinjectWebInject is a perl-based tool using XML files for driving HTTP-based tests. Sample test specification:

    description1="short description"
    description2="long description"
    verifypositive="verify this string exists"
    verifynegative="verify this string does not exist"

The interesting thing is that GUI for the tools shows response time graph that may be important for performance analysis of a system under test.


Mechanize is a Python library that mimics web browser based on Perl module WWW:Mechanize. Sample session source code:

br = mechanize.Browser()"")
# follow second link with element text matching regular expression
response1 = br.follow_link(text_regex=r"cheese\s*shop", nr=1)
assert br.viewing_html()
print br.title()
print response1.geturl()
print  # headers
print  # body

# Browser passes through unknown attributes (including methods)
# to the selected HTMLForm.
br["cheeses"] = ["mozzarella", "caerphilly"]  # (the method here is __setitem__)
# Submit current form.  Browser calls .close() on the current response on
# navigation, so this closes response1
response2 = br.submit()

Sometimes, our application do heavily depends on JavaScript + DOM tree management to handle functionality, so we have to employ more advanced tools.

Basic JavaScript support


268HtmlUnit tries to understand JavaScript used on HTML pages to handle AJAX-based pages. It’s written in Java. Sample source code:

public class homePage {
  public static void main(String[] args) throws Exception {

    final WebClient webClient = new WebClient();
    final HtmlPage page = webClient.getPage("website name here");
    HtmlElement usrname = page.getElementByName("username");;
    HtmlElement psswrd = page.getElementByName("password");;
    HtmlSelect select = (HtmlSelect) page.getElementById("cmbProducts");
    HtmlOption option = select.getOptionByValue("ITDirect");
    select.setSelectedAttribute(option, true);
    HtmlElement signin = page.getElementByName("SignIn");;


HttpUnitHTTPUnit is a Java library that allows to emulate browser (has some JavaScript support as HtmlUnit). Sample source code:

    public void testGetForm() throws Exception {
        ServletRunner sr = new ServletRunner( "web.xml" );       // (1) use the web.xml file to define mappings
        ServletUnitClient client = sr.newClient();               // (2) create a client to invoke the application

        try {
            client.getResponse( "http://localhost/PoolEditor" ); // (3) invoke the servlet w/o authorization
            fail( "PoolEditor is not protected" );
        } catch (AuthorizationRequiredException e) {             // (4) verify that access is denied

        client.setAuthorization( "aUser", "pool-admin" );        // (5) specify authorization and
        client.getResponse( "http://localhost/PoolEditor" );     //     invoke the servlet again

Sometimes basic JavaScript implementation are not sufficient, so you have to employ full-featured browser to do the work.

In-browser solutions


mdn-logo-smXPCOM is an interface to Mozilla-based web browsers. You can control browser from many languages (Python, C++, …).


It’s a method to inject small piece of JavaScript to any web page. First tests have shown random driver using this tool is perfectly possible.

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>