Monthly Archives: October 2013

HTTP random client driver – existing solutions review

I’m searching a solution (or just a library) that will allow to deliver random driver for HTTP-based applications. Ideally the tool will cover randomly state space of UI with random input to every HTML control found on page.

Firstly, lets review options that do not support JavaScript, only forms + HTTP functionality:

No JavaScript support

WebInject

webinjectWebInject is a perl-based tool using XML files for driving HTTP-based tests. Sample test specification:

<case
    id="1"
    description1="short description"
    description2="long description"
    method="post"
    url="http://myserver/test/login.jsp"
    postbody="username=corey&password=welcome"
    verifypositive="verify this string exists"
    verifynegative="verify this string does not exist"
    logrequest="yes"
    logresponse="yes"
    sleep="3"
/>

The interesting thing is that GUI for the tools shows response time graph that may be important for performance analysis of a system under test.

Mechanize

Mechanize is a Python library that mimics web browser based on Perl module WWW:Mechanize. Sample session source code:

br = mechanize.Browser()
br.open("http://www.example.com/")
# follow second link with element text matching regular expression
response1 = br.follow_link(text_regex=r"cheese\s*shop", nr=1)
assert br.viewing_html()
print br.title()
print response1.geturl()
print response1.info()  # headers
print response1.read()  # body

br.select_form(name="order")
# Browser passes through unknown attributes (including methods)
# to the selected HTMLForm.
br["cheeses"] = ["mozzarella", "caerphilly"]  # (the method here is __setitem__)
# Submit current form.  Browser calls .close() on the current response on
# navigation, so this closes response1
response2 = br.submit()

Sometimes, our application do heavily depends on JavaScript + DOM tree management to handle functionality, so we have to employ more advanced tools.

Basic JavaScript support

HtmlUnit

268HtmlUnit tries to understand JavaScript used on HTML pages to handle AJAX-based pages. It’s written in Java. Sample source code:

public class homePage {
  public static void main(String[] args) throws Exception {

    final WebClient webClient = new WebClient();
    final HtmlPage page = webClient.getPage("website name here");
    HtmlElement usrname = page.getElementByName("username");
    usrname.click();
    usrname.type("myusername");
    HtmlElement psswrd = page.getElementByName("password");
    psswrd.click();
    psswrd.type("mypassword");
    HtmlSelect select = (HtmlSelect) page.getElementById("cmbProducts");
    HtmlOption option = select.getOptionByValue("ITDirect");
    select.setSelectedAttribute(option, true);
    HtmlElement signin = page.getElementByName("SignIn");
    signin.click();
    System.out.println(page.getTitleText());
    webClient.closeAllWindows();
  }
}

HTTPUnit

HttpUnitHTTPUnit is a Java library that allows to emulate browser (has some JavaScript support as HtmlUnit). Sample source code:

    public void testGetForm() throws Exception {
        ServletRunner sr = new ServletRunner( "web.xml" );       // (1) use the web.xml file to define mappings
        ServletUnitClient client = sr.newClient();               // (2) create a client to invoke the application

        try {
            client.getResponse( "http://localhost/PoolEditor" ); // (3) invoke the servlet w/o authorization
            fail( "PoolEditor is not protected" );
        } catch (AuthorizationRequiredException e) {             // (4) verify that access is denied
        }

        client.setAuthorization( "aUser", "pool-admin" );        // (5) specify authorization and
        client.getResponse( "http://localhost/PoolEditor" );     //     invoke the servlet again
    }

Sometimes basic JavaScript implementation are not sufficient, so you have to employ full-featured browser to do the work.

In-browser solutions

XPCOM

mdn-logo-smXPCOM is an interface to Mozilla-based web browsers. You can control browser from many languages (Python, C++, …).

Greasemonkey

It’s a method to inject small piece of JavaScript to any web page. First tests have shown random driver using this tool is perfectly possible.

Java / Servlet probe based on web.xml / error-page tag

Now we have Java (J2EE Servlet based probe implementation) on board! Sample stacktrace from Java servlet:

event counter: 3
javax.servlet.ServletException: Exception details
at com.randomtest.test.ExceptionTest.doGet(ExceptionTest.java:15)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:324)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:242)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:181)
at org.jboss.modcluster.catalina.CatalinaContext$RequestListenerValve.event(CatalinaContext.java:285)
at org.jboss.modcluster.catalina.CatalinaContext$RequestListenerValve.invoke(CatalinaContext.java:261)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:88)
at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:100)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.jboss.web.tomcat.service.request.ActiveRequestResponseCacheValve.invoke(ActiveRequestResponseCacheValve.java:53)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:362)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:654)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:951)
at java.lang.Thread.run(Thread.java:679)

The installation is pretty simple (few lines in web.xml file + RandomTestErrorHandler.class on CLASSPTAH), you have to customize your server URL location:

    <!– randomtest infrastructure –>
<servlet>
<display-name>RandomTestErrorHandler</display-name>
<servlet-name>RandomTestErrorHandler</servlet-name>
<servlet-class>com.randomtest.RandomTestErrorHandler</servlet-class>
<init-param>
<param-name>RANDOMTEST_URL</param-name>
<param-value>http://localhost/randomtest-server.php</param-value>
</init-param>
<!– To show any config problems instantly –>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>RandomTestErrorHandler</servlet-name>
<url-pattern>/RandomTestErrorHandler</url-pattern>
</servlet-mapping>

<!– Error types to be handled by randomtest –>
<error-page>
<error-code>404</error-code>
<location>/RandomTestErrorHandler</location>
</error-page>
<error-page>
<exception-type>java.lang.Throwable</exception-type >
<location>/RandomTestErrorHandler</location>
</error-page>

 

PHP probe and server aggregated report added

Probe code for PHP has been prepared. Basic usage is:

putenv(“RANDOMTEST_URL”, “http://localhost/path/to/randomtest-server.php”);
require(“randomtest-probe.php”);

You can set those in common script included by every part of your application. With that any error, warning, notice are reported to the server immediately.

Additionally server uses local DB storage now to count frequency of stacktraces and shows them starting from most frequently visible, an example:

RandomTest.net report

event counter: 14
BEGIN RANDOMTEST EVENT process:/home/darek/public_html/randomtest.net/src/probes/c/test1.exe
sigsegv_handler
./test1.exe: MyClass2::myMethod2(char const*)+0×9
./test1.exe: MyClass1::myMethod1(int)+0×12
./test1.exe: main()+0×15
/lib/i386-linux-gnu/libc.so.6: __libc_start_main()+0xf3
./test1.exe() [0x8048561]: ??
RANDOMTEST EVENT END

event counter: 11
BEGIN RANDOMTEST EVENT
Undefined variable: undefined_var
/home/darek/public_html/randomtest.net/src/probes/php/test1.php:10 randomtest_error_handler
/home/darek/public_html/randomtest.net/src/probes/php/test1.php:20 fun_C
END RANDOMTEST EVENT

(…)

Note the server has collected stacktraces from two totally different environments: C++ standalone program and PHP script run on the server. That means you can track events effectively in big, heterogeneous systems.

The probe-server pair has right now the very minimal but useful functionality included. You can track your local development / QA team activities by catching every error / warning / notice that might occur during manual testing and might remain unnoticed. Full stacktrace included might help greatly with error localization and fix.

 

First basic probe for C/C++ environment

The source code for first C/C++ probe implementation has been just published. By using the following construct in your profile files (an example URL):

LD_PRELOAD=./randomtest-probe.so
export RANDOMTEST_URL=http://localhost/randomtest-server.php

You can collect crashes (SIGBUS/SIGSEGV) from any C/C++ application without using gdb, even without source code available. An example:

BEGIN RANDOMTEST EVENT process:./src/probes/c/test1.exe
sigsegv_handler
./test1.exe: MyClass2::myMethod2(char const*)+0×9
./test1.exe: MyClass1::myMethod1(int)+0×12
./test1.exe: main()+0×15
/lib/i386-linux-gnu/libc.so.6: __libc_start_main()+0xf3
./test1.exe() [0x8048561]: ??
RANDOMTEST EVENT END

Note that some C/C++ programs might need the following LFLAGS / CFLAGS:

  • –rdynamic to save symbolic names for standalone executables – linker
  • -funwind-tables to embed exceptions metadata even for C code – compiler
  • -exceptions for QT configure (if QT is used) to store unwind tables on a stack – configure