Consistent, automated baseline testing of AJAX/POX/POJO/XHTML

by Rick Jelliffe

The last few months I've been doing some consulting for Allette Systems on their software engineering procedures, based on some ideas I've been developing for Topologi. The idea is to first figure out what the state of the art is for Java coding (development environment, best practises and related topics), then to use that as the goal for tooling up other development environments for other languages to the same level: JavaScript, C++, XHTML, XML, XML Schemas, CSS, and XSLT.

Such topics as: automatic code checkers, test runners, build systems, diagramming, profilers, metrics, and so on.

Java is particularly rich in great tools: Eclipse, Findbugs, JLint, and so on. Just yesterday I found 4 potential bugs in 10 minutes of an 9early) code review using JLint to check for potential null references. But languages like JavaScript are traditionally and perhaps intrinsically poor. Moving to a more more mature and transparent software development process involves being able to demonstrate to customers and management why they should believe some software meets the desired functionality and quality.

Now sure we also have an ISO 9126 program in place, so that we can make sure we concentrate on the important issues in each project. And sure an ongoing risk management process. But as Java's tooling has improved, it has only served to more sharply show JavaScript (and XLST's) poor tooling.

The big thing that I have discovered so far out of this effort is the Selenium test system for JavaScript. It looks like revolutionizing the testing of JavaScript. So far in our testing, it works brilliantly for simple pages. But for AJAX pages which open different windows, there are persistant problems connected to JavaScript's anti-spyware single-origin visibility rules. (We have to add some hidden buttons to pages to allow communication with popup windows and their data. Not so neat, but workable.) In a way, Selenium is JavaScript what JUnit is to Java. And if Java needs JUnit, then JavaScript needs Selenium more!

3 Comments

J David Eisenberg
2006-06-28 07:45:24
Those interested in finding out more about Selenium should go to their web site.
Josh Graham
2006-07-20 05:28:52
G'day Rick,


First, thanks for getting me on the XML path nearly a decade ago. Your XML/SGML cookbook was intrinsic to learning, and other books in your series (like XML Web Apps) were highly influential.


Secondly,


As a Selenium user and ThoughtWorker, it's pleasing to note a luminary outside of the QA space touting what we believe is a great tool (it's certainly helped many of our clients with automated functional testing of web pages).


It can be used for strict testing of structure and content of any XML document - viz SOAP, XHTML, PlainOldXml (POX), but of course for this to happen, the document must be retrieved by a browser. Not the fastest way to do such a thing, but is useful for visual feedback. We also use WebUnit (HTMLUnit, HTTPUnit, JUnit) for this sort of thing.


That got me thinking of transforming the now almost defunct XMLUnit to use Schematron (you've heard of that, right? ;-]) to extend JUnit (and perhaps HTTPUnit) for XML document testing.


There's also a thought to using Schematron for Selenium assertions, on top of the basic HTML table / FIT-style assertions it comes with by default. This would give a very rich, very flexible, yet still quite simple declarative construct for specifying tests. Ruby is also wonderful if you want to craft an imperative, domain-specific testing language.


To overcome your popup blocking issues, take a crack at the new Selenium Remote Control. It obviates the need to install any Selenium stuff in your web app (it's in effect, a client-side reverse proxy) and thus you can Selenium test ANY web site. I suspect it'll remove the popup issue as the browser is retrieving all resources from "localhost". If not, we can raise a ticket as it's an important issue and AJAX pages are fully intended to be testable.

Rick Jelliffe
2006-07-20 17:57:02
Thanks Josh...a fellow Sydneysider!