Testing PHP

by Chris Shiflett

Related link: http://shiflett.org/archive/80

Testing PHP

I guess this is my belated ApacheCon blog. I had a lot of fun as usual, and
I got a chance to meet a few new people and hang out with old friends. The
that Geoff and I gave
went really well, and it ended up being as funny as we had hoped. More
exciting than the talk is the project behind it. I think we've created the
single best testing framework for PHP applications, and I'm going to try to
describe it for those who missed the talk.

There are a couple of resources available for download:

What's so great? Apache-Test -
a testing framework for Apache that adheres closely to strict testing
ideologies. Geoff and I have added features to Apache-Test that let you use it
to test PHP applications. In addition, we have provided a PHP implementation of
- a popular CPAN module that has some very simple functions that make writing
tests easy.

The idea for this project began at
OSCON, where Geoff and I
were discussing testing tools and methodologies. The Perl community has
embraced testing as a key component of a developer's skillset, whereas the PHP
community still relies heavily upon echo
and manual testing with a browser. Since Geoff is a Perl guy, I'm a PHP guy,
and we happen to be good friends, we decided to create a way to let PHP
developers benefit from Perl's mature testing tools and methodologies. Since
Geoff is really a mod_perl guy and interested in all things Apache, he is a
big fan of the Apache-Test framework. This seemed like a great tool to provide
to PHP developers, so we proposed a talk about it. Once the talk was accepted,
we had to deliver.

What makes Apache-Test so great for PHP developers? It provides several key
features not available (to my knowledge) with any other testing framework:

  • You don't have to write any Perl. This certainly isn't a unique
    feature, but one problem with our attempt at being funny in the naming of
    the talk is that people tend to think that using this framework requires
    you to know Perl, and that's just not true.

  • You don't have to write any tests in your application. One thing that
    bugs me about several of the testing approaches I have seen is the
    necessity of including tests within the application itself. Thus, your
    tests can affect the behavior of your application, which totally ruins the
    concept of testing. You see, testing an application should be an example
    of a
    control experiment,
    where there is only one thing that changes - your application. If all
    tests pass, you make changes to your application, then some tests fail,
    you want to be assured that the cause of the problem is the changes you
    made to your application, not tests that you have written. In fact, a test
    shouldn't even affect any of the other tests, much less the actual
    application you're testing. I think this obvious flaw in existing
    approaches has stymied our adoption of testing as a necessary skill.

  • Writing tests is easy. I think this is a key feature. If writing tests
    is a hassle, no one will want to do it. Let's be honest - developers are
    lazy. In fact, we take pride in being lazy. The PHP implementation of
    Test::More that is now included in Apache-Test provides simple functions
    for writing tests. You can use these with any programming paradigm,
    which means you don't have to use an object-oriented design, but you can
    if you want.

  • The tests are repeatable. This means that you can repeat your entire
    test suite as many times as you want, and running the tests doesn't affect
    the environment (or future iterations of the same tests) in any way.

  • The tests are automatic. You don't have to manually browse your
    application or execute tests. You just type make test, and
    Apache-Test takes care of everything else.

  • You get a self-contained and pristine Apache environment. Simply not
    including tests in your code isn't good enough - you want to have as much
    consistency in your environment as possible. Apache-Test gives you a
    separate Apache environment with its own configuration (including a
    pristine httpd.conf and php.ini) using your build. You augment the default configuration in a separate file, so that every change must be deliberate (it is also a good practice to keep your changes to the real Apache and PHP in a separate file, so that you can be sure that it has the exact same configuration as your testing environment). When
    you type make test, Apache-Test starts the server, runs your test
    suite, generates a report, and stops the server.

  • Your application is executed with the real PHP, not the command line
    client. While the command line client might behave the same way, it's not
    guaranteed to do so, and it represents a variable that ruins the ideology
    of testing.

  • It's very mature and stable. Major companies and open source projects
    have been using Apache-Test for years. It's a proven tool.

These are the major features (that I can think of) that I find particularly
appealing. I'm sure there are others. Perhaps a nice side-effect is the
potential for cross-pollination between the Perl and PHP communities.

If you're interested in learning more, please download the
demo and the
slides from our talk. To run
the demo, you'll need
Apache-Test 1.16
or greater. You can find the most recent version from

This testing framework is only available for Apache.


2004-12-03 07:05:37
irc channel
interested folks can drop by #perl-php on irc.perl.org for help, etc.