I told Marcus Baker a couple of weeks ago that I would like to lend a hand in figuring out the best way to get SimpleTest working with Apache-Test. I actually was able to put quite a bit of time into this today and have achieved some results finally. Now, these results are far from a solution. All I have managed to do thus far really is to understand enough about the inner workings of Apache-Test so I can actually get a "self-created" test working inside of the Apache-Test environment.
The first thing I had to do was to get a copy of Chris Shiflett and Geoff Young’s proof-of-concept files for testing php using Apache-Test. When I first dug through them I was fairly confused there seemed to be a significant amount of perl code. Much more than I figured most php developers would be comfortable with. Almost all of these pieces of code had a note at the top of the file indicating that the code had been automatically generated. It turns out (after an hour of tinkering) that these files were actually generated by the Apache-Test framework. I was thrown off a bit at first due to the fact that before I ever ran any of the scripts in their package these files already existed. So now that I have figured that out I am making a "widdled down" version of Chris and Geoff’s example that makes how the framework works much more apparent.
First, to help some of you understand why I removed some of the files I will give you a brief overview of how Apache-Test works with php.
I can’t stress enough the importance of realizing that Apache-Test CANNOT be compared to SimpleTest or PHPUnit. Apache-Test by itself does not run any tests. It is simply a framework that is used to run tests that have already been written; previously in perl, now in PHP. Apache-Test basically just creates another instance of an apache server on a seperate server. This server is then used to perform live tests. Their example actually uses a perl library called TestRequest. The same however could most likely be done via some of SimpleTest’s browser emulation features. This is one area where I need to do more research.
Apache-Test is actually just a perl-based framework that was created in order to test various aspects of the Apache httpd server; Most notably the modules. From what I understand however, many perl developers have actually begun using the framework to assist in quick and relatively painless testing of their own perl packages and programs. Chris Shifflet is now bringomg some of that "perl testing prowess" to the php world by teaming up with Geoff Young to get a version of Apache-Test that is capable of easily running tests written in PHP.
Now, The basics of the file structure is this. The root of the testing project MUST have two things. The first being Makefile.pl and the second being the ‘t’ directory. The Makefile.pl is actually ran by perl to create a make file that will be used to launch the testing. The ‘t’ directory is the meat of the whole framework. The ‘t’ directory holds all information that is going to be specific to your testing project. There are also a file or two that will almost always be the same. I’ll get to that later however.
There are three folders within the ‘t’ directory that are VERY important. The first folder is the conf folder. This will eventually contain the configuration for both the apache server and mod_php. If you need to modify either you can do so via the extra.conf.in file. This file will accept apache directives which means via the php_value and php_flags directives you can also modify some aspects of your php configuration. The actual configuration files (httpd.conf and php.ini) are generated when you run the make test command.
The next folder is the htdocs folder. This folder actually contains a a copy of the code you are testing as if it were on a production server. This can be used to test your application at the browser level. These tests could be accomplished in SimpleTest’s WebTester package. If you are not testing an actuall web-app, just an API you have created, then chances are very good you do not really even need much in this directory
The third and final folder we need to be concerned with is the response folder. This folder is where you will actually put all of the tests you would like to run. These tests can be a combination of either unit tests or browser level tests. The file structure here is extremely important. The only folders that the framework will recognize as containing tests are the folders that begin with the word Test. The framework will be able to pick up on these files and create *.t files that will be recognized and called from the framework. That was a fairly rough overview, let me paste straight from some of Perl’s documentation.
Apache-Test will look for PHP test scripts in that match the following pattern:
Where Foo and bar can be anything you like, and t/response/Test* is case sensitive. When this format is adhered to, Apache-Test will create an associated Perl test script called t/foo/bar.t, which will be executed when you call make test. All bar.t does is issue a simple GET to bar.php, leaving the actual testing to bar.php. In essence, you can forget that bar.t even exists.
The only other thing that you have to keep in mind is that whatever code you use to run your tests, it MUST output a TAP compliant protocol. I will go into more detail on that in my next blogging. For now I will just say that to the best of my knowledge the test-more.php library (included in the package) is currently the only php library that is TAP compliant. I have plans to implement an extension(s) to SimpleTest to allow it to also output Tap compliant protocol. I just haven’t finished yet :).
I have modified the original PHP/Apache-Test package to get rid of some of the unnecessary auto-generated files. The original package again was written by Chris and Geoff, the only changes I have made are removing some files that just don’t need to be there. (Ones that are generated by make test already.) All credit for the coolness of this belongs to and should go to them.
As you look through the files most of them should be fairly apparent as to what they do. There are a few tests in the t/ folder that are used to do some browser emulation testing. The reason I have left those there is that as of now there is no way in PHP to write these tests (short of modifying an existing library or creating a new one.) This is another problem I plan to help remedy through SimpleTest.
So without any further adieu, a condensed kinda easier to dissect version of PHP working with Apache-Test: apache-test-demo-light.tar.gz