PHPDBUnit – Testing DB interaction with PHPUnit

It’s been a while since I have put anything in the heavily neglected little site of mine so I thought I would start posting some updates on a few little projects I have been working on. I have been spending a considerable amount of time traveling for work so I am not quite as far along in these projects as I would like to be, but I have been making some major head way.

The project I would like to talk about right now is a port of Java’s DBUnit for php that will compliment the PHPUnit test suite. For those of you not familiar with DBUnit, it is a framework that allows you to easily and quickly test php code that modifies and works with database data. The whole idea behind unit testing is testing functions using well defined input and then comparing the output of that function with the values that you expect the function to produce. With database code this is somewhat difficult because you have to ensure that your database is in a known state prior to running your tests. Currenty, in order to do this properly you would find yourself adding significant amounts of code to your test to simply enter data into your database and then you will add even more code to compare the resulting database values with what your expected output. Ending up with something like this:

pdo = new PDO(SOME_PDO_DSN);
        $this->pdo->exec("TRUNCATE test_table");
        $this->pdo->exec("INSERT INTO test_table (test_table_id, col1, col2, col3) VALUES (1, 'value1', 'value2', 'value3'),(2, 'value4', 'value5', 'value6')");

        $this->test_class = new Foo($this->pdo);
    }

    public function tearDown()
    {
        $this->pdo = null;
    }

    public function testDeleteByCol1()
    {
        $this->test_class->deleteByCol1('value1');

        $statement = $this->pdo->query("SELECT * FROM test_table");
        $rows = $statement->fetchAll(PDO::FETCH_NUM);

        $this->assertEquals(array(2, 'value4', 'value5', 'value6'), $rows);
    }

}
?>

This is a fairly simple example and they really only get more complicated from this point forward. In order to the equivelant test using PHPDBUnit you would create a test class as follows:

test_class = new Foo($this->getConnection());
        parent::setUp();
    }

    /** Must be defined in all dbunit tests **/
    protected function getDataSet()
    {
        return new PHPDBUnit_DataSet_FlatXML('_data/dataset.xml');
    }

    /**
     * This is just an arbitrary function that is getting
     * a database connection...this could be implemented anyway
     * you see fit.
     */
    private function getConnection()
    {
        /**
         * I don't necessarily advocate this method of getting a
         * connection. This part of DBUnit has just not been
         * implemented fully yet.
         */
        return $GLOBALS['phpdbunit_connection'];
    }

    public function testDeleteByCol1()
    {
        $this->test_class->deleteByCol1('value1');

        $this->assertDataSetEquals(new PHPDBUnit_DataSet_FlatXML('_data/expected.xml'), $this->getConnection()->getDataSet());
    }

}
?>

You will also have to provide two xml files for the above code. This first xml file (dataset.xml) is used to seed the database before every test. The second xml file (expected.xml) is used to compare your database after the test is ran. These two files allow to get the code seeding your database out of your test cases and into their own xml files. There are also other options for generating data sets, I will discuss these options as well as the xml formats for datasets in a future post.

The database connection functionality is built overtop of PDO and support is planned for all standard pdo drivers. The big question mark for me right now is the ODBC driver. The PDO connection is wrapped by a PHPDBUnit_Database class that provides the functionality to generate datasets from the database to test your data and ensure it is correct.

The seeding is controlled by two functions that I did not choose to override in my example: getSetUpOperation() and getTearDownOperation(). These functions simply return a database ‘operation’ that is to be performed on the database connection using the data set returned by getDataSet. The default setUp operation is a clean insert. This is a truncate operation immediately followed by an insert of all data in the dataset. The default tear down operation is a null operation. You have several built-in possibilities for these operations such as Truncate, refresh, and update. You can also create your own operations if necesary.

My plan is to keep the interface of PHPDBUnit as close as possible to DBUnit while maintaining a more phpish way of doing things. I am planning on the first beta release of PHPDBUnit in July. Of course I welcome feedback on any features you would like to see. In the coming days and weeks I will continue to post more information and examples of how to use DBUnit in your tests.

Posted in PHP | 10 Comments

81.4 is evil

I know many of you all know pretty well that floating point precision and computers don’t play nicely with each other. I think I learned about it in the first book on ‘C’ I ever read, but I had never really been bitten by it until today. I was working with a piece of code today at the office that was throwing an error saying two values weren’t zeroing out when they clearly should have been. I narrowed down the problem and after a bit of time decided that 81.4 is a terrible, terrible number. For example:

<?php
$total = 100 - 81.4;
//$total should be 18.6
echo "18.6 == {$total}? " .
    ($total == 18.6 ? 'yes' : 'no') .
    "\n";
var_dump(18.6 - $total);
?>

So, let that be a lesson to us all. No matter how simple the calculation, don’t blindly trust floats.

Posted in PHP | 7 Comments

Come Work With Me!

I have recently been very encouraged by the number of job opportunities available to PHP developers lately. In light of this I thought I would make everyone aware of yet another excellent opportunity for a highly motivated, well-versed PHP programmer. The company I recently started working for (The Selling Source) is still hiring PHP Developers. I can say without a doubt that it’s a great company to work for and they take very good care of their programmers. Read on for details.
Continue reading

Posted in PHP | Leave a comment

A New Direction

It’s been quite some time since my last blogging. Most of the reason for this is that I have been extremely busy with my current job as well as a few other responsibilities. However, the time has now come for me to move on and try something new. Early this month I accepted an offer to work for The Selling Source in Las Vegas. It is a hard move since I have been employed by my current employer, FT11 Interactive, for my entire professional career. I am looking forward to the change and the new people I will meet. Since I will be moving away from a smaller startup type atmosphere I am hoping I will have a little more time to work on some of the little projects I have been chipping away at for the last year.

Late Static Binding

I haven’t been able to touch this in what seems like ages and have (unfortunately) completely missed the boat for PHP 5.2. But I am hoping to take a look at the current patch, come up with some solid use cases and tests, and submit the whole package to PHP Internals for review.

Scriptable Browser for PHP 5.1+

This is actually a fairly new project and one that I haven’t put alot of time into yet. The goal behind this is to create the scriptable browser and then soon thereafter create a binding for it so it can be easily used with PHPUnit 3. I am sure there are at least 3 or 4 people wondering why I don’t just work with SimpleBrowser or Selenium, and the simple answer to that is: Cause that’s no fun.

Refactoring Tools

This little project is actually one that I haven’t even talked about yet. I am finding that there is a real lack of tools to assist in refactoring php scripts. What tools we do have (aside from testing tools) are not at all php specific. With the advent of Greg Beaver’s PHP lexer it has all of a sudden became fairly trivial to do in-depth source code analysis. I have almost completed a small code metrics suite and will (hopefully) be releasing it to the public soon.

A Bigger, Badder Looking Site

I promised myself around 2 years ago that I would create a site that wasn’t a s9y stock theme, I have yet to do that, so that’s also going to be on my agenda for the very near future.

So, I have to thank FT11 Interactive and it’s management for 5 excellent years, and while I am sad to be leaving I would be lying to say I am not excited about this new opportunity. Hopefully now I can hit some of those conferences :P .

Posted in PHP | 2 Comments

Finally used PHPUnit and I like it

I have been working quite a bit lately with the cvs version PHPUnit3. I have spent alot of time in testing using SimpleTest and I have also given some time to test-more.php. However, up until just recently I had honestly never used PHPUnit before. The big motivational factor for me (other than to try something new) was to see how the code coverage feature was coming along.

I would have to say that after a few weeks of tinkering with it I am impressed. Now, keep in mind that I am talking about software that is still in development, so a degree of lattitude is required. The featureset of PHPUnit 3 is great. Here are a few notable features (in my mind):

  • Mock objects
  • Code Coverage Analysis
  • Multitude of test logging options
  • Built well for testing >php5 applications

Using the framework is an absolute breeze and there is excellent documentation in the form of a free book (draft version) written by Sebastian Bergmann and published by O’Reilly called "PHPUnit Pocket Guide." The API is also documented quite nicely and serves as a good second-source for learning how to do specific things.

The most impressive part of this experience in my mind was how runnable the software is straight out of CVS. Most of the time when I am crazy enough to use CVS version I always seem to get my copy of the source at a point where it is very unstable and requires tweaking just to get the software to fulfill its basic requirements. PHPUnit 3 on the other hand was able to run in it’s standard mode without a flaw. The only thing that was missing that I felt I needed was html reports for the pass/fail status of the unit tests. The report templates and code was already to a point however that it was actually fairly trivial for me to make the changes to it that I thought would be needed to make it work. I have since passed those changes along to Sebastian and they have been added to CVS, so html reports are now in the cvs version as well. It felt good to be able to contribute a little bit to a very worthwhile project.

Sebastian has posted about the new reporting features on his blog that include some screenshots so I won’t bother to rehash over all of that. You can just go check them out on his site.

Posted in PHP Testing | 3 Comments

I <3 Regexs

I was toying around with regexs this morning and learned a few things I had just never taken the time to learn before. In any case, I came up with this:

#((?:(?<!\\)\"(?:(?:(?:[\x20\x09]*[\r\n])?[\x20\x09]+)?(?:[\x01-\x08\x0b-\x0c\x0
e-\x1f\x7f\x21\x23-\x5b\x5d-\x7e]|\\[\x01-\x09\x0b-\x0c\x0e-\x7f]))*(?:(?:[\x20\
x09]*[\r\n])?[\x20\x09]+)?\"|[^"(]+|\\["(]|^)+)(?:(?:(?:(?:(?:[\x20\x09]*[\r\n])
?[\x20\x09]+)?(?P<comment>\((?:(?:(?:[\x20\x09]*[\r\n])?[\x20\x09]+)?(?:[\x01-\x
08\x0b-\x0c\x0e-\x1f\x7f\x21-\x27\x2a-\x5b\x5d-\x7e]|\\[\x01-\x09\x0b-\x0c\x0e-\
x7f]|(?P>comment)))*(?:(?:[\x20\x09]*[\r\n])?[\x20\x09]+)?\)))+(?:(?:[\x20\x09]*
[\r\n])?[\x20\x09]+)?|(?:[\x20\x09]*[\r\n])?[\x20\x09]+))#

(of course ignoring line breaks.) An ‘atta boy’ (or girl as the case may be) to anybody that can tell me what this would be used for :P .

Posted in PHP | 7 Comments

Intercepting php mail() spam with sendmail and formail

I have noticed alot of people talking about mail header inections this past week. I have actually had a run in with this on one of my servers this last week too. On one of our servers where we host around 50 clients I have recieved in the neighboorhood of 7000 delivery failure notices to the server’s postmaster account. Almost all of them being failed spam messages. I was told that sendmail client on the server was for sure configured such that it couldn’t be used directly via smtp as an open relay, which means the e-mails were most likely coming from a compromised mail form script. So I spend an entire day tracking all of them down and patching them. I was pretty sure I got them all, but then the next day I got a bunch more failures which dated the original e-mail about 26 hours after the holes were all patched.

So, I was trying to think of a way to be able to monitor the php mail() so I can have it flag me when it detects possible spam being sent with it. Then I was reminded of an article fom the November 2005 php|architect by Ben Ramsey title mail() hacks. The article basically looks at various ways to intercept the mail() function for site testing purposes. So I am now using a variant of the same technique to catch spam being sent through php.

Php allows you to specify the path of your sendmail program via the sendmail_path ini option. Whatever program this points to is then passed a formatted e-mail message (with headers) via its STDIN whenever the php mail() program is called. So simply put you can swap out any program/script for sendmail to allow you to log e-mails, redirect it, or even modify it’s headers.

A while ago I wrote a script that I use with my servers’ postmaster accounts to notify me whenever there are any serious problems. This lets me know when there are any abnormalities (a large number of delivery failures, missing accounts, server errors, reports etc.) without me ever actually having to sift through each an every e-mail. So I figured it would be fairly trivial to just add some logic in there to sift through mail submitted via the mail() functions. The only problem was to find a way to intercept the messages.

Unix has a handy little program called formail (yes with one ‘m’) that makes it extremely easy to edit an email’s headers via the command line. So using this program I am now able to add a Bcc line with my address to any function sent with mail. In order to make it easy for my script to determine which e-mails come from php I am also adding a custom header: "x-php-formmail: yes." Also since I amp having a severe problem with injections I have taken the extra, temporary step of removing all other Bcc Addresses but my own and moving them to a different custom header.

The script looks like this:

/usr/sbin/formail -R bcc x-original-bcc -f \  -A "Bcc: my@email.com" \   -A "x-php-formmail: yes" \| /usr/sbin/sendmail -t -i

Eventually, once I am sure that I have all the holes patched I will remove the ‘-R bcc x-original-bcc’ and at that point usage of this script will be completely transparent from a programming perspective.

Curse the spammers.

For those of you who would like more information about mail header injections I found this site. It has some pretty decent information: What they are, how they happen, how you fix them, it’s all covered.

Chris Shiflett also did a podcast with Marcus Whitney recently about mail header injections.

Then of course there is always the good ‘ole php manual. The comments have quite a bit to say about mail header injections. However, I recommend caution when reading through them, as always there of some good recommendations and some very, very bad recommendations.

Posted in PHP | 3 Comments

Late Static Binding in PHP

I have just sent an e-mail to the internals list for PHP that implements late static binding. Many of you probably remember hearing about all of this a few months ago not too long after the Zend Framework webcast where they showed off an API that appeared to be somewhat impossible to create short of some poor hacks with terrible performance.

A good summary of the problem this patch fixes can be found on Joshua Eichorns blog: http://blog.joshuaeichorn.com/archives/2006/01/09/zactiverecord-cant-work/

So, for a more detailed summary :) of why this doesn’t work yet.

Take the following code:

<?php
class Foo {
    protected static $staticVar = 'Foo';
    public static function test() {
        echo self::$staticVar . "\n";
    }
}
Foo::test();

class Bar extends Foo {
    protected static $staticVar = 'Bar';
}
Bar::test();
?>

The above code seems fairly straight forward. What happens behind the scenes is that a Foo class is created and has the static function added to it’s function table (among other things.) At that time the class Foo is also added to the Foo::test function struct that is added to that table. So, the when Foo::test() is called, the engine looks for the Foo class, finds the test function in Foo’s function table and executes it.

So, when Bar is defined, it creates the Bar class. Then it notices that Bar extends foo and adds all of Foo’s functions, properties, and what not to the function table, property table, etc. Now, when this is done no additional changes are made to the Foo::test function struct to show that there is a new class (Bar) that can call it. So when you call Bar::test() it will find Bar, and find Foo::test but when the test function is ran no information is passed to the function to let it know that Bar was the class that called it. To put it more succinctly, Foo and Bar both know about ::test, but ::test has no idea who Bar is. The function will always execute in Foo’s class scope. So self:: always calls Foo::. Now, this isn’t always a problem. The only time it becomes an issue is if you have an inherited static function that wants to call another static function, static property, or constant that has been redefined in a child class.

The change I made basically allows the engine to store the class calling the static function. The caller can then be accessed using ‘static’ (as opposed to self.) This should open the door to many other framework implementation possibilities (First thing that comes to mind for me is a Rubyish Active Record object.)

So, if you take part in the PHP-DEV mailing list and you want late-static-binding let them know.

Posted in PHP | 8 Comments

TAP Compliant PHP Testing Harness

I have been doing a fair bit of thinking concerning testing over the last couple of days. This was partially driven by a couple of emails Paul and I have sent back and forth as well as a posting in his blog. I am of course a big proponent of TAP. I think it provides alot of possibilities for integrating the current libraries that offer TAP support (test-more.php, SimpleTest + TAP-Reporter, PHPUnit2.) However, having libraries that support TAP compliant output are honestly of little extra value if there isn’t a way to read it and subsequently aggregate the data.

TAP Compliancy allows you to use any framework/harness that supports TAP. However, to date, the only harness I could find for reading TAP was perl’s Test::Harness and the only framework was Apache-Test. Now, I will be the first to admit that there is a fairly steep learning curve when it comes to using Apache-Test. Once you get the hang of it, things get pretty simple. Getting the hang of it though tends to be a problem. Because of this I thought it was time for someone to create a harness in php that is capable of being used to build a framework.

So, I spent my free time the last couple of days working on such an item and this is what I came up with: test-harness.php

This is a very rough first draft, if you will. I have successfully used it to run a 1,000 file testing suite that I use for my firm’s internal API. It should be stated immediatly that my goal is NOT to mimic Test::Harness. My goal is to simply provide another option for people to use in their testing.

It is fairly simple to use. All you need to do is place it in your tests’ folder, make a few configuration changes (if necessary) and you will be ready to go. The script does depend on your test-files being self sustaining (to a point) and they must also output TAP compliant results. So basically if you can run your individual test files from the command line you should be alright.

There are three different configuration settings that need to be looked at. They can all be found at the top of the test-harness.php file. The first one is the TAP_PHP_CLI constant. This simple needs to be the path to your php-cli. The second one is TAP_PHP_CLI_ARGS. This can be used to pass command line arguments straight to php. I anticipate (as the default value shows) that this will be used most often to set the include_path for the testing scripts. The final setting is the $_EXCLUDE_FILES array. This is used to specify which files the harness should ignore when cycling through the directories. test-harness.php should of course always be set. I have also set test-more.php as I was using this library for most of my testing.

Here is a brief list of the current features:

  • Executes each test file in its own environment. This is accomplished by executing each test seperatly in it’s own php-cli process.
  • Supports the full TAP protocol. Currently there is no PHP library (afaik) that fully supports TAP however I do know there are plans to enable this in at least test-more.php. So when that happens the harness will be ready.
  • Supports Multiple levels of verbosity. -s will not output anything (this can be used in automated build processes when you only need to know if the tests as a whole pass or fail.) –detail will output the overall test results as well as the TAP output of each test. By default only the overall pass/fail result of each file and a summary of failed tests, skipped tests, and bonus tests are displayed.
  • Uses a return value to indicate overall success or fail. The program will return 1 on success and 0 on fail.

This is the first version of this file and I plan on releasing many more in the very near future. Some features I have in mind for the future are:

  • Better test filtering. Executing every file in the directory is just not always the best way to do things. I would like to set it so that only *.php (and maybe *.phpt) files are run as well as maybe introducing a default directory to store php files that aren’t to be tested. Basically I want to get rid of the $_EXCLUDE_FILES array.
  • Support for running the harness over http and recieving pretty output.
  • Make the script more interactive (optionally for the cli)
  • There is also a host of additional features that I scribbled down on a notepad. If anybody else has any good ideas you are more than welcome to post them.
Posted in PHP Testing | 4 Comments

Updated Simpletest+Apache-Test

A couple days ago I came across a post in Chris Shiflett’s blog concerning a tutorial that Geoff Young and he were giving at JApacheCon. Part of this post was giving some information on a small extension I wrote for SimpleTest called TAP Reporter.

I never realized anybody was actively using the extension and as a result I failed to develop it or even update the version I have on my site. Chris, Geoff, Marcus Baker, and I have since opened discussion on possible mods to Simple Test as well as a little discussion on how to make PHP testing an overall nicer experience. So it looks like there will be some changes to the TAP Reporter in the very near future.

That all being said, I do have an updated version of the tarball that fixes some bugs in tap-reporter.php and also enables a few more TAP featuers. The new features are support for SKIP and TODO directives as well as support for ‘bail out!.’ The code is very well documented (yay me) so I don’t think there’s much of a need for me to describe how to use it.

If you want some general information on TAP, Apache-Test and how they relate to SimpleTest then take a look at my older posts on the subject. A link to the new tar ball can be found below.

simpletest_apache-test.tar.gz

Posted in PHP Testing | Leave a comment