Although Symfony’s original baked-in test framework, Lime, is being replaced by PHPUnit in Symfony 2, I figure I should post a fix for those, like me, who will be stuck for the foreseeable future in Symfony 1.x with an older suite of tests.

In general, Lime works pretty well except when running multiple tests in the harness. For example, when you run a specific test or tests:

$ ./symfony test-unit util
1..14
# isEmailValid()
ok 1 - returns an integer
ok 2 - test@test.com is a valid email address
ok 3 - test@test is not a valid email address
...

Lime plays along nicely. But when you run all of your tests via the test harness like so:

$ ./symfony test-unit
happyTest......................................................dubious
Test returned status 1
sillyTest......................................................dubious
Test returned status 1
...

the results can be dubious.

“Dubious” here means that the test crashed somehow during execution and so Lime doesn’t know if it would have passed or failed. To see what’s going on, pop open symfony/vendor/lime/lime.php. Depending on your version, you should see a block like the following inside lime_harness->run():

ob_start(array($this, 'process_test_output'));
passthru(sprintf('%s -d html_errors=off -d open_basedir= -q "%s" 2>&1', $this->php_cli, $file), $return);
ob_end_clean();

Never mind that the above passthu may not work on Windows (see ticket #5437), if you comment-out the output buffering and run your tests again, you’re likely to discover that Symfony can’t autoload Symfony libraries, base classes, or what-have-you; for example:

$ ./symfony test-unit
Fatal error: Class 'sfCore' not found in D:\app\test\unit\happyTest.php on line 8
Call Stack:
0.0004 70248 1. {main}() D:\app\test\unit\happyTest.php:0

The easiest fix is to work around this whole executing tests via passthru nonsense. A good old “include”, like Pake does when you run one test at-a-time, will do just fine. Patch lime.php:

ob_start(array($this, 'process_test_output'));
$return = include($file);
ob_end_clean();

Then add a “return 0;” to the end of each of your tests. (Yes, the Lime harness expects zero to indicate success..)

Now the harness should run just fine:

$ ./symfony test-unit
happyTest......................................................ok
sillyTest......................................................ok
All tests successful.
Files=2, Tests=10

Of course, this will break the harness for the functional tests, but that’s a fix for another post.