A few simple questions to better vet offshore development vendors

A friend of mine recently ran into a fairly common problem: Trouble with an offshore vendor.  Bad offshore development can be a black hole for both time and money.

He pinged me for advice on how to evaluate if they could really do the job.  Here are some questions I recommended that he pose to them:

  • Can I receive background history of the developers working on the project? Will I be able to discuss issues with them directly?
  • What bug/issue tracking system do you use? Can I get access to this?
  • What automated testing system do you use? Will I be able to see the results of each build?

And of course, ask for a list of references.  Now, if you’re like me you will probably receive this list, verify that the companies / names are real via Google, LinkedIn, etc. and leave it at that.  Don’t.  Call at least two of the references and speak with them in detail.  Ask them about the above questions.

And if you can swing it, travel out to the meet the company personally.  Turn it into a vacation.  Better a vacation than black hole purgatory.


Format a CSV string in Twig

If you’ve ever had to format a string of values in most languages then you’re certainly familiar with the ugly logical hoops you have to jump through just to get the formatting to look good.

Here’s a nice way to do it in Twig:

So given a dirty array of phone numbers:

['123-4567', '', '321-7654']

the above snippet will strip empty values, chomp extraneous whitespace, and output:

Phones: 123-4567, 321-7654

Validating multiple files in Symfony 2.5

Symfony 2.5 implemented handy multiple file upload support, e.g. input fields that look like this:

<input id="form_resumes" multiple="multiple" name="form[documents][]" required="required" type="file" />

Unfortunately there seem to be some problems in the Validator component. For example, it’s difficult to require at least one file. Likewise the All constraint doesn’t play nicely with validation groups. Here are a few workarounds.

To require at least one file but not more than three, use the Count constraint in combination with All and NotNull:

This works around a bug where 'min'=>1 is ignored.

If you’re using validation groups, you may notice that the All constraint is not being applied. Here’s how to fix it:

Note line 11: 'groups' => 'Test',

Technically we shouldn’t have to specify validation groups for the All constraint; in fact there seems to be no way to do this in YAML. Hacking it in as an option to All on the PHP side however seems to be working for me. YMMV. And as far as I can tell this last one is fixed in 2.6.

Know your CSRF timeouts

Web development frameworks such as Symfony these days bake-in a number of security features, CSRF protection being a fairly common one. I recently made the mistake of not scrutinizing Symfony’s CSRF protection defaults, which led to occasional confusing CSRF errors after a web app had been deployed.

CSRF protection embeds a code into both the generated form and some client-side storage mechanism (cookies, though I suppose this could work with an HTML5 client-side database or whatnot). On submit, if the value embedded into the form and the cookie value match, then we know the form is submitted by the user who originally requested it.

In Symfony, the CSRF cookie lifetime can be configured, defaulting to null, which falls back to session.cookie_lifetime value from php.ini. I had originally set this to be one hour, probably based on configuration examples from a year or so back..

Anyway, app users lately have been leaving forms open for more than an hour. This is a good thing: It means they are treating our web app more like a client side app, which indeed it is meant to replace. Took some sleuthing, however, to figure out why I was seeing the occasional CSRF error when everything else seemed hunky dory. I’ve now set this to

which will retain the cookie for the length of the browser session.
Lesson learned. Know your timeouts. And for that matter, plan some time to review old assumptions.

tinyint(1) showing as bit?

If Java is showing you tinyint(1) as a boolean bitfield, that’s probably because MySQL recently changed the behavior of tinyint:

As of MySQL 5.0.3, a BIT data type is available for storing bit-field values. (Before 5.0.3, MySQL interprets BIT as TINYINT(1).) In MySQL 5.0.3, BIT is supported only for MyISAM. MySQL 5.0.5 extends BIT support to MEMORY, InnoDB, BDB, and NDBCLUSTER.

To convince your JDBC driver to show you tinyint for the tiny little int that it actually is, rather than as a boolean, try setting the tinyInt1isBit=false parameter when connecting.

Fixing the Dubious Symfony Test Harness

Although Symfony’s original baked-in test framework, Lime, is being replaced by PHPUnit in Symfony 2, I figure I should post a fix for those, like me, who will be stuck for the foreseeable future in Symfony 1.x with an older suite of tests.

In general, Lime works pretty well except when running multiple tests in the harness. For example, when you run a specific test or tests:

$ ./symfony test-unit util
# isEmailValid()
ok 1 - returns an integer
ok 2 - test@test.com is a valid email address
ok 3 - test@test is not a valid email address

Lime plays along nicely. But when you run all of your tests via the test harness like so:

$ ./symfony test-unit
Test returned status 1
Test returned status 1

the results can be dubious.

“Dubious” here means that the test crashed somehow during execution and so Lime doesn’t know if it would have passed or failed. To see what’s going on, pop open symfony/vendor/lime/lime.php. Depending on your version, you should see a block like the following inside lime_harness->run():

ob_start(array($this, 'process_test_output'));
passthru(sprintf('%s -d html_errors=off -d open_basedir= -q "%s" 2>&1', $this->php_cli, $file), $return);

Never mind that the above passthu may not work on Windows (see ticket #5437), if you comment-out the output buffering and run your tests again, you’re likely to discover that Symfony can’t autoload Symfony libraries, base classes, or what-have-you; for example:

$ ./symfony test-unit
Fatal error: Class 'sfCore' not found in D:\app\test\unit\happyTest.php on line 8
Call Stack:
0.0004 70248 1. {main}() D:\app\test\unit\happyTest.php:0

The easiest fix is to work around this whole executing tests via passthru nonsense. A good old “include”, like Pake does when you run one test at-a-time, will do just fine. Patch lime.php:

ob_start(array($this, 'process_test_output'));
$return = include($file);

Then add a “return 0;” to the end of each of your tests. (Yes, the Lime harness expects zero to indicate success..)

Now the harness should run just fine:

$ ./symfony test-unit
All tests successful.
Files=2, Tests=10

Of course, this will break the harness for the functional tests, but that’s a fix for another post.

Getting Flymake to work with Emacs nXhtml

If you develop in PHP on Emacs, then you’ve probably got a hook setup to run flymake-mode on PHP files. And if so, then you’ve also probably noticed that this doesn’t work with nXhtml, which will often return an error like “mumamo can’t find the file” (or some such), causing flymake-mode to disable itself.

The culprit is likely to be a bind like this:

(add-hook 'php-mode-hook (lambda () (flymake-mode t)))

which says, “turn on flymake-mode when php-mode is started”. (Sacha Chua has a post that recommends using this from couple of years back.) The problem, as far as I can tell, is that nXhtml tries to process the fleetingly available _flymake file rather than the original PHP script whenever it toggles into php-mode.

A better hook to use is the find-file-hook, like this:

(add-hook 'find-file-hook 'flymake-mode)

With this, Flymake will properly validate PHP chunks in nXhtml mode, as well as any other files that Flymake is smart enough to process.