Soon after upgrading the two Housing servers to FreeBSD 7, a broken script was shown to me. The script was run via cron(8) and needed to connect to a Sybase database server for the purpose of gathering dining information. The problem was that this connection was failing. Basic troubleshooting revealed that we were opening a connection and then resetting it immediately. I suspected pf(4), but I disabled it and the behavior did not change.
After a lot of debugging of the script itself (I suspected broken code in the application itself), I decided to create a very simple script that simply tried to connect and query the Sybase database.
$dbh = new PDO('dblib:host=SERVER;dbname=DB', 'USER', 'PASS');
foreach ($dbh->query('SELECT TOP 10 foo FROM bar') as $row) {
print_r($row);
}
The script failed from the command line (something about Server Not Found), but worked when executed via Apache/mod_php. At this point I had played with the settings in freetds.conf and debugged enough to know that my configuration was correct. I compared phpinfo() output via Apache to CLI, and both were basically the same. Throughout this time I also tried to verify that things were built correctly. I even tried upgrading to FreeTDS 0.82 (0.64 is the only one in ports). None of these things fixed it. Debugging the code in FreeTDS revealed that connect(2) was returning -1 but not setting errno. I discussed this on their mailing list—at this point it looked like a FreeBSD bug to them, and I agreed. But the same script worked on Alvis (ResTek development server) and any other server I tried (all were on FreeBSD 7). I had rebuilt the ports enough times with various options that I was fairly certain nothing was linking to an old library. I did this using truss(1), ldd(1) and various other debugging tools, none of which revealed anything obvious.
Last night I decided to do this the tedious way. I reinstalled each PHP extension (that wasn’t included in the lang/php5-extensions meta-port) a few at a time and tested whether or not the script ran. Long story short, installing the php5-pdo_oci or php5-oci8 ports broke the script. This of course made no sense to me. There are, however, local FreeBSD port-specific patches in both of those ports.
Both of these change the way that the linking is done. My understanding is that they add the working directory (when it builds) to the linker path, in this case when searching for “oci8”. What I think might be happening is that there is an exported symbol whose name conflicts with one that php5-pdo_dblib uses. Could it be “connect”? I’m not entirely sure. I am not experienced at debugging these sort of things, but I wouldn’t be surprised if that was the issue.
Using nm(1) with grep reveals something interesting:
Script breaks (with patch included in port):
Script works (after removing patch):
As I said, most of this is purely speculation but what I’m guessing is that when built with those included patches (that add the working directory to the linker path) the connect symbol name (included in one of the object files in the working directory) conflicts with the standard connect(2) in libc, but the former is used.
This might explain why the connect() would return -1 but not set errno?
I’d be curious what somebody with more knowledge about this would add/change about this hypothesis.