dead fish

Debugging with MacPorts PHP binaries and Eclipse PDT 3.0

You know the times, when things should really go fast and easy, but you fall from one nightmare into another? Tonight was such a night… but lets start from the beginning.

To debug PHP you usually install the excellent XDebug and so did I with the port command sudo port install php5-xdebug. After that php -v greeted me friendly on the command line already:

PHP 5.3.8 (cli) (built: Sep 22 2011 11:42:56) 
Copyright (c) 1997-2011 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2011 Zend Technologies
  with Xdebug v2.1.1, Copyright (c) 2002-2011, by Derick Rethans

Eclipse Indigo and Eclipse PDT 3 was already installed, so I thought it should be easy to set up the XDebug debugging option in Eclipse. Under “PHP > PHP Executables” I therefor selected /opt/local/bin/php as my CLI version and selected “xdebug” as debugging option.

A first test however showed me that the execution of a test script did not load any module into the PHP interpreter beforehand (for reasons I could only guess, because Eclipse error log kept quite). Looking at the output of phpinfo() from my test script and php -i from command line showed me the difference: The PHP option “Scan this dir for additional .ini files” was empty when PHP ran inside Eclipse, but was properly set when PHP ran from command line (or in an Apache context).

Asking aunt Google brought up this issue that shed some light into my darkness: The directory where additional modules reside is configured as a compile time option in PHP and defaults to /opt/local/var/db/php5 on MacPorts and exactly this can be overridden by either calling PHP with -n -c options or by setting the PHP_INI_SCAN_DIR environment variable.

Having no access to the actual PHP call from inside Eclipse I tried to go down the environment route, but that did not lead to any success. While the variable was recognized as it should on the normal command line (e.g. PHP_INI_SCAN_DIR= php -i disabled the load of additional modules), in Eclipse’ run configuration dialog, in the section environment variables, this was not recognized at all. I tried a little harder and configured the variable inside ~/.MacOSX/environment.plist, logged out and in again, restarted Eclipse obviously, but had no luck either.

The only viable solution I came up with was to place all the single extension= and zend_extension= entries directly into my php.ini and disable the individual module.ini files altogether. At least I can now run and debug properly, but this solution is of course far from being ideal – as soon as I add a new PHP module or want to remove an existing, I have to remember to edit the php.ini myself.

By the way, I also tried to use Zend’s debugger (and PDT plugin) as an alternative. While somebody else already ranted about that the Zend guys have been unable to provide the Zend Debugger for PHP 5.3 as a standalone download (which hasn’t changed to date), PHP 5.2 debugging worked nicely with the old Zend PDT plugin.

Of course, none of my needed PHP modules were loaded and I really needed PHP 5.3 support, so I had to follow the same route the other guy did and downloaded all of the ZendServer glory (a 137MB download, yay) just to get the right ZendDebugger.so. After extracting the .pax.gz archive from the installer package I quickly found it underneath usr/local/zend/lib/debugger/php-5.3.x/, copied it to my extension directory and added an ini file to load that one instead, just to find out shortly afterwards that the Zend binary was i386 only and MacPorts of course compiled everything nicely as x86_64 – php was of course unable to load such a module.

Well, the moral of the story is – go for Xdebug and don’t loose the track. And, let us all hope that Eclipse PDT is developed further, so the remaining glitches like the one above are fixed.

Posted in Coding, MacPorts, Tips | No Comments »

Exception chaining in Java

If you catch and rethrow exceptions in Java, you probably know about exception chaining already: You simply give the exception you “wrap” as second argument to your Exception like this

try { ... }
catch (Exception e) {
  throw new CustomException("something went wrong", e);
}

and if you look at the stack trace of the newly thrown exception, the original one is listed as “Caused by:”. Now today I had the rather “usual” use case of cleanup up a failing action and the cleanup itself was able to throw as well. So I had two causing exceptions and I wanted to conserve both of them, including their complete cause chain, in a new exception. Consider the following example:

try { ... }
catch (Exception e1) {
  try { ... }
  catch (Exception e2) {
     // how to transport e1 and e2 in a new exception here?!
  }
  throw e1;
}

My idea here was to somehow tack the exception chain of e1 onto the exception chain of e2, but Java offered no solution for this. So I hunted for my own one:

public static class ChainedException extends Exception {
  public ChainedException(String msg, Throwable cause) {
    super(msg, cause);
  }
  public void appendRootCause(Throwable cause) {
    Throwable parent = this;
    while (parent.getCause() != null) {
      parent = parent.getCause();
    }
    parent.initCause(cause);
  }
}

Now I only had to base the exceptions I actually want to chain on ChainedException and was able to do this (in fact I based all of them on this class):

try { ... }
catch (ChainedException e1) {
  try { ... }
  catch (ChainedException e2) {
    e2.appendRootCause(e1);
    throw new ChainedException("cleanup failed", e2);
  }
  throw e1;
}

Try it out yourself – you’ll see the trace of e1 at the bottom of the cause chain of e2. Quite nice, eh?

Posted in Coding, Tips | No Comments »

guitone license change

Guitone, my little GUI frontend for the monotone SCM, is currently licensed according to the terms of the GPLv3+ and was previously – before version 0.8 – licensed under GPLv2+. Newer development however forces me to re-license it again, this time under slightly less restrictive “copyleft” terms, under LGPLv3.

The reason for this is my usage of the Graphviz library to render monotone revision trees. Graphviz is released under EPL (older versions under CPL even), and this license is strictly incompatible to any pure GPL version. I contacted the Graphviz guys and I also contacted the legal affairs support of the FSF and checked the options I had and the result is now that I have to adapt, one way or another. I could either use the “command line interface” of Graphviz or choose another license for guitone. I didn’t want to go the first route, simply because I have a working implementation and because I didn’t want to make slow calls into external binaries, so a new license had to be chosen on my side.

So, starting with the upcoming version 1.0 guitone is licensed under LGPLv3. I contacted the previous contributors of guitone and all parties are ok with the license change, so the actual license change will pop up in the current development’s head shortly.

Thanks for your interest.

Posted in guitone | No Comments »

Access the Android menu in VirtualBox on a Mac host

If you’re desperately trying to get the Menu button in an Android x86 installation working under VirtualBox on a Mac OS X host – whose keyboard of course doesn’t have this “context” / “menu” key Windows keyboards have on the right – you might find the touch-only-device mode in Android x86 handy:

1. Click on the clock in the status bar to enable / disable this mode altogether
2. A swipe from the left to the right emulates the Menu button function
3. A swipe from right to left emulates the Back button function
4. Simply clicking on the status bar brings you to the Home screen

(Source)

Posted in android, Free Software, Tips | No Comments »

vsftpd setup – the missing information

So I set up vsftpd on my Debian box the other day. I wanted a simple virtual users setup, so I created /etc/vsftpd, moved and symlinked my vsftpd.conf into this directory (to keep Debian happy) and also added a /etc/vsftpd/users.txt file – the source for my db(1) database that pam should use, after I compiled it with

$ cd /etc/vsftpd && db4.6_load -T -t hash -f users.txt users.db

Now that the database existed I went to /etc/pam.d/vsftpd and configured it there

session optional    pam_keyinit.so force revoke
auth    required    /lib/security/pam_userdb.so \
                    db=/etc/vsftpd/users.db
account required    /lib/security/pam_userdb.so \
                    db=/etc/vsftpd/users.db

but when I tried to log in, vsftpd always reported 530: Login incorrect (and of course I specified an existing user with a correct password). So what went wrong?

After struggling with it for quite some time I noticed that my /var/log/auth.log contained output from pam:

vsftpd: pam_userdb(vsftpd:auth): Verify user `foo' with a password
vsftpd: pam_userdb(vsftpd:auth): user_lookup: could not open database `/etc/vsftpd/users.db': No such file or directory

Huh?! Of course /etc/vsftpd/users.db exists – though it is only read-/writable by root (600), this shouldn’t matter much, because vsftpd runs as root anyways.

Well, the nice thing about the internet is that there is usually at least one person who already had the same problem like you and eventually solved it – and that was the case here as well:

It turned out that pam_userdb.so silently appends .db to the given path, so all I had to do to make it work was stripping off my .db in /etc/pam.d/vsftpd:

session optional    pam_keyinit.so force revoke
auth    required    /lib/security/pam_userdb.so \
                    db=/etc/vsftpd/users
account required    /lib/security/pam_userdb.so \
                    db=/etc/vsftpd/users

If you look into pam_userdb(8) you won’t find any hint about that – even worse, the example in the man page uses the explicit .db suffix as well (at least here on Lenny).

Anyways, I have now an easy-to-manage ftp server and one reason less to trust anyhow into DropBox and friends

Posted in Debian/Ubuntu Administration | No Comments »

Einführung die Versionsverwaltung mit monotone

Mathias Weidner hat eine deutschsprachige Einführung in die Versionverwaltung mit monotone veröffentlicht. Er behandelt darin die Grundlagen der verteilten Versionsverwaltung, die ersten Schritte mit monotone, sowie die täglich anfallende Arbeitspraxis mit einem Versionskontrollsystem. In späteren Kapiteln widmet sich Mathias auch erweiterten Themen, etwa wie monotone auf die individuellen Bedürfnisse des Nutzers angepasst werden kann und liefert weitere nützliche Hinweise für Ein- und Umsteiger von anderen Versionskontrollsystemen.

Das Buch ist CC BY-SA 3.0 lizensiert und kann als Paperback über lulu bezogen werden. Eine Vorversion in PDF-Format ist ebenfalls verfügbar; da der Autor die Quellen selbst in monotone verwaltet, sollten auch diese demnächst verfügbar sein.

Vielen Dank an Mathias für seine Bemühungen!

Posted in Free Software, German, monotone | No Comments »

Taxbird nun in MacPorts verfügbar

Der ein oder andere kennt Stefan Siegls kleines Programm Taxbird eventuell schon: Es erleichtert Selbstständigen unter Linux die Abgabe von Umsatzsteuervoranmeldungen gegenüber dem örtlichen Finanzamt.

Mac-User blickten jedoch bislang weitesgehend in die Röhre, da der offizielle Elster-Client bis heute nur unter Windows verfügbar ist. Da die Komponenten von Taxbird (die mit Gtk geschriebene GUI und eine Bibliothek zum Datentransfer, die auf den passenden Namen “libgeier” hört), Standard-Bibliotheken und Tools benötigt, habe ich mich an einen Port gemacht und diesen heute fertiggestellt.

Wer das ausprobieren möchte, der möge einfach sudo port install taxbird ausführen. Beim ersten Start erkundigt sich Taxbird nach html2ps, welches optional zur Übersetzung des HTML-Reports in PostScript genutzt werden könnte. Ich habe hierfür bislang auf einen weiteren Port verzichtet, da html2ps weder über eine Standardinstallationsroutine verfügt und außerdem etliche weitere Abhängigkeiten mit sich bringen würde. Die HTML-Reports sollten daher vor dem Versand einfach abgespeichert werden.

Feedback ist willkommen!

Posted in Free Software, German, MacPorts | 2 Comments »

ACLs on Mac OS X

This is a short follow-up of an earlier post where I explained how one can set ACLs on FreeBSD. Today now I’ll do the same on another BSD variant, namely Mac OS X, and guess what, the guys from Cupertino implemented ACL management in a completely different manner.

The first thing to notice is that starting with Mac OS X 10.6 (Snow Leopard) ACLs are always enabled and cannot be disabled as in earlier versions. All ACL commands are baked into the chmod command and parsed from a string you give it with the ‘+a’ option. The basic syntax here is

$ chmod (+|-|=)a#? '<user> (allow|deny) <list-of-permissions>'

Since there is no setfacl on Mac OS X, there is no getfacl either, so ACLs are instead queried by the special option -e of ls:

$ ls -le .
[...]
-rw-r--r--+ 1 john users  175  5 Jun 00:23 foo
0: user:dave allow write

Permissions include the usual read, write, delete, add_file, and add_subdirectory as well as more exotic ones like {read,write}extattr, {read,write}writesecurity and chown. (Read up chmod‘s man page what these are for.)

There are, however, two more important ones to notice, namely file_inherit and directory_inherit. These two let you spread your permissions nicely to sub objects and thus let you for example set up a directory, in which a pool of users is allowed to access, modify and delete each other’s files:

$ chmod +a 'john allow read,write,delete,add_file,add_subdirectory,file_inherit,directory_inherit' /data
$ chmod +a 'dave allow read,write,delete,add_file,add_subdirectory,file_inherit,directory_inherit' /data

The above example gives john and dave inherited read, write and delete permissions to all file objects underneath /data.

Since ACLs are executed in order, they can also be set in an ordered manner. chmod has the +a# option for that, where # is the position into which the ACL should be added. Similarily, existing ACLs can be edited with =a#, where again # marks the position of the ACL to edit, and deleted with -a#.

Finally, if one wants to get rid of all ACLs of a specific node, chmod -N <path> will do the job.

Thats it, have fun playing with ACLs on Mac OS X!

Posted in Tips | 3 Comments »

SQLite to MySQL

Migrating a database dump from SQLite to MySQL can be somewhat of a hassle. After various trials I came up with the following that worked for me quite well:

echo .dump | sqlite3 mydb.db |\
  egrep -v "\b(BEGIN TRANSACTION|COMMIT|PRAGMA|sqlite_sequence)\b" |\
  perl -pe 's/^([^'\''"]*)"([^"]+)"/\1`\2`/' |\
  perl -pe 's/\bautoincrement\b/auto_increment/' |\
  mysql -uroot mydb

The most important line is probably line 3: SQLite encapsulates all table and column names in double quotes and this is not understood by MySQL, so we’re replacing them with backticks. One must be careful here, though, because data INSERTs might also contain double quotes and to avoid having these wrongly replaced as well we only replace them until we encounter the first single quote (these are used in SQLite to encapsulate strings).

Line 3 might also have to be called several times in a row in case multi-column indexes are part of the schema as well, as it will only replace the very first, but no subsequent matches. (I couldn’t get zero-width lookbehind to work properly to remedy this problem; help is welcome.)

Posted in Free Software, Tips | No Comments »

jazzlib – an alternative for reading ZIP files in Java

Java had zip-reading capabilities for a long time, naturally because jar files are simply compressed zip files with some meta data. The needed classes reside in the java.util.zip namespace and are ZipInputStream and ZipEntry.

Recently, however, ZipInputStream gave me a huge headache. My use case was as simple as

• read the zip entries of a list of zip files (each varying in size, but usually around 20MB)
• skip to the zip entry that has a certain name (a single text file with only two bytes of contents)
• read the contents of this zip entry and close the zip

Doing this for about 25 files took my Pentium D (2GHz) with 3GB of RAM roughly 20 seconds. Wow, 20 seconds really? I created a test case and profiled the code in question separately with YourKit (which is a really great tool, by the way!):

It got stuck quite a bit in java.util.zip.Inflater.inflateBytes – but that seemed to use native code, so I couldn’t profile any further.

So I went on and searched for an alternative of java.util.zip – and luckily I found one with jazzlib, which provides a pure Java implementation for ZIP compression and decompression. This library is GPL-licensed (with a small exception clause to prevent the pervasiveness of the GPL) and comes in two versions, one that duplicates the single library classes underknees java.util.zip (as a drop-in replacement for JDK versions where this is missing) and one that comes in its own namespace, net.sf.jazzlib.

After I went for the second version, I restarted my test and it only took about 7 seconds this time. At first I thought that there must be some downside to this approach, so I checked the timings for a complete decompression of the archive, but the timings here were on par with the ones from java.util.zip (roughly 5 seconds for a single 20MB file).

I haven’t tested compression speed, because it doesn’t matter much for my use case, but the decompression speed alone is astonishing. I wonder why nobody else stumbled upon these performance problems before…

Posted in Coding, Work | No Comments »