Little Nickname Science

Its not that I’m a newbie, but sometimes I just feel that way. People start and talk in acronyms on IRC, and then it comes out that these acronyms stand for apparently famous people in the Free Software world I should know…

Lets start with an easy one: rms.

Yeah, that was easy. If you come across this nick in an IRC channel, make sure you don’t talk about the advantages of proprietary software, Richard Matthew Stallman may just jump at you and bash you with a big club.

Now, who might be esr?

Actually I learned about his nick not too long ago. He stumbled into the #monotone channel on OFTC last December and asked about the backgrounds of the monotone project. He prepared a paper of modern revision control systems at that time (I only have a dead link where it used to reside), but I guess most people will rather know him from one particular essay anyways, “The Cathedral and the Bazaar”: Eric S. Raymond.

Now, if esr is Eric S. Raymond, who might be rse?

If you’re working with web servers, and here in particular with the one from the Apache Foundation, you probably know of this one swiss army knife ™ which solves all your redirecting / load balancing / other weird use case problems – mod_rewrite! But actually, the person I’m talking about, is also highly popular for being the main author / founder of other popular Open Source software project, like OpenSSL, OpenPKG or RPM5: Ralf S. Engelschall. If you catch him somewhere and you’re using his software on a daily basis (like I do), do not forget to praise him ;).

Now the last, pretty hard one for me at least, I just learned today: Who’s famous for his nick drh?

I felt pretty stupid when I learned about his nick; after all I’m using his software on a daily basis as well – indirectly at least. He’s the author of SQLite (monotone’s database backend), the creator of CVSTrac (the inspiration for the nowadays widely used Trac project) and he has of course, like the other people I introduced here, his own wikipedia entry. I’m speaking of D. Richard Hipp.

So this was my Little Nickname Science. If you’ve similar anecdotes I’d love to hear them!

Perverted Logic / Monotone Summit

If you like to have a laugh today, read this. Found via Rob Weir’s blog.


The Monotone Summit goes on and its quite fun to meet and work with all the people I just know from IRC or the mailing list. I mostly worked on automation stuff so far and improved some things in guitone. A longer blog entry with other impressions (and maybe some pictures) will follow within the next days…

A Linux tale… [updated]

My wife and I seem to share a lot of activities recently. We finally gathered a few friends around here with which we do regularily stuff like playing card games and she even got me to take Salsa dancing lessons with her. Beside that my political interests and meetings are quite regular as well, so it was just natural to think about a event planning solution which would allow us to view, add and edit events. I decided we need a calendar server!

So I started looking around for calendar server solutions. I already had a rather hacky webdav-alike setup on my Virtual Private Server (VPS) which consisted of a single ICS file, secured by a simple HTTP authentication and made writable through a PHP script, but this doesn’t seem to be the right solution for me. I wanted something better, something stable, something nice – and I apparently found it: Apple released its Darwin Calendar Server, which is the base of their commercial calendar server solution on Mac OS X Server, as Open Source under the Apache Software License 2.0 some time ago. “This will replace the old hackish setup on my VServer!”, I thought…

I logged into the server and quickly downloaded the project’s sources from the public SVN repository. The README file stated that it should apparently be enough to start the `run` script in the base directory which would take care of everything, i.e. download and install all dependencies which would be needed for the software to run. This even worked flawlessly (with only one little modification). Up to that point I was very cheerful – until I actually tried to start the server:

IOError: [Errno 95] Operation not supported: \
'/opt/calendarserver/CalendarServer/twistedcaldav/test/data'

Uh? Ah well, there was something in the README I overlooked:

WEBDAV PROPERTY STORAGE
-----------------------

For starters, twisted.web2.dav requires Bob Ippolito's xattr
library to access file system extended attributes, which are
used to store WebDAV properties.  File system extended 
attributes are available on all file systems in Mac OS X, and
on some file systems on Linux and FreeBSD.  Another 
alternative is to implement a new property store class which
does not use extended attributes. The Apache HTTP Server, for
example, uses a database to keep track of properties on files,
and this method works on many more platforms and file systems.

xattr? Hrm… I never heard of that before – but Google shed light into my darkness: As Wikipedia amongst others states, they are “[…] a file system feature that enables users to associate computer files with metadata not interpreted by the filesystem […]”, basically simple key-value pairs of arbitrary size and content. A quick grep over the config file of the currently running kernel showed me that at least support was already compiled into it:

$ cat /boot/config-2.6.11.4-20a-smp | grep XATTR
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_REISERFS_FS_XATTR=y
CONFIG_DEVPTS_FS_XATTR=y
CONFIG_TMPFS_XATTR=y
CONFIG_CIFS_XATTR=y

Cool! I read somewhere that in this case one would just have to add the mount option `user_xattr` to /etc/fstab and remount the filesystem in question and everything would be fine… Well, it would be fine if my /etc/fstab would actually contain an entry for `/`! Being on a virtual server started to get a little annoying.

But I didn’t capitulated, not yet. “Wasn’t there a way to create and mount a filesystem from a disk image?” I wondered. I used this technique a couple of times to mount ISO images and browse their contents back in my old Linux days. I read a bit further on this topic and found many helpful ressources on this topic. At first I needed to create an image file and format that with a filesystem:

# 128 blocks of 1MByte size
$ dd if=/dev/zero of=caldavdata.image bs=1M count=128
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 0.506394 seconds, 265 MB/s
$ mkfs.ext3 -v caldavdata.image
mke2fs 1.36 (05-Feb-2005)
caldavdata.image is not a block special device.
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
32768 inodes, 131072 blocks
6553 blocks (5.00%) reserved for the super user
First data block=1
16 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks: 
        8193, 24577, 40961, 57345, 73729

Writing inode tables: done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 28 mounts
or 180 days, whichever comes first.
Use tune2fs -c or -i to override.

“Great! Now I only need to mount that baby!” I thought:

$ mount caldavdata.image /opt/caldavdata -t ext3 -o loop
mount: could not find any device /dev/loop#

Indeed, there was not one single block device in /dev starting with `loop` – in fact the only two block devices I found there at all were related to the virtual server’s own file system. In my naive thinking I continued: “Hey, if this block device is not existent, I’m pretty much sure that I can create it with mknod!”. Indeed, mknod can create any kind of char or block devices. This site pointed me to the correct needed major and minor numbers for such a device:

$ mknod /dev/loop0 b 7 0 # 'b' stands for block device
$ ls -lah /dev/loop0 
brw-r--r--  1 root root 7, 0 Feb  1 21:25 /dev/loop0

“Success!” I thought and tried the mount command again:

$ mount caldavdata.image /opt/caldavdata -t ext3 -o loop
mount: Could not find any loop device. Maybe this kernel does
       not know about the loop device?
       (If so, recompile or `modprobe loop'.)

“Yeah, apparently it does not know about device” and ran the proposed `modprobe loop`:

$ modprobe loop
FATAL: Module loop not found.

Hrm… could it be that loop wasn’t compiled as module? Grepping the config told me yes:

$ cat /boot/config-2.6.11.4-20a-smp | grep LOOP
CONFIG_CC_ALIGN_LOOPS=0
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_CRYPTOLOOP=m

So, the last thing I could think of (as an ignorant Linux user) was to reboot the server and hope that it would somehow recognize the loopback file.

Of course, this did not work out.

I’m now in the sad process to get used to Google Calendar

[Update: Actually the only bad things about Google Calendar until today are

a) I don’t have full control of the server-side storage / backup

b) I can’t publish events to Google via iCal – well, at least I could not until today.

Apparently Google offers an RSS-alike API to access calendar data and somebody wrote a small Java server which “translates” between this custom RSS format and the standard ical format. It allows file-based synchronization (i.e. particularily useful for iCal which can’t just embed and write CalDAV ressources) and even contains a small web server to do the translation live and in place – how neat is that?!

You can download GCalDaemon here.]

Taking appcasts to the next level

Appcasts are RSS feeds which make use of the <enclosure> tag which links to a binary file (usually a software package) somewhere on the internet. Sparkle, a library drop-in for Objective-C/Cocoa Mac OS X development, implements and expands this format to use it as transport description for application updates. With this, Sparkle makes it possible to automatically trigger application updates in a well-defined, consistent way. It downloads, unpacks and replaces the original binary and afterwards restarts the updated application.

I’m sure most of the Mac users out there have already seen it; many popular programs support self-updates through appcasts and Sparkle (hell, even guitone has an appcast after I managed to hack Sparkle support into this Qt project), it makes the whole process of updating non-Apple applications on a Mac so much easier and fun. With the advent of AppFresh, which aggregates all the registered appcasts from all locally installed applications and then updates them all-at-once, appcasts become an interesting idea for software distribution at a larger scale.

Now, GNU/Linux users will probably laugh at me when I tell them from this “superior” technique, because appcasting is a well-known technique in the free software world: A software update (if not automatically triggered) is usually done in the background by modern Linux distributions through yum or apt, with full support for version and dependency checking, mirror localisation and much more. Interestingly, these software distribution systems never made the jump to other operating systems – and yes, I know of MacPorts and Fink, but both are different story, because they distribute mostly GNU software.

There is one thing which distinguishs appcasts from GNU/Linux distribution systems, and this is rooted in the way software is developed on both platforms. GNU/Linux software projects usually just provide the sources, which are then build for the specific distribution and platform by so-called packagers. These packagers have not necessarily anything to do with the actual project. Because there are so many distros out there, a software gets quite a lot repackaged, even if some of these packages basically serve the same platform and architecture.

Mac OS X software (and Windows software as well) is usually distributed directly in binary form, ready to use for the defined platform. The “packager” role is here an internal role of the software creator and the distribution – if not done through the software creators home page – usually happens via big download sites like macupdate.com, cnet.com and others. All these websites are isolated “applications”, meaning that they use different internal data formats, are usually closed and don’t work together in most cases.

So, the new hype, not only in content creation branch, is syndication. And that is exactly what appcasts are about: Based on RSS 2.0 they provide a general format to describe software updates. Platforms like iusethis.com use the feed and provide additional services, f.e. a global “usage counter” for a particular software package. However the problem with the current appcast format pushed by Sparkle is, that it is itself an isolated application, because it can only provide “some package with some version to download somewhere” and includes no information about other platforms, architectures or download locations. This may be ok as long as you either have different appcasts for all these different platforms or only target one platform (in case of Sparkle Mac OS X).

But what if you create an application which should run on all major platforms? You certainly want to provide your users the same look’n’feel and features on all these platforms, and this is where my proposal drops in. My aim is to create a cross-platform format for software updates which is suitable to distribute update information for any platform through one single stream of information. Based on this format people could create those kind of library drop ins like Sparkle not only for Mac OS X, but for multi platform toolkits like Qt, wxWidgets, even for .NET. Software developers would then simply add those in their projects, provide the syndicated format on a webserver, and are done.

So again, what are the advantages here?

  • Based on the specification, an application update would “feel” the same on every platform, maybe with slightly differences between the different toolkit implementations
  • The syndicated format, which extends RSS 2.0, could serve as common content feed in any feedreader and, at the same time, transport the needed update information for the application drop ins
  • The feeds are decentralized published; the power is taken back to the software creator as he determines how he publishes his software
  • Other services similar to iusethis.com could arise which would provide social functionality around software updates, caching, mirroring and more

Of course the whole proposal would probably make most sense for Windows and Mac OS X software creators, since, as I already mentioned, distribution in the GNU/Linux world happens quite differently (and these people usually don’t like to build binaries themselves, they let packagers from the different distros do this work). But why should efforts like AutoPackage not be supported, if they’re interested? And I think commercial Linux software distributors would be interested in that as well, just because it would strengthen their relationship with their customers quite a lot.

What follows is a quick effort how such a format could look like. It currently doesn’t contain the RSS part, because I think you all know how this would look alike, I plan to put one <software> node into each <item> which would then stand for an invididual software release. Below the software node, the following tree would be expanded (which should be mostly self-explanatory):


  2.2.3
  
    
      
        source
        application/x-compressed
        2928399933
        012346789ABCDEF012346789ABCDEF
      
      manual
      
        
        
      
    
    
      
        binary
        application/octet-stream
        2928399933
        012346789ABCDEF012346789ABCDEF
      
      app-replace
      
        
        
      
    
    
      
        binary
        application/octet-stream
        2928399933
        012346789ABCDEF012346789ABCDEF
      
      installer
      
        
        
      
    
  

Of course some concepts are stolen from Sparkle (f.e. hashs to check the data integrity of downloaded files), and one could put a lot more stuff in this. I once thought about adding support for “upgrade paths” into it, f.e. if a user skips a couple of versions, he should be notified if his current installation can be updated (using binary patches) or has to be upgraded (which requires a full installation), but I think bandwidth is cheap nowadays and it would make a first shot of this format and its implementation far too complex.

So, this is it, what do you think? I’d love to hear your feedback! I’m in contact with the developers of AppFresh and they signalled interest as well. Stay tuned for more news in the future!

Should Trolltech make their products support GPLv3?

Since the GNU General Public License v3 had been [released last week](http://gplv3.fsf.org/) I’m actively thinking about switching the license for some of my projects, f.e. [guitone](http://guitone.thomaskeller.biz), which are currently GPLv2 “or later”. The main reason for doing this for me personally is to protect my software even better from any possible [Tivoization](http://en.wikipedia.org/wiki/Tivoization), i.e. no patent restrictions, closed hardware, or anything else should hinder people to use, modify and distribute it further.

Looking more in detail at the license possibilities [Trolltech](http://www.trolltech.com) gives developers which use their Qt/Qtopia platform, upgrading Qt projects from GPLv2 to GPLv3 doesn’t seem to be possible: The problem is that even though Qt4 can be licensed under GPLv2, it does **not** contain the “… or – at your option – any later version” clause of the license, and since GPLv2 and GPLv3 are incompatible with each other, as Richard Stallman [told us here](http://gplv3.fsf.org/rms-why.html), you therefor can’t legally link GPLv3 code with those GPLv2 libraries:

> When we say that GPLv2 and GPLv3 are incompatible, it means there is no legal way to combine code under GPLv2 with code under GPLv3 in a single program. This is because both GPLv2 and GPLv3 are copyleft licenses: each of them says, If you include code under this license in a larger program, the larger program must be under this license too.? There is no way to make them compatible. We could add a GPLv2-compatibility clause to GPLv3, but it wouldn’t do the job, because GPLv2 would need a similar clause.

([Source](http://gplv3.fsf.org/rms-why.html))

There _might_ be a way out of this dilemma, in case Trolltech doesn’t want to put Qt/Qtopia directly under “GPLv2 or later”/GPLv3: There already seems to exist a “Trolltech GPL Exception version 1.0”, but the text is currently unavailable from the [Troll’s site](http://trolltech.com/products/qt/gplexception). A temporary copy of it can be found in the Subversion repository of the libqxt project. The Exception states that you can use a couple of other accepted Free Software Licenses in your project and still use the GPL version of Qt:

> […] 1) The right to use Open Source Licenses not compatible with the GNU General Public License: You may link software (hereafter referred to as “Your Software”) against the Licensed Software and/or distribute binaries of Your Software linked against the Licensed Software, provided that:

> A) Your Software is licensed under one of the following licenses: […]

and what follows is a list which includes licenses like the Apache Software License, the LGPL, the BSD license, the Mozilla Public License, to just name a few. Furthermore:

> […] B) You must, on request, make a complete package including the complete source code of Your Software (as defined in the GNU General Public License version 2, section 3, but excluding anything excluded by the special exception in the same section) available to Trolltech under the same license as that granted to other recipients of the source code of Your Software. […]

…this seems only fair…

> […] C) Your or any other contributor’s rights to:

> i) distribute the source code of Your Software to anyone for any purpose;

> and

> ii) publicly discuss the development project for Your Software and its goals in any form and in any forumare not prohibited by any legal instrument, including but not limited to contracts, non-disclosure agreements, and employee contracts. […]

… yes I hope so!

Now I’m unsure if this exception is granted to every (free) software project using Qt. Currently, GPLv3 is not on the list, but if this exception is valid for everyone (not just paid customers of Trolltech), maybe it could be expanded to allow GPLv3 as well and therefor serve the Open Source community

Whatever route Trolltech will go, I hope they at least notice that there is **a demand from the Open Source community** to clarify the licensing issues with respect to GPLv3, and I hope this happens in the near future…

Edit 2007-08-09

It seems as Trolltech put back the GPL Exception in place, though the GPLv3 is still not listed as a valid license. One Trolltech engineer said however recently in a comment on the Labs Blog:

… As for GPL v3, we are actively working on it. From what I understand, it is a bigger step and not something we can just provide with an Exception.

(Source)

So its good to know that they’re aware of it!