Bitrot, or, Be Sure Your Bugs Will FInd You Out

The Jargon File defines bitrot as the observation that unused programs or features will often stop working after sufficient time has passed, even if ‘nothing has changed’. If you ask me that "unused" is superfluous. In the last week two frequent users of CanalplanAC have reported the same bug, in an existing piece of code, that must have been there for the best part of a year. This is not the first time this has happened. And I'm not talking about an existing bug that has been tickled into life by some other change here, this is something that is doing just what it has always done ever since the feature was implemented (the actual bug is discrepencies between the counts of photographs submitted and the "my photographs" page).

In the same week I've found a quasi-bug in the gazetter options (there is developmental stuff in there that doesn't do anything apart from confuse people). I've also had so many new places submitted that I am finally having to get round to writing a proper, automated, way of adding them rather than cutting-and-pasting into the master data file.

It's also been pointed out to me while new places are in limbo (they've been submitted but not added) they cannot be found in a gazetter search (that's not entirely true, they can be found if you happen to remember their ID, but there is no way to find it out if you don't know it). To fix this will require a lot of work in the never-stable innards of the placefinder. It will complicate them so much that this is the prod I need to re-write this critical bit of code in C (it's now taking noticeable numbers of seconds to do a search).

And that's a week's worth. I'm not complaining - it's great that I have so many enhusiastic users who are busy adding photos, correcting coordinates, adding places and just generally picking nits off the program. But sometimes the net effect is that the your current taget (adding a new otions feature to limit POIs) is getting futher away, not closer!

Standalone CanalplanAC - what the heck is happening?

Chris Hunter asked the above question - in a more polite way - in the first ever comment to this blog. It was one I was reserving for when I wasn't feeling inspired to write about what I was doing, and hadn't posted for a while. But it wouldn't be fair to leave it unanswered until then.

In short, it's stalled at the alpha-test version: three selected people - experts at breaking Web CanalplanAC (WCAC from now on) were sent disks and asked to do their worst. They sent me lots of comments, and gave me lots to think about.

Then my only Windows machine went belly-up the way only a Windows machine can, and I've not got round (or, frankly, felt justified in chucking several hundred pounds of the family silver at Standalone CanalplanAC (SCAC) development - as that's the only thing I'd need it for) to replacing it. My spare time over the last year has been like Chapter 13 of a Jasper Fford novel.

That put an immediate hold on things. In the meantime, three other developments took place, all of which deserve - and will eventually get - articles of their own. The first of these was the addition of Google Ads to WCAC; this funded a broadband connection which in turn lead me into the wonderful world of Google Maps - the second new development. The third was my licensing of POI data to improve the database.

Now all of those provide arguments against me putting effort into SCAC. The first is a far nicer way to earn money from the site than selling the software, the second won't work without a broadband Internet connexion, and distributing the data behind the third would break my licensing conditions.

There are so many nice things about developing on-line software like WCAC, and so many horrible things about developing stand-alone software like SCAC. Here are the first that spring to mind:

• Release cycles. Stand-alone software needs to be nicely packaged, stable, without experimental features or egregious bugs (obligatory anti-Microsoft joke omitted). Web software can be the opposite: if you break it, never mind - you can fix it tomorrow. You can put a half-baked idea up and see what happens.
• Support. The web software is free. If it doesn't do what you want, then that's fine - you've lost nothing but a bit of time. If I sell software I'll feel a moral obligation to support people, send out patches for bugs etc (oa-Mjo).
  
• Peeking behind the scenes. You can do really cool things when the site is live and you own it. For example, every time the placefinder kicks in and suggests a match this is logged. At intervals I pour over the logs, then dry them out and tweak the auto-match code to make it actually reflect what people are doing.
• Consistency. You are developing for a single system. You have to deal with a collection of web browsers, but that's all you have to deal with. You're not dealing with the niceties of SP2, home v professional, Windows 95, 98, 2000, XP, Vista, whatever. None of which you have, and all of which you have near religious objections to paying money for.
• Independence. SCAC relies on the Abyss webserver. I've just heard that Aprelium have released a new version, and (surprise!) the patching of the configuration file that CanalplanAC's installation does fails miserably. I'ts not their fault - they don't know I'm relying on them, and even if I were, so what? But it just sums up the complexities of putting things on other machines with other software, rather than on your own server.
  

What it ulitmately boils down to is fun. This isn't what I do for a living and probably isn't what I want to. I get very little reward for it (and don't want too much - as long as I can justify all my income from the software on expenditure on hardware I can keep the taxman from taking his hunk out of it). Hacking together a new feature, throwing it out, watching it break, fixing it, getting nice comments from people (keep them coming - they really do inspire me to keep working) and getting "wouldn't it be nice if it did this) is a great way to spend your time. Working your way systematically down a list of known bugs and misfeatures, fixing them and not touching anything else interesting at the same time, regression testing the whole lot, etc, etc is the right way to do it if you're getting paid for it. But it's not much of a hobby.

I'm not saying I'm giving up the idea of ever releasing a stand-alone version. But I think it would need a major shift in something before I did. And a stand-alone would never be as featureful, data-rich, up-to-date or responsive as the web site.

More on POI options

I've now manually assigned all the existing POI sets to one of a small number of "groups", and have automatically given each set an ID (I do like having an editor in which I can write keyboard macros with incrementing numbers in them!). These are output during POI data creation and then run through a bit of code to collate and sort them by group. The IDs will be used to save your preferences, which means that if I move a set from one group to another, or rename it, your options settings will not break.

The next thing is to go from the collated sets to an HTML page.

What I'm doing - places of interest

One of the big efforts I'm making at the moment is to integrate all the POI data I've got into the database. What this involves is creating, and testing, rules that say what format the raw data files are in (in particular, some contain telephone information that can be parsed out using a regular expression) and how to convert them into entries in the place database. Having licensed the whole lot of data it seems silly not to use as much of it as possible.

But it's becoming obvious that for some areas it is starting to get a bit silly, and the map starts to vanish under the symbols. So I'm going to have to write an "option" selector that lets you chose which sorts of places you want showing (probably both at a "group" level - so you can exclude all historic sites for example, and at an individual level - so you can exclude a particular pub chain for example).

This won't be a particularly difficult piece of work, but it's fairly complicated, involving making changes to the data generation to automatically generate the configuration file - so that there is no reprogramming involved when new POI sets are added; writing code to generate the option file from the configuration file, and display it, and to load and save from a database somewhere; and to the gazetteer code to only show (both as maps and as text) places that you are interested in.

Should be a few week's work when fitted in along with everything else in life (a lot of next weekend will be lost to a beer festival!).

First thoughts

I've been in the habit of embedding chatty bits in various bits of the CanalplanAC website for some time now, but what with the news, the todo list, the FAQs etc, it all gets a bit messed up. Then there is the guestbook, discussions on uk.rec.waterways and so on.

Suddenly I realised that a blog would be a much better and easier way of doing things, so here it is. I'm going to use it to put up information on what I'm thinking of doing and working on; what's bugging me at the moment, notes of really nice things that people have done and so on. I've sent an informal target of a couple of posts a week minimum - let's see how I do.

It's also a great opportunity for you, by commenting on my posts, to influence the future development of CanalplanAC.