Archives: September 2008

International 419-SMS

I don’t know if I get more spam than other people because my phone numbers are public, or if I’m just more of a touchy, grouchy git than other people and just react to every spam I get.

So, here is another SPAM-related post.

I just got an SMS from +256715316646 saying:

CONGRATULATION- You have won a TOYOTA LANDCRUISER VX worth $80,000 US DOLLARS.Contact the manager DR: STEPHEN through +254720043297 (JAPAN INT. MOTORS) THANKS.

Look closely. Firstly, it’s a 419-scam via SMS. I’ve never seen one of those before. 2nd, it’s from Uganda (a Uganda Telecom cellphone number). Third, it wants me to call someone in Kenya (a Safaricom cell number).

What on earth does one do to stop such things? Either this is run by a group with people on the ground in both countries, or they are using online bulk-SMS services. But International SMSs aren’t cheap. And I have no idea who to complain to about this. Let’s hope that this kind of spam doesn’t become common, because it’s virtually unstoppable if it does (like International e-mail spam).

Update: This page seems to have become a rant-board for people getting this spam. If it isn’t plainly obvious, here’s what to do: nothing. Don’t respond, don’t give them details, it’s a scam!.

Update 2: I won’t be replying to any of these comments any more - read the update above.

Update 3: Comments disabled, as this post is collecting more of the same comments. Don’t try and contact me personally if you have got a similar message, I’m not interested.

Ctrl-Alt-shortcuts considered harmful

I’ve written about this before, but Ctrl-Alt-workspace switching key-presses nail me routinely.

Let’s go through some history:

We have Ctrl-Alt-Delete, the “three-fingered-salute”, meaning reboot, right? That combination was designed to NEVER be pressed by accident. And it never used to be.

The X guys needed a kill-X key-press, as things can sometimes get broken in X. So they chose Ctrl-Alt-Backspace, which is also a pretty sensible combination. It’s very similar to Ctrl-Alt-Delete, so we remember it, and backspace has milder connotations than delete, so we understand it to mean that it’ll only kill a part of the system.

X also has some other Ctl-Alt- shortcuts. Some of these are also suitably obscure, i.e. NumPad+ and NumPad-. Others like Ctrl-Alt-F1 mean change to virtual console 1. That one might do by accident, if you are an old WordPerfect user, but should be safe enough otherwise. They were designed to look like big brothers of (and even work as) standard VT-changing behaviour.

For changing workspaces, Alt-F1 style key-presses were used, mimicing VT-changing key-presses. This is great for *nix users, but people coming from Windows expect Alt-F4 to close a program, not take them to workspace 4.

So GNOME came along, and decided that instead, they’d use Ctrl-Alt-Arrow key-presses to change workspace. That’s fine, but it’s a pretty common action, so I’m often holding down Ctrl-Alt without even thinking about it. If I start editing something and press delete/backspace, before I’ve released Ctrl-Alt, boom! And I run screaming and write a blog post.

Now, I know that Ctl-Alt-{Delete,Backspace} can be disabled (even if the latter is a little tricky to do), but I’d really like to change them. I like to be able to kill X without using another machine and ssh, I just don’t like this to happen by accident. And no, the solution isn’t for me to change my workspace-changing keys, because this problem must affect every GNOME user, not just me.

Dangerous key-presses should be really unlikely key-presses. Alt-SysRq- key-presses are good in this regard, they’ll always be unlikely. (Oh, and they are insanely useful.)

Political Compass

I’m not an Internet-meme person, but #clug has been rather into the Political Compass Tests recently (Thanks to Michael Gorven’s graphing of our scores).

Those of us in the sensible quadrant of the graph are rather worried about the distance that some people are from us. I mean, they must be total nutters :-)

To help them understand the incorrectness of their ways, Jonathan suggested that we write up our choices and reasoning. I haven’t read anyone else’s reasoning yet, but here is my reasoning for each choice in the test. It might well be different to the last time I did it, but that’s probably caused by me thinking about my choices rather than the embarrassment of publishing them. (I’m right, remember). On #clug, we acknowledge that peoples’ views change and keep a history of past scores, although the graph doesn’t display that (nudge nudge Michael).

If you haven’t done this test, and are interested, maybe take it before you read any further. It’ll take you less than half an hour, and promises a little food for thought.

Page 1: Just a few propositions to start with, concerning - no less - how you see the country and the world.

If economic globalisation is inevitable, it should primarily serve humanity rather than the interests of trans-national corporations.
Agree - Yes it should, but if globalisation is inevitable, how can we make it serve specific goals. It’ll do what it wants to do. I go with agree on principle, but with the understanding that there isn’t much we can do to make this a reality.
I’d always support my country, whether it was right or wrong.
Strongly Disagree - No way. While we all have a bit of national pride I make my own decisions. I’d seriously consider emigrating if our government were to go mad and started invading random nations (no Lesotho doesn’t count).
No one chooses his or her country of birth, so it’s foolish to be proud of it.
Agree - Yes, you’re born where you are born, and you are welcome to national pride. The values we hold that lead to such a pride are in a large part determined by our up-bringing, we can’t self-bootstrap. I’m an Italian citizen, but I’ve never spent more than a month in the country (and on that visit, I was barred from leaving the country because they wanted me for national service).
Our race has many superior qualities, compared with other races.
Strongly Disagree - What? Let’s leave that one there.
The enemy of my enemy is my friend.
Disagree - I don’t pick enemies easily, so the enemy of my enemy is unlikely to be an enemy. I won’t align myself with someone unless our disagreements have common ground.
Military action that defies international law is sometimes justified.
Strongly Disagree - It might be justified that doesn’t mean I’ll agree with it. It’s still illegal. If there is cause for such action, International Law should probably be amended.
There is now a worrying fusion of information and entertainment.
Agree - Not a well phrased question. Information and entertainment go hand in hand well, we all know how boring bland writing is. But there is a worrying trend in current media toward sensationalism — this can readily be considered entertainment.

Page 2: Now, the economy. We’re talking attitudes here, not the FTSE index.

People are ultimately divided more by class than by nationality.
Strongly Agree - Nationality doesn’t matter that much these days. Class still appears to.
Controlling inflation is more important than controlling unemployment.
Agree - I stand under correction here, I’m not an economist. I’d imagine that they both need to be kept under control, but that letting inflation get out of control will quickly lead to bigger unemployment problems.
Because corporations cannot be trusted to voluntarily protect the environment, they require regulation.
Strongly Agree - Most large corporations care a lot more about their shareholders returns than the environment. And many of the worst environmental disasters can be pinned on corporations chasing profit.
“from each according to his ability, to each according to his need” is a fundamentally good idea.
Disagree - It’s a nice thought, but actually getting this out of people is non-trivial. It requires more trust than free market does.
It’s a sad reflection on our society that something as basic as drinking water is now a bottled, branded consumer product.
Agree - I never have a water bottle on me when I need it, and drinking fountains are way too scarce. Dare I even question how one gets brand loyalty in that market?
Land shouldn’t be a commodity to be bought and sold.
Strongly Disagree - Well how should it be worked then? Considering the current housing market, land is really important to people.
It is regrettable that many personal fortunes are made by people who simply manipulate money and contribute nothing to their society.
Agree - Regrettable, but inevitable. Whatever works for them. Fortunately they can do that and still be interesting people.
Protectionism is sometimes necessary in trade.
Strongly Agree - I think history shows that. “Sometimes” is a useful qualifier there :-)
The only social responsibility of a company should be to deliver a profit to its shareholders.
Strongly Disagree - How is that a social responsibility? That’s a financial responsibility.
The rich are too highly taxed.
Disagree - Again, IANAE. The rich are pretty good at dodging tax anyway, but high tax on the rich is bad for the economy.
Those with the ability to pay should have the right to higher standards of medical care.
Strongly Agree - Why not? Worst case scenario, it can help subsidise health-care and inject money into the system.
Governments should penalise businesses that mislead the public.
Strongly Agree - People trust large companies. Sad but true.
A genuine free market requires restrictions on the ability of predator multinationals to create monopolies.
Strongly Agree - Monopolies don’t help anyone but themselves (and that only in the short-term). They certainly don’t lead to a free-er market.
The freer the market, the freer the people.
Disagree - To a small extent, yes. highly controlled markets are don’t give their citizens much choice in business, beyond that, I disagree.

Page 3: Now a look at some of your personal social values …

Abortion, when the woman’s life is not threatened, should always be illegal.
Strongly Disagree- No need to compound already-painful mistakes.
All authority should be questioned.
Strongly Agree - We had a big argument about this one at CLUG dinner this week. I think Jeremy (who was arguing against me) was interpreting this question incorrectly.
An eye for an eye and a tooth for a tooth.
Strongly Disagree - How does that help anyone?
Taxpayers should not be expected to prop up any theatres or museums that cannot survive on a commercial basis.
Strongly Disagree - Many things that are supported by the treasury aren’t commercially viable. Of course some unprofitable theatres and museums aren’t worth propping-up, but we trust that such decisions can be made.
Schools should not make classroom attendance compulsory.
Strongly Disagree - Children cannot be expected to make such a decision for themselves, and there are some screwed-up parents out there who shouldn’t be given such a powerful way to disadvantage their children.
All people have their rights, but it is better for all of us that different sorts of people should keep to their own kind.
Strongly Disagree - If, as a nation, we give people rights then we have to respect those rights. Otherwise emigrate.
Good parents sometimes have to spank their children.
Agree - IANAP. I think this is true, although it can be kept down to a very small value of sometimes.
It’s natural for children to keep some secrets from their parents.
Strongly Agree - It’s a nasty world out there, and people have to learn how to protect themselves. Also, this goes both ways, parents keep secrets from their children. Quite simply I think it would be unhealthy not to agree with this statement.
Possessing marijuana for personal use should not be a criminal offence.
Agree - Illegalising drugs doesn’t help anything, although like many legal things marijuana can be harmful.
The prime function of schooling should be to equip the future generation to find jobs.
Disagree - Call me an idealist, but I’d like to think of schooling as more than that. It should teach you how to be a human (by our definition), and should give you the tools to discover and reach for ambitions. These may lead to jobs.
People with serious inheritable disabilities should not be allowed to reproduce.
Strongly Disagree - I don’t think the argument for this is remotely compelling enough to impose such a restriction.
The most important thing for children to learn is to accept discipline.
Strongly Disagree - What an awful summary of childhood. Anyway, I don’t know if I did learn that.
There are no savage and civilised peoples; there are only different cultures.
Agree - I can only agree with this, not strongly, as our civilisation rests on a certain amount of “civilisation” in the population. So we have to draw lines somewhere. We call the other side of that line “savages”.
Those who are able to work, and refuse the opportunity, should not expect society’s support.
Agree - Depends on what they are doing (persuits of knowledge have to be exempted), but our society does require that, yes. Personally, when I’m not working on something I feel unhappy so I’m biased towards working.
When you are troubled, it’s better not to think about it, but to keep busy with more cheerful things.
Agree - Very loaded question, entirely depends on what the trouble is. For me, it’s often procrastination which requires work to push through it.
First-generation immigrants can never be fully integrated within their new country.
Disagree - This is quite possibly true for many migrants, but there are many counter-examples. Many countries are very similar, and migration doesn’t necessarily mean that much change.
What’s good for the most successful corporations is always, ultimately, good for all of us.
Strongly Disagree - Corporations are successful in their own niches, such things are usually heavily biased towards the situation and against the general good.
No broadcasting institution, however independent its content, should receive public funding.
Strongly Disagree - The achievements of the BBC (for all its foibles) are a great counter-example here.

Page 4: … and how you see the wider society.

Our civil liberties are being excessively curbed in the name of counter-terrorism.
Strongly Agree - And it doesn’t seem to be helping, either. Fortunately this hasn’t reared its head too much in South Africa yet.
A significant advantage of a one-party state is that it avoids all the arguments that delay progress in a democratic political system.
Disagree - We’ve seen in South Africa that a virtually one-party state can still have such arguments, they don’t delay progress, and the arguers quickly get branded as whiners. (Btw, I support the DA)
Although the electronic age makes official surveillance easier, only wrongdoers need to be worried.
Strongly Disagree - History shows this to be incorrect. Such powers get abused.
The death penalty should be an option for the most serious crimes.
Strongly Disagree - No matter how bad the prison system is, lets assume that everyone has the possibility of rehabilitation, unless there is something very psychologically wrong with them (in which case they probably wouldn’t and shouldn’t be eligible anyway).
In a civilised society, one must always have people above to be obeyed and people below to be commanded.
Disagree - The people who are to be obeyed must be subject to the same law.
Abstract art that doesn’t represent anything shouldn’t be considered art at all.
Disagree - “But is it art?” Why should art involve representations. Music doesn’t. There comes a point when art stops, but we all draw that line in different places.
In criminal justice, punishment should be more important than rehabilitation.
Strongly Disagree - People are remarkably resilient to punishment, and no it doesn’t seem to help, only the threat of it helps.
It is a waste of time to try to rehabilitate some criminals.
Disagree - It probably is, but can you pick those criminals out for me?
The businessperson and the manufacturer are more important than the writer and the artist.
Strongly Disagree - Not in my books.
Mothers may have careers, but their first duty is to be homemakers.
Strongly Disagree - I’m happy with a father being a home-maker. And I’m happy with working parents, although obviously they shouldn’t neglect their children.
Multinational companies are unethically exploiting the plant genetic resources of developing countries.
Agree - Yes some are, although this is a very broad statement.
Making peace with the establishment is an important aspect of maturity.
Agree - I don’t know if it’s important so much as inevitable. Certain parts of the establishment are welcome to be overthrown in my books.

Page 5: If you got through that okay, you’ll find these propositions on religion a breeze.

Astrology accurately explains many things.
Strongly Disagree - Not sufficiently for me.
You cannot be moral without being religious.
Strongly Disagree - This one probably depends on definitions.
Charity is better than social security as a means of helping the genuinely disadvantaged.
Disagree - I don’t really know what to say here, but I can’t support that statement.
Some people are naturally unlucky.
Agree - Some people have really bad things happen to them. It sucks. That doesn’t mean that they attract bad luck.
It is important that my child’s school instills religious values.
Strongly Disagree - No thanks.

Page 6: Finally, a look at sex.

Sex outside marriage is usually immoral.
Strongly Disagree - Definitions strike again.
A same sex couple in a stable, loving relationship, should not be excluded from the possibility of child adoption.
Strongly Agree - Why not? There are many things that can be bad for a child’s childhood, the lack of a specific gender in a parental role is common already. What other arguments are there against this?
Pornography, depicting consenting adults, should be legal for the adult population.
Strongly Agree - I’ve got nothing against this.
What goes on in a private bedroom between consenting adults is no business of the state.
Strongly Agree - I don’t think there is one set of morals that fits all.
No one can feel naturally homosexual.
Strongly Disagree - I wouldn’t know, I’m under guidance on this one.
These days openness about sex has gone too far.
Disagree - In many areas it probably hasn’t gone far enough.

The end: My current political compass:

Economic Left/Right: -4.62
Social Libertarian/Authoritarian: -6.26

My Political Compass Graph

I’ve moved fractionally up since I last took the test (-4.62, -6.41) that’s it. Now I can go and read other peoples’ justifications for their choices.

Split-Routing on Debian/Ubuntu

My post on split-routing on OpenWRT has been incredibly popular, and led to many people implementing split-routing, whether or not they had OpenWRT. While it's fun to have an exercise as a reader, it led to me having to help lots of newbies through porting that setup to a Debian / Ubuntu environment. To save myself some time, here's how I do it on Debian:

Background, especially for non-South Africa readers: Bandwidth in South Africa is ridiculously expensive, especially International bandwidth. The point of this exercise is that we can buy "local-only" DSL accounts which only connect to South African networks. E.g. I have an account that gives me 30GB of local traffic / month, for the same cost as 2.5GB of International traffic account. Normally you'd change your username and password on your router to switch account when you wanted to do something like an Debian apt-upgrade, but that's irritating. There's no reason why you can't have a Linux-based router concurrently connected to both accounts via the same ADSL line.

Firstly, we have a DSL modem. Doesn't matter what it is, it just has to support bridged mode. If it won't work without a DSL account, you can use the Telkom guest account. My recommendation for a modem is to buy a Telkom-branded Billion modem (because Telkom sells everything with really big chunky, well-surge-protected power supplies).

For the sake of this example, we have the modem (IP plugged into eth0 on our server, which is running Debian or Ubuntu, doesn't really matter much - personal preference. The modem has DHCP turned off, and we have our PCs on the same ethernet segment as the modem. Obviously this is all trivial to change.

You need these packages installed:

# aptitude install iproute pppoe wget awk findutils

You need ppp interfaces for your providers. I created /etc/ppp/peers/intl-dsl:

unit 1
pty "/usr/sbin/pppoe -I eth0 -T 80 -m 1452"
lcp-echo-interval 20
lcp-echo-failure 3
maxfail 0
mtu 1492


unit 2
pty "/usr/sbin/pppoe -I eth0 -T 80 -m 1452"
lcp-echo-interval 20
lcp-echo-failure 3
connect /bin/true
maxfail 0
mtu 1492

unit 1 makes a connection always bind to "ppp1". Everything else is pretty standard. Note that only the international connection forces a default route.

To /etc/ppp/pap-secrets I added my username and password combinations:

# User                     Host Password  *    s3cr3t *    passw0rd

You need custom iproute2 routing tables for each interface, for the source routing. This will ensure that incoming connections get responded to out of the correct interface. As your provider only lets you send packets from your assigned IP address, you can't send packets with the international address out of the local interface. We get around that with multiple routing tables. Add these lines to /etc/iproute2/rt_tables:

1       local-dsl
2       intl-dsl

Now for some magic. I create /etc/ppp/ip-up.d/20routing to set up routes when a connection comes up:

#!/bin/sh -e

case "$PPP_IFACE" in
   exit 0

# Custom routes
if [ -f "/etc/network/routes-$IFACE" ]; then
  cat "/etc/network/routes-$IFACE" | while read route; do
    ip route add "$route" dev "$PPP_IFACE"

# Clean out old rules
ip rule list | grep "lookup $IFACE" | cut -d: -f2 | xargs -L 1 -I xx sh -c "ip rule del xx"

# Source Routing
ip route add "$PPP_REMOTE" dev "$PPP_IFACE" src "$address" table "$IFACE"
ip route add default via "$PPP_REMOTE" table "$IFACE"
ip rule add from "$PPP_LOCAL" table "$IFACE"

# Make sure this interface is present in all the custom routing tables:
route=`ip route show dev "$PPP_IFACE" | awk '/scope link  src/ {print $1}'`
awk '/^[0-9]/ {if ($1 > 0 && $1 < 250) print $2}' /etc/iproute2/rt_tables | while read table; do
  ip route add "$route" dev "$PPP_IFACE" table "$table"

That script loads routes from /etc/network/routes-intl-dsl and /etc/network/routes-local-dsl. It also sets up source routing so that incoming connections work as expected.

Now, we need those route files to exist and contain something useful. Create the script /etc/cron.daily/za-routes (and make it executable):

#!/bin/sh -e

wget -q -O /tmp/zaroutes
size=`stat -c '%s' /tmp/zaroutes`

if [ $size -gt 0 ]; then
  mv /tmp/zaroutes "$ROUTEFILE"

It downloads the routes file from cocooncrash's site (he gets them from, aggregates them, and publishes every 6 hours). Run it now to seed that file.

Now some International-only routes. I use IS local DSL, so SAIX DNS queries should go through the SAIX connection even though the servers are local to ZA.

My /etc/network/routes-intl-dsl contains SAIX DNS servers and proxies:

Now we can tell /etc/network/interfaces about our connections so that they can get brought up automatically on bootup:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0
iface eth0 inet static

auto local-dsl
iface local-dsl inet ppp
        provider local-dsl

auto intl-dsl
iface intl-dsl inet ppp
        provider intl-dsl

For DNS, I use dnsmasq, hardcoded to point to IS & SAIX upstreams. My machine's /etc/resolv.conf just points to this dnsmasq.

So something like /etc/resolv.conf:



# IS:

If you haven't already, you'll need to turn on ip_forward. Add the following to /etc/sysctl.conf and then run sudo sysctl -p:


Finally, you'll need masquerading set up in your firewall. Here is a trivial example firewall, put it in /etc/network/if-up.d/firewall and make it executable. You should probably change it to suit your needs or use something else, but this should work:

if [ $IFACE != "eth0" ]; then

iptables -F INPUT
iptables -F FORWARD
iptables -t nat -F POSTROUTING
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -i eth0 -s -j ACCEPT
iptables -A INPUT -i ppp+ -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -j DROP
iptables -A FORWARD -i ppp+ -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i eth0 -o ppp+ -j ACCEPT
iptables -A FORWARD -j DROP
iptables -t nat -A POSTROUTING -s -o ppp+ -j MASQUERADE

Drupal Hacking

I apologise for my last post on this topic, it probably wasn't very interesting :-)

I've done the Drupal 6 upgrade, and it was relatively painless. Most modules ported smoothly, a few required me to learn how to port modules to Drupal 6, and one I just gave up on.

On the whole, the porting is simple, has a pretty good howto on the topic. A few APIs have changed, and that's about it. A great tool to help with this is the coder module, which knows about the API changes, as well as Drupal's coding standards.

I've added the GeSHi module for code syntax highlighting (apologies for the planet-spam caused by this), and I've moved from marksmarty to markdown + typogrify (which I had to port to Drupal 6). I'm not too happy with the geshi colour-scheme and indenting, but it does a good enough job. I should write a "command prompt" mode for it, but that can wait for now...

Akismet is currently totally broken for Drupal 6, even if it's labelled as being in beta. I got about half way through porting it before giving up and switching to mollom, which looks like a pretty good replacement (and it takes care of the sign-up form too).

Finally, the subject of input-filters. Drupal lets you define a "default filter", but that filter has to be available for everyone, even comments. So your default filter has to protect against XSS. I'd much prefer it if commenters used a simple, locked-down input-format, and I used a nice markdown format.

I'm not the only one to notice this, and it seems like it'll be fixed in Drupal 7. Until then, I'm using remember-filter which remembers that I use markdown, and all the commenters use the default, locked-down filter. (Again, ported.)

On private RSS feeds

For those of you who are wondering what my recent Google Reader shared item comment was all about, here you go. I’ve explained it over IM twice, and I think it deserves a proper blog post:

We all like having RSS-feeds for everything, right? That way we can catch up with the world in one place.

So Facebook have RSS feeds for friends’ status updates, notes, and shared posts. These feeds look something like And all the feeds have the same key.

Yes, we’d rather they used HTTP-Digest password authentication, but not many RSS readers support that, and you’d never give anyone that feed url, right?

Well, no. If I read something cool in one of these Facebook-feeds in Google Reader and I share it with my Google Reader friends, they’ll all get the full feed URL. Now they can read all my friends’ status updates, notes, and shared items.

One of my Facebook friends might be paranoid, and writing about very personal stuff on Facebook. As a Facebook user, he could have set his privacy settings so that only his friends can read his notes. However, now all my Google Reader friends can too.

In this case, this isn’t a big problem, because there’s very little interesting content on Facebook, and hopefully no trade secrets. Obviously these problems apply to services besides Facebook and Google Reader, but these are good examples. Also a friend of mine shared his key recently ;-)

But it gets worse, Google Reader has a feed directory and feed discover page. Searching it reveals lots of such ID, key combinations. And generally Googling reveals 30-odd such pairs that have leaked onto the general Internet.

So. If you are implementing RSS feeds with private data in them, please don’t use an in-URL key. Rather submit patches to all your favourite feed-readers adding support for HTTP-authentication (and in the case of Google Reader, maybe don’t use it for private feeds).

Drupal 6

I’ve been playing with Drupal 6 while helping my parents set up a website for their choir. I’m impressed, it just keeps getting better. I’ll be upgrading this site in the next day or two.

I had to patch a few modules for Drupal 6 support, but it’s really easy to do. I only waited this long because most of the modules I used took a while to get Drupal 6 support, but in retrospect, I needn’t have.

I host a few websites for various people and causes using Drupal, as described here. Now I’m feeling the urge to work on Drupal stuff again, and hope to make some big improvements to this site soon. I’m thinking Activity Stream type stuff for a start (thanks Vhata).

In other news, I have been helping a house-mate set up a website for his magazine in WordPress. I’m amazed how much PHP you need to mangle to get wordpress to do what you want. Watching someone who has no programming experience at all do this stuff can be both entertaining and depressing. What a terrible introduction to programming… The WordPress API scares me, it uses URL-encoded parameters to many functions for a start. And php isn’t exactly a well-designed language.

Well, I suppose I learned to program in BASIC 2.0 - everyone has to start somewhere…

Spelunking in M&G's Zapiro Archive

Those who follow me will know that I used to maintain a web frontend to the Mail & Guardian Online Zapiro archive.

M&G used to have a rather crufty website. Subscriber-only content was trivial to access (for non-subscribers), URLs were ugly, and dinosaurs roamed in the far corners of the site. It had RSS feeds, but not an RSS feed for the zapiro archive (or any specific-interest RSS feeds for that matter). I don’t check websites, I read RSS feeds.

Me being a young geek with a little too much spare time, I put together, as a ~200-line PHP script (with no SQL DB) that was really nice to use (in my books) and gave me a Zapiro RSS feed.

When they noticed, the powers at be at M&G weren’t too impressed with it, because it deprived them of eyeballs (and hot-linked their Zapiro images). However I felt satisfied that I was merely providing a fair-use access to their content and allowing people to follow it who wouldn’t have been able to otherwise. The site never got much traffic, so thus far it’s not been a serious problem.

Around June this year, M&G redesigned their website, and I don’t think I even noticed (did I say something about them not having decent feeds?). This redesign broke the machinery in but I didn’t notice that because Zapiro had taken a sabbatical earlier this year, and was going weeks without posting cartoons.

Enough back-story. Point is I took a look at the new M&G Zapiro Archive this evening and was shocked. Before I go into all my problems with it, let me just disclaim that they are rather nit-picky but if these problems weren’t there they site would be a hell of a lot more usable:

  • There are still no useful RSS feeds. There is a rather terse selection of general feeds.
  • The Archive menu only goes back to 2001. M&G has zapiro cartoons going back to 1999.
  • Archive menu URLs are in /Month/Year format. Did anyone even think about URL-scheme when they were designing?
  • Their tagging feature while using multi-select widgets only allows single tags to be selected (oh, and it requires Javascript)
  • Each cartoon has two URLs. Ok, I guess they weren’t thinking about URL scheme.
  1. Today’s cartoon has the /zapiro/all/ URL. Yesterdays /zapiro/all/1, etc. going back to the begging of time (currently residing at /zapiro/all/1870). Way to go with permalinks guys. Oh and did you notice that they are all titled “Latest Zapiro”?
  2. Clicking on the “Comments” link or using the “Archive” menu below takes you to something like /zapiro/fullcartoon/1. Oh, except 1 gives us a non-existent cartoon at the beginning of this Unix Epoch. But take a closer look: it has tags associated. Can anyone say WTF?.

The insanity continues: 2 gives us a cartoon from September 1999. 3-25 are more non-existent wonders, and then things go backwards in time until 36 which jumps us to June 3 2008. (Hmm, I think that may have been around the M&G redesign launch date.)

We move forward in time until 40, when we start moving backwards from May 2008, through many seas of well-tagged gaps, to … well somewhere. (OK, so I got bored and didn’t manually crawl 2000 pages, but would you?) Some cartoons are in totally the wrong position, we randomly move backwards and forwards and sideways.

Finally things settle down, and we go forwards again (with gaps of course) from 2054 to today’s cartoon at 2101 — a fine Zapiro specimen if every I saw one.

Why was I doing all this mind-numbing crawling you ask? Well I wanted to know if I could do anything to make my Zapiro scraper work again. The answer? Not simply. They don’t have any sensible way to locate the cartoon from a specific day, short of crawling the entire archive and recording the URLs found. I don’t think there is any logic to this LSD-induced URL scheme.

URL schemes matter. This seems to be something that the big guns haven’t noticed. I don’t think it’s a co-incidence that the most expensive CMSs out there have the worst URLs, whereas Wordpress and Drupal (with pathauto) encourage sensible URLs and are Open Source.

Sure, most users don’t change what they see in the address bar, but if people are going to link into your site, you should provide nice permalinks. Then, if you want anyone to build anything on top of your site (where anyone includes yourself), it would really help if you had a sane URL scheme. Finally, it gives you geek-cred. :-)

While I think of a better way to get my scraper working again, Happy Spelunking!

September GeekDinner

Looks like the September Geekdinner list is filling up nicely. To anyone on the waiting list, keep an eye on that wiki right up to the last moment: we Capetonians are notorious for dropping out at the last minute, especially if the weather is bad. I’d expect a reasonable number of drop-outs - we thought the last dinner was going to be overflowing, and there was still space at the end.

I’ve just done a round of updates on Planet GeekDinner and I’m glad to see a good sprinkling of new faces (or is that geeks with new websites?). If you’d like your GeekDinner related posts to be syndicated on the planet and I’ve missed your blog or got the wrong website please let me know.