hackdiary

The Chumpologica - an RSS feed combiner

Today we relaunched The Daily Chump, a multi-author IRC-based blog, with a shiny new look and a new feature: the chumpologica. This is a webpage (and RSS 1.0 feed) that aggregates entries from blogs written by regular contributors to the Chump. It's built using about 150 lines of python and takes its configuration from a FOAF file.

Update: I've now packaged and released this project as a proper distribution.

FOAF has a predicate in its vocabulary for indicating that a person has a weblog. bloginfo.py uses the Redland RDF framework to read the FOAF file and find the details of each weblog. It uses the very latest version of Redland's python support (which contains support for iterators, allowing nice syntax such as: for statement in model: print statement.subject) which is only available in the nightly source snapshots at the time of writing.

chumpologica.py is the program start point. It uses Mark Pilgrim's RSS parser to read in the RSS feeds, and puts the entries into a listed sorted by date. The date is taken from the entry's Dublin Core date property, unless that's missing in which case the time of retrieval is used. A hash is kept of entries seen on previous runs so that we don't get duplicate entries. After a run, the entries and hashes are pickled (serialised) into a file for reuse on the next run.

If new entries were found, Mark Nottingham's RSS.py is invoked to output a new RSS 1.0 file using the newest 30 items. Weblog information taken from the FOAF file is stored along with each entry using standard Dublin Core tags such as relation to point to the original blog and creator to give the name of the writer. The resulting RSS file is sufficient to be transformed using XSLT into the final webpage.

Using OpenOffice Draw to author RDF

OpenOffice's vector-drawing app has a very nice Visio-like 'connector' tool that can link objects together with lines. It's very easy to put together labelled directed graph diagrams and have the lines re-flow as you move the nodes around. It also has a well-documented XML file format, which got me thinking that graph drawings could be converted into RDF automatically. I wrote some python code to read in OO files and print out n-triples.

A sample RDF graph drawn in OpenOffice looks like this:

(original file)

The output of the script run over this file is:

_:a11055796046 <http://example.com#pred2> <http://example.com#1> .
_:a11055796046 <http://example.com#pred1> "text" .
<http://example.com#1> <http://example.com#foo> <http://example.com#bar> .

Rectangles or ellipses with no text on them are taken to be bNodes. Those with text have URIs if they look like URIs (start with http:// or similar) and those with "quoted text" are literals (the quotes are discarded). Predicate URIs are taken from the text on the connectors.

The code uses the xpath processor from PyXML and is written for python 2.2. It's worth setting up the 'Default' style in OpenOffice Draw so that it puts arrows on the end of each connector, otherwise it's easy to get confused about which way round the connectors are.

www2003 developers day talk

I gave a developer-oriented version of my semantic web "photos and metadata" talk at www2003 developers day. The slides are available online and there are a bunch of demos to look at.

The demos:

• Wordnet namespace entry for building
• Picdiary home page
• RSS for Picdiary home page
• Picdiary polaroids view
• Pictures of Edd Dumbill
• Pictures of Edd Dumbill, RSS source
• Wordnet-based search for skyscraper
• Wordnet-based search for building
• Wordnet-hierarchy-nav for structure

The logs on the IRC #www2003 channel during the talk are interesting to compare with the slides.