Although source for an earlier version was previously available, this new version 1.0 includes all the improvements made by myself and Dave Beckett for Planet RDF. I’ve also refactored the code to use an external configuration file, and packaged it up using python distutils with a simple control script. This means that a single global installation of the system can be used to run multiple clogica sites on a single machine. It’ll also make it easy to build distribution packages for Redhat and Debian.

Update: I’ve now packaged and released this project as a proper distribution.

FOAF has a predicate in its vocabulary for indicating that a person has a weblog. bloginfo.py uses the Redland RDF framework to read the FOAF file and find the details of each weblog. It uses the very latest version of Redland’s python support (which contains support for iterators, allowing nice syntax such as: for statement in model: print statement.subject) which is only available in the nightly source snapshots at the time of writing.

chumpologica.py is the program start point. It uses Mark Pilgrim’s RSS parser to read in the RSS feeds, and puts the entries into a listed sorted by date. The date is taken from the entry’s Dublin Core date property, unless that’s missing in which case the time of retrieval is used. A hash is kept of entries seen on previous runs so that we don’t get duplicate entries. After a run, the entries and hashes are pickled (serialised) into a file for reuse on the next run.

If new entries were found, Mark Nottingham’s RSS.py is invoked to output a new RSS 1.0 file using the newest 30 items. Weblog information taken from the FOAF file is stored along with each entry using standard Dublin Core tags such as relation to point to the original blog and creator to give the name of the writer. The resulting RSS file is sufficient to be transformed using XSLT into the final webpage.

• convert foaf:mbox entries to privacy-protected foaf:mbox_sha1sum entries.
• normalise statements of the form “PICTURE depicts PERSON” to “PERSON depiction PICTURE”.
• smush disparate references to the same person into references to a single definition of that person.
• extract depiction triples from a model and copy just the bare minimum of information related to those depictions

So I wrote foaftool, a Java class that uses Jena. The tarball also contains a couple of servlets that can be used to transform existing content on the web.

The first servlet will transform foaf:mbox triples in FOAF data into appropriately-encoded foaf:mbox_sha1sum triples. This makes Edd’s FOAF file look like this. Using an extra querystring parameter, it optionally converts foaf:depicts triples to foaf:depiction. foaf:depicts isn’t actually in the official FOAF schema at the time of writing, although it is in informal use in many places as it sometimes makes for more elegant modeling. Normalising to foaf:depictions makes working with large amounts of FOAF data simpler.

Writing the smushing code was an entertaining diversion. Smushing is important when merging multiple RDF sources. Say you have two sources, edd1.rdf and edd2.rdf, showing where to find photos of Edd. When merged, the graph structure looks like this:

This is because without smushing, the anonymous nodes that both have Edd’s email address are not equated. The smushed version (smushed on mbox_sha1sum using a foaftool servlet) looks like this:

With the data in normalised and merged form, I want to extract just the triples of the form “X foaf:depiction [picture uri]” and the related foaf:name and foaf:mbox_sha1sum triples. With the foaftool code, I can now merge and extract depictions from any number of RDF sources.

Comments and bugfixes are very welcome; the code has only been tested as far as the junit tests included in the tarball.

Compare the HTML version to the SVG version. There’s nothing in the SVG version that couldn’t be rendered in HTML; the interest was in doing the job entirely client-side.

The system builds links into foafnaut and my wordnet picture hierarchy when it finds appropriate metadata about a picture.

Jim’s parser page says the RDF parser needs at least version 5.5 to run under IE. Javascript in SVG is a pain to debug in mozilla; you don’t seem to be able to use mozilla’s javascript console or debugger.

Subject: Re: [RSS-DEV] RSS and Libraries
From: mb@p…
Date: Tue Dec 17, 2002 9:18 pm

On Mon, Dec 16, 2002 at 05:14:33PM +0000, Libby Miller wrote:
> On Mon, 16 Dec 2002, se teague wrote:
> > I’m so happy to find this group! I’ve been developing an
> > RSS/Perl/PHP/MySQL based rss program to generate news and events for the
> > academic library I work for. i do have a question though; We haven’t
> > tackled associating images with RSS items rather than channels. Has
> > anyone done this before? Any best practices out there?
>
> I rather like what Matt Biddulph’s been doing, e.g.:
> http://www.picdiary.com/rss/barcelona_conf.rss

Thanks Libby. To (over)elaborate:

My site (http://www.picdiary.com) is just a blog of collections of
pictures. There’s not much text, the pictures are the main focus and
just have a few lines of description. For a while it’s been lacking good
search facilities, and was built out of static HTML.

I decided to use RSS 1.0 as the ‘container’ format for each picture
collection. RSS works just as well for pictures-as-items as it does for
html-pages-as-items – they’re all just web links. So I wrote a bunch of
simple perl scripts to read in RSS and display them as web pages. For
example, The link Libby gave above can be viewed at

or of course it can be viewed in a newsreader/aggregator.

If you look in the HTML source of that page, you’ll see a line like
this:

<link rel=”alternate” type=”application/rss+xml” title=”RSS”
href=”http://www.picdiary.com/rss/barcelona_conf.rss”>

This is following the rss autodiscovery standard so that spiders can
find the RSS metadata from the webpage.

Using RSS 1.0′s RDF capability, I can add all sorts of metadata to
channels and items with a few extra tags in the RSS. That particular
feed uses FOAF to say that certain pictures are of people like Libby.
Other (more recent) feeds such as
http://www.picdiary.com/rss/highwalk.rss also use Dublin Core to give
the creator and date of the channel, and Wordnet to list keywords
associated with each picture (item).

There’s a (prerelease) search engine that indexes all the RSS and uses
the Wordnet terms (and exploits Wordnet’s synonym and hypernym
relationships between words to give better search results) at
http://www.picdiary.com/cgi-bin/search.pl?word=building
If you look ‘behind the scenes’ of that search engine at
http://www.picdiary.com:8180/rss/search?word=building
then you’ll see that the search results themselves are returned as
dynamically-built RSS channels then rendered into HTML by some perl. So
you could subscribe to the search results if you wanted to.

Ramble ramble. There’s quite a lot to say, feel free to followup or mail
me off-list for more information.

Cheers,
Matt.