Monthly Archives: December 2008

Let it Snow

It’s grey and wet outside, so let’s have some snow to brighten the place up a little. The snow is only on the front page, so if it annoys you, just click on a link to another page, or click on the “Stop Snowing” link over to the left. Or just click here

Snow script courtesy of Scott Schiller

Set Manager Updates

In the three years since I created it, my Flickr Set Manager hasĀ  turned out to be amazingly popular, with one of the most-liked features being the automatic updating of sets. Of course, the problem with being popular, is that there are then an awful lot of sets to update. And I do mean an awful lot.

I am sometimes asked if sets can be automatically updated more often than they currently are – perhaps every hour, or if I can arrange for them to be updated at a specific time of day. The answer is very simple – no, that’s just not really possible. The script that keeps the sets updated runs continuously, and just about manages to keep up with the amount of work it currently has to do.

There are a couple of things that need to be considered here. The first is the processing power required to keep these sets up to date. Yesterday, I moved over to a new server that has plenty of spare capacity – that should help keep things running smoothly for the forseeable future. The second thing that needs consideration is the Set Manager’s use of the Flickr API.

For some time now, I have been stretching the limits of my Flickr API key somewhat – Flickr allow free non-commercial use of the API, but they do ask you to limit calls to the API to an acceptable rate. As the popularity of the set manager has increased over time, so has its usage of the Flickr API, and the time has come to restructure things a little.

I have spent some time recently rewriting large chunks of the Set Manager, and the first set of changes were rolled out yesterday as part of the site move. This particular batch of changes are all about making the application more Flickr-friendly.

  • The set generation code has been extensively rewritten and is now much more efficient – particularly for random sets.
  • There are quite a few sets in the system that are still marked as “automatically regenerate” but are actually now redundant (for example, the searches used to construct them no longer ever return any matching photos). With immediate effect, automatically generated sets that continually fail to find matching photos will have automatic regeneration disabled.
  • The rebuilding of sets is now throttled at a much lower level than previously. This means that it may now be longer than a single day before sets are refreshed. I will be monitoring this interval, and continue to fine tune things. The goal is still to have sets updated once per day, but there may be a short hiatus whilst the new code settles in.

These ‘behind-the-scenes’ changes are largely invisible to the user, but are the precursor to something much more interesting. There is a whole new Set Manager in development, with a shiny new interface. There’s still some work to do, but I’m hoping we’ll be ready for launch this side of the new year. Here are a few screenshots as a sneak preview:

Set Manager

New Server

Over the weekend, moved to a new home. Previously, it shared a home with a couple of other fairly heavily used web sites and applications, so I’ve now moved it to a new server.

The DNS updates were made yesterday afternoon and by now everyone should be seeing the new site.

I’ve also taken the opportunity to have a bit of a purge and a tidy up. There’s a new look and feel, which I expect will continue to evolve, a lot of the old, stagnant content has been stripped out, and a new WordPress blog put in its place.

Most of the software tools and applications remain in place, just where they used to be.

If you notice anything awry, or anything I missed that remains broken following the move, do let me know.

A Fresh Start

As is traditional with blogs, my old blog got sadly neglected and consequently withered and died.

I figured it was time to spruce up the web site a little and start afresh with a new blog that I can end up never posting to.

The tidy up and reorganisation is still going on, so you may notice things go awry from time to time – I’ll try to keep disruption to a minimum.