I modified a couple python scripts floating around to take an RSS feed containing <link>’s to other blogs from, say Dapper feeds, run the autodiscovery routines to find any feeds provided by those linked blogs and output an OPML file. Seemed like a pretty obviously useful function but I couldn’t find anyone else who had done it.
Credits:
- Ultra-liberal RSS feed locator (aka RSSfinder) by Mark Pilgrim.
- All-RSS-within-one-click finder by Phillip Pearson
download rsslist2opml.py.
You will also need the RSSfinder script linked above as a dependency. I’ll mirror the version I used.
The outputted OPML has been vetted to work with Yahoo Pipes (quirky, I must say) and validated. I tested it with this OPML feed aggregator and this Fetch OPML as RSS converter.
Caveats
- It does not do any checking for duplicates in the incoming RSS feed. It does prevent the source site with the original RSS feed from being included (there is usually a link from this domain at the header of Dapper RSS).
- It is written (though this could easily be modified) to only export one feed per inputted item even if RSSfinder finds more than that.
- It takes quite a while to run (several minutes if the list is long–it is “ultra-liberal” after all), I might enhance it so with some form of caching so that you can just use it as CGI and not worry about it trying to regenerate the entire OPML every time you load the script.
- First python jab so it is likely messy.
Example
I used Dapper to make a nice RSS feed out of this list of Peace Corps blogs in Tanzania. The script will take the RSS feed as input and output OPML of the linked blog’s feeds. This can be aggregated (slowly) by Yahoo Pipes. I’m playing with other display options.
Update
So when I went back and started working on my little application of this OPML that I’d been working on I noticed Yahoo had made a new release of Pipes. They added a Feed-Autodiscovery pipe. Joy, my work is redundant! Ah well, the joy is in the chase. Maybe someone could use it for something like the freeform, traditional text-based feed pipeline platform, also by a Yahoo Dev, FeedMagik2.
My relatively functional app is live at Peace Corps Feeds.