RSS Feeds from websites without RSS Feeds

RSS Feeds are great. You can follow all the online content you care about in one place. Emails of comic updates will not get buried and you can occasionally get updates from infrequent blogs, like this one. I personally use NewsBlur.com.

But some websites limit the RSS Feeds available and do not get specific.

For example, I enjoy reading Wall Street Journal (WSJ) Sports because they discuss larger trends in sports, rather than game highlights. WSJ only offers 6 RSS Feeds, with Sports sometimes being part of the Lifestyle feed. I want to get all updates for WSJ Sports.

There are multiple different solutions and all have limiting free versions. Some monitor a webpage and detect differences (like an image or text difference) and others require more user input. Some are RSS readers themselves and others just generate RSS feeds.

The best solution for WSJ Sports has been to generate a RSS feed using FetchRss.com. They will monitor 5 feeds (others do 2) and check daily (plenty enough for WSJ Sports). They only keep a 5 article history (not good for active websites) but NewsBlur does the article saving. If you want faster updates (up to the minute), all services will charge for that.

FetchRss uses CSS Selectors that the user needs to identify to generate the RSS feed. In a few months, the selectors on WSJ Sports has not changed, but this method is prone to breakages. The visual guide is good at picking out selectors without looking at code and will work mostly. Avoid using the selectors of articles in the Featured or the Top sections of websites, since then you will not get all the articles a website publishes (not the goal of a RSS Feed).

These are the selectors I am using for my feed and this is my WSJ Sports feed.

News item
#latest-stories > article.WSJTheme--story--XB4V2mLz

Headline
div.WSJTheme--headline--7VCzo7Ay

Summary
p.WSJTheme--summary--lmOXEsbN

Illustration
div.WSJTheme--lazy-load-wrapper--1TOawUTd

This entry was posted in How-To and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s