What RSS Needs
Sunday, 25 August 2024
More than twenty years ago, Web feeds were all the rage. Not proprietary news feeds on Facebook or ‘X’ – openly defined, direct producer-to-user feeds of information that you had total control over. Without ads. ‘Syndication’ meant that publishers could reach wider audiences without intermediaries; ‘aggregation’ meant that you could get updates from everyone you were interested in without having to hop all over the Web.
I’m talking about RSS and Atom, of course. I have fond memories of the community that launched this, having started the Syndication Yahoo! Group and later going on to co-edit the Atom specification. Since that period of busy activity, however, the underlying technology hasn’t seen much care or attention. There are some bright spots – podcasts have effectively profiled RSS to create a distributed ecosystem, and ActivityPub has taken the mantle of social feeds – but the core ‘I want to track updates from the Web in a feed reader’ use case has languished.
Despite that lack of attention, the feed ecosystem is flourishing; there are many feeds out there, helped by things like automatic feed generation in platforms such as Wordpress. Right now, I’m subscribed to more than a hundred feeds, tracking everything from personal blogs to academic publications to developments in competition law to senate inquiries, and I check them multiple times a day almost every day.
It’s just that feeds could be so much more with some love and directed care – something that could jump from a niche use case to a widespread ‘normal’ part of the Web for many.
It’s also a good time to revitalise feeds. When Google killed Reader years ago, no one questioned their right to do so, even if we grumbled about it. Now, however, regulators are much more aware of the power that platforms have to tilt markets to their benefit, and many are calling for more decentralised approaches to functions that are currently only provided by concentrated intermediaries. People are also more wary of giving away their e-mail addresses for newsletters (the old-tech solution to feeds) when e-mail addresses are rapidly becoming the replacement for tracking by third-party cookies.
With that in mind, here are some of the areas where I think RSS needs some help.
Community
Communication between implementers of a technology is important; it facilitates coordination for addressing bugs and interoperability problems, and smooths the introduction of new features.
Unfortunately, the feed ecosystem has little such coordination. There are few opportunities for developers of software that consumes or produces feeds to talk. This lack of coordination is compounded by how diverse the ecosystem is: there are many implementations on both sides, so it’s hard to improve things for any one actor.
This situation reminds me of the state of the HTTP protocol in the early 2000’s. Back then, that protocol’s implementers were exhausted, because the Web was still scaling up rapidly, and their software needed to mature to match it. HTTP/1.1 had shipped in the late 90’s, and no one was willing to discuss what comes next: they were too busy. Even if they did want to talk about the protocol, there was no natural ‘home’ for it – and that lack of community resulted in numerous interoperability issues and one-off workarounds (anyone remember the X-Pad header field?).
What we did to improve HTTP suggests some possible paths forward for feeds. In 2007, we established the HTTP Working Group in the IETF to act as a home for the protocol. At first it was just a few people who had the time and interest, but over time we had more and more of the implementer community paying attention, eventually taking on HTTP/2.
Not everyone has the time or willpower to participate in standards, however. So, several years ago we started holding the HTTP Workshop, an informal community get-together for implementers and practitioners, where we can discuss our experiences, problems, and proposals to keep the protocol healthy and active.
Both of these approaches could be used to revitalise the feed implementer community over time, if we can get a core of people interested.
User Agency
Feed readers are an example of user agents: they act on behalf of you when they interact with publishers, representing your interests and preserving your privacy and security. The most well-known user agents these days are Web browsers, but in many ways feed readers do it better – they don’t give nearly as much control to sites about presentation and they don’t allow privacy-invasive technologies like cookies or JavaScript.
However, this excellent user agency isn’t well-defined, and we don’t even know if it’s consistent from reader to reader. We need a common understanding of what a feed reader is and what it isn’t, so that users can evaluate whether their reader is a ‘good’ one, and so we can make principled decisions about what a feed reader does and doesn’t do when we extend them.
I started to write about this a while back in Privacy Considerations for Web Feed Readers, but it fizzled out due to the lack of an active community (see above).
Interoperability Tests
Feed readers need to behave in predictable, compatible ways; otherwise publishers won’t know how their content will be presented to users, and won’t trust them to do it. Most readers have settled on a latent profile of the Web stack when they show feed content, but it’s uneven, and that variability limits the use cases for Web feeds.
For example, many YouTube content creators are looking for alternatives because they don’t want to be at the mercy of Google’s algorithm; some are setting up their own Web sites to host video, but are finding that it’s difficult to hold their users’ attention in a sea of choices. Feeds could help – if video interoperates cleanly in feed readers. Does it? We have no idea.
Another example: some feeds that I view in the excellent Reeder show the ‘hero’ image twice: once because it shows up in the entry’s metadata, and once in the content. I suspect that’s the case because the reader that the publisher used didn’t show the metadata-sourced image. Interop tests would have picked this up and helped to create pressure for one way to do it.
Let’s not even get started on feed autodiscovery.
Creating interop tests requires both resources and buy-in from the developer community, but if we want Web feeds to be a platform that publishers create content for, it’s necessary: the Web has set the bar high.
Best Practices for Feeds
Publishers need stronger and more current guidance for how to publisher their feed content. Some of that is the basics: for example, ‘test your feeds, including for autodiscovery’ – although interop issues (per above) makes that a difficult task.
We should also go further and share good patterns. For example, as someone who uses a feed reader, I’m annoyed by all of the ‘subscribe’ banners I see when I click through to the site – it should know that I came from a feed reader. How? If the feed’s links contain a query string that indicates the source, the Web page should be able to hide ‘subscribe’ banners and use cookies to remember that I’m a feed subscriber.
(I can do something about this one more easily than the others by updating the RSS and Atom Feed Tutorial. I’ll put that on my TODO list…)
Browser Integration
Web browsers used to know about feeds: you’d know whether a page had a feed, and could subscribe (either locally, or have it dispatched out to a separate reader). The user experience wasn’t great, but it at least made feeds visible on the Web.
Now, the browser vendors have ripped feed support out, seeming to have the attitude that feeds can be accommodated with extensions. That’s true to a point: Reeder has a Safari extension, for example, and it lets you know when there’s a feed on the page and subscribe to it.
However, using an extension has privacy implications: I need to trust my feed reader to see the content of every Web page I go to serve this function. That’s not great. Also, most users can’t be bothered to walk through the steps of adding extensions: it’s not ‘normal’ on the Web if you have to modify your browser to do it.
Feed support should be built into browsers, and the user experience should be excellent. It should be possible to dispatch to a cloud reader. It should be possible to have customised subscription flows. It should work out of the box so people don’t have to struggle with installing privacy-invasive extensions.
However, convincing the browser vendors that this is in their interest is going to be challenging – especially when some of them have vested interests in keeping users on the non-feed Web.
Authenticated Feeds
Some publishers want to gate their feeds behind a subscription – for example, ars technica has both free and subscriber-only feeds. Right now, that’s only possible through clunky mechanisms like capability URLs, which are less than ideal for shared clients like Web feed readers, because they can ‘leak’ the subscription information, and don’t benefit from caching.
We might not be able to do better than that, but it’s worth considering: would publishers trust third parties like cloud feed readers enough to delegate subscription authentication to them? What would that look like?
Publisher Engagement
Finally, one of the downsides of feeds from a publisher standpoint is that you get very little information about how your feed is used. That’s a huge benefit from a privacy perspective, but it also hinders adoption of Web feeds, forcing people into subscribing by e-mail (which has its own privacy issues).
I recognise that some are fine with this: personally, most of the feed content I consume isn’t commercial, and it’s great. However, if we can make feeds more ‘normal’ by providing some limited feedback to publishers in a privacy preserving way, that could be very good for the ecosystem.
For example, if the Web browser were able to indicate to the Web site what proportion of a site’s audience uses feed readers, publishers can get an idea of how large their potential feed audience is. Keeping in mind that feed readers are likely much ‘stickier’ than other delivery mechanisms, this could be quite attractive.
Or, if feed readers were able to give publishers an indication of what articles were viewed, that would give them the information they need to optimise their content.
On the Web, these kinds of tasks are currently performed with privacy-invasive technologies, often using third-party cookies. Feed readers could take advantage of newer privacy tech like Privacy Preserving Measurement and Oblivious HTTP to provide these functions in much smarter and targeted ways.
However, doing so would require coordination between implementers (see: Community) and a deep respect for the user in how they’re designed (see: User Agency).
What Else?
I’m sure there’s more. I wrote this primarily to gauge interest: who’s up for taking Web feeds to the next level?
If this is interesting to you, I’d love to hear about it. I asked for an IETF mailing list about feeds to be set up a while back, and while it hasn’t been used yet, that’s probably the best place to start – please subscribe and post there!