Implement RFC5005, "Feed Paging and Archiving".

[jameysharp]

This standard, published in 2007, allows feed readers to efficiently
page through the entire history of a feed, rather than being limited to
the most recent 10 posts.

This implementation supports RFC5005 section 2, "Complete Feeds", for
sites with 10 or fewer posts; and section 4, "Archived Feeds", for sites
with more than 10 posts.

[jameysharp]

I haven't written any significant Ruby before, so I guess I'm not surprised that CI reports there are a bunch of code style issues. I'll have to figure out how to use Rubocop after I get some sleep, I guess. Meanwhile, I'd love to hear what you think of this pull request!

I have a copy of this code running on my personal blog so you can find the feeds it generated from https://jamey.thesharps.us/feed.xml.

[benbalter]

@jameysharp do you have any sense of adoption of the spec? E.g., do any major readers respect/use it? A quickly google search led me to samuelclay/NewsBlur#1109, which a few weeks ago you suggested NewsBlur would be an "early adopter"?

My concern based on the diff is that this adds a lot of complexity, and I'd hate to make the code harder to maintain for a feature no or few readers will use.

[jameysharp]

Sure, I can understand that. The problem I'm trying to tackle is there's a catch-22: if publishers don't implement it, readers won't implement it, and vice versa.

I heard from Manton at micro.blog that a Jekyll implementation of this standard would solve a problem he has with podcasts, and I've heard about the same podcast problem from quite a few people. Podcast publishers currently have to choose between listing all their episodes in one feed so iTunes sees them all, or using a smaller feed to avoid making other feed readers choke. If we can drive adoption of this standard, they won't have to choose.

My personal interest is for webcomics and serial stories, which I built https://www.comic-rocket.com as a special-purpose reader for. I fully expect a new wave of Comic Rocket-like tools to appear if people start offering full-history feeds.

This is the third publisher-side implementation I've written in the last few weeks, so naturally it doesn't seem very complicated to me. Maybe it can be refactored so it doesn't seem complicated to you either? I certainly sympathize with your concern about maintenance burden.

[pathawks]

I heard from Manton at micro.blog that a Jekyll implementation of this standard would solve a problem he has with podcasts

This plugin does not create a podcast feed, so iTunes isn't really part of the equation here. I would be very curious to see a list of feed readers that would support this feature.

[pathawks]

With 004bd3b I have fixed most of Rubocop's complaints. ✌️

The last remaining issue is that generate is too complicated. Some of the logic will need to be extracted into a helper method.

[jameysharp]

Thanks for the Rubocop fixes! I've refactored further to eliminate its complaint about the AbcSize metric, and now there are no Rubocop complaints. Hooray! Is this better?

[samuelclay]

@jameysharp do you have any sense of adoption of the spec? E.g., do any major readers respect/use it? A quickly google search led me to samuelclay/NewsBlur#1109, which a few weeks ago you suggested NewsBlur would be an "early adopter"?

My concern based on the diff is that this adds a lot of complexity, and I'd hate to make the code harder to maintain for a feature no or few readers will use.

Funny to see this thread linked here. NewsBlur is about to launch RSS backfill for premium archive subscribers, and the NewsBlur blog recently moved to Jekyll. So I went to check if Jekyll supports backfill and it seems it will as soon as this PR is merged. Anything I can do to help prioritize that? Happy to blog about Jekyll supporting RSS pagingation, joining WordPress in automatically enabling feed paging.

The impetus for bringing complete archives to NewsBlur is how misinformation and censorship are changing the way we consume news, and a searchable archive pushes back on the effort to rewrite the past. We need access to news and information published in the past because we need to be concerned about how our own present is remembered in the future. Having historical context, in the same interface and reader as today's news, buys optionality for those finding themselves in oppressive societies.

Thanks @jameysharp for bringing this together!

[benbalter]

do you have any sense of adoption of the spec? E.g., do any major readers respect/use it?

@samuelclay thanks for reviving this old thread. Given that it's been nearly four years, are you able to share (🤞🏻) any news around broader adoption of the proposed spec (or insight into if it may be accepted)?

[samuelclay]

@samuelclay thanks for reviving this old thread. Given that it's been nearly four years, are you able to share (🤞🏻) any news around broader adoption of the proposed spec (or insight into if it may be accepted)?

When surveying competing news readers, I do know that we all are pretty aggressive about feature parity. I'm about to weave archiving throughout the interface, as I help surface older stories that are related to the latest stories and make it easier to bridge connections between stories.

Organizational features like this have been a regular addition to all modern news readers. Plus I'm going to market this feature in anticipation of more stories like the Clay Shirky blog disappearing.