Handling of almost-duplicates?

[tony-si]

Hi,

I subscribe to this BBC news feed at https://feeds.bbci.co.uk/news/rss.xml. They seem to feel the need to re-publish items quite frequently with slight modifications, e.g. title or pubDate modified, but the same guid. Feeder seems to show these as separate items rather than updating the existing item, so sometimes I'm seeing the same item two or three times, or even more. Below is an example of an item where only the pubDate has changed, the two versions were captured 50 minutes apart or so. I didn't want to report this as a bug, because I'm not sure if it's intended behaviour or not. Is it this way on purpose?

        <item>
            <title><![CDATA[Suspensions after 'Israel' crossed out on document]]></title>
            <description><![CDATA[The staff at a private company which works with the Home Office were suspended after an investigation.]]></description>
            <link>https://www.bbc.co.uk/news/uk-68367366</link>
            <guid isPermaLink="false">https://www.bbc.co.uk/news/uk-68367366</guid>
            <pubDate>Thu, 22 Feb 2024 07:33:03 GMT</pubDate>
            <media:thumbnail width="240" height="135" url="https://ichef.bbci.co.uk/ace/standard/240/cpsprodpb/DFF2/production/_128303375_f3f724100295edbb769bdfa6f85db3071821a537.jpg"/>
        </item>

        <item>
            <title><![CDATA[Suspensions after 'Israel' crossed out on document]]></title>
            <description><![CDATA[The staff at a private company which works with the Home Office were suspended after an investigation.]]></description>
            <link>https://www.bbc.co.uk/news/uk-68367366</link>
            <guid isPermaLink="false">https://www.bbc.co.uk/news/uk-68367366</guid>
            <pubDate>Thu, 22 Feb 2024 09:49:28 GMT</pubDate>
            <media:thumbnail width="240" height="135" url="https://ichef.bbci.co.uk/ace/standard/240/cpsprodpb/DFF2/production/_128303375_f3f724100295edbb769bdfa6f85db3071821a537.jpg"/>
        </item>

Tony

[spacecowboy]

If they have the same guid and are part of the same feed, they should not show up again.

Having items be updated is an expected and supported feature.

So either you are subscribing to 2 different feeds which get the same items posted sometimes (you'll see both in that case) OR they actually changed the guid in one of their updates. Sadly changing guids is not uncommon for some publishers.

[tony-si]

Thanks for the reply. I am only subscribed to one feed other than "Feeder News" - see screenshots - so my dupes are definitely coming from one feed :). I can't obviously see what is causing Feeder to display those dupes, but I don't see those dupes in my desktop RSS reader (Feedbro), so something is causing Feeder to display dupes where Feedbro does not. I obviously can't see what Feeder is doing internally, so I can only really speculate on the cause. I can repeat what I did, save the feed XML periodically from my browser, and try to see what the issue might be. If you have any other suggestions for diagnostic steps I should take, please let me know.

[spacecowboy]

not much you can do in feeder
you'd need to check the guid field in the database.

you didn't enable alternateid for the feed, right?

[spacecowboy]

and its clear from tve screenshot that bbc updated the article because the title has a fixed hyphen

so I'm 99% sure they changed the guid when they changed to title

[tony-si]

By alternateid, is that the "Generate extra unique IDs" option? No, it's off.