include_links breaks the extraction for https://news.ycombinator.com

[shivanker]

Just as the title says. Attaching screenshot as an example.

[adbar]

Hi @shivanker, extraction of main content from what is actually a summary page is tricky, but there is a bug here indeed.

[HammadRafique29]

I go through the source code, and found out (windows) that the include_links feature is working well. The only problem is that the base base_url passed is somehow is None.

I have printed the target link which looks like this.

In above, you can see there is no base_url (used to create a relative url)

You can pass the Url paremeter to get the full url

Here is the output of above code: