You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
That could work too. Maybe it would be useful to add a way to pass in a User Agent to the Scrape.X function calls. I notice that you already have fetch(url, html_headers \ []), and that could just be passed onto the next set of functions.
Some sites don't play well without User-Agents (e.g. ESPN).
HTTPoison by default sends hackney which is blocked, so it could be useful to set it to
["User-Agent": "Elixir"] or some similar string to bypass these problems.
The text was updated successfully, but these errors were encountered: