Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to enable google's #57

Open
nklapste opened this issue Dec 12, 2019 · 0 comments
Open

Unable to enable google's #57

nklapste opened this issue Dec 12, 2019 · 0 comments

Comments

@nklapste
Copy link

By default google searches will filter duplicate/similar results. Normally, to override this behavior the filter=0 HTTP parameter is included in the google search request's URL.

I believed the following scrape_job config would provide this override:

let scrape_job = {
    keywords: ["my duplicate generating search"],
    search_engine: 'google',
    google_settings: {
	filter: 0, 
    },
};

However se-scraper still iterates over google search results as if the google's duplicate filtering is still enabled.

Any suggestions to force se-scraper / google to not omit similar search results?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant