Skip to content

Commit

Permalink
fix: add googleOther bot (#10912)
Browse files Browse the repository at this point in the history
Recognize this new bot from google.

**Important**: already applied in prod.
  • Loading branch information
alexgarel authored Oct 21, 2024
1 parent 8cb1873 commit d56760a
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion lib/ProductOpener/Display.pm
Original file line number Diff line number Diff line change
Expand Up @@ -1016,7 +1016,7 @@ sub set_user_agent_request_ref_attributes ($request_ref) {
my $is_crawl_bot = 0;
my $is_denied_crawl_bot = 0;
if ($user_agent_str
=~ /\b(Googlebot|Googlebot-Image|Google-InspectionTool|bingbot|Applebot|Yandex|DuckDuck|DotBot|Seekport|Ahrefs|DataForSeo|Seznam|ZoomBot|Mojeek|QRbot|Qwant|facebookexternalhit|Bytespider|GPTBot|ChatGPT-User|cohere-ai|anthropic-ai|PerplexityBot|ClaudeBot|Claude-Web|SEOkicks|Searchmetrics|MJ12|SurveyBot|SEOdiver|wotbox|Cliqz|Paracrawl|Scrapy|VelenPublicWebCrawler|Semrush|MegaIndex\.ru|Amazon|aiohttp|python-request|ImagesiftBot|Diffbot)/i
=~ /\b(GoogleOther|Googlebot|Googlebot-Image|Google-InspectionTool|bingbot|Applebot|Yandex|DuckDuck|DotBot|Seekport|Ahrefs|DataForSeo|Seznam|ZoomBot|Mojeek|QRbot|Qwant|facebookexternalhit|Bytespider|GPTBot|ChatGPT-User|cohere-ai|anthropic-ai|PerplexityBot|ClaudeBot|Claude-Web|SEOkicks|Searchmetrics|MJ12|SurveyBot|SEOdiver|wotbox|Cliqz|Paracrawl|Scrapy|VelenPublicWebCrawler|Semrush|MegaIndex\.ru|Amazon|aiohttp|python-request|ImagesiftBot|Diffbot)/i
)
{
$is_crawl_bot = 1;
Expand Down

0 comments on commit d56760a

Please sign in to comment.