Undetected user agents #100

JuMiLo · 2018-10-18T12:02:41Z

Hi,

the following user agents doesn't get detected.

Agents with "browser->using" object:
Mozilla/5.0 (Linux; Android 8.0.0; SM-G950F Build/R16NW; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/69.0.3497.100 Mobile Safari/537.36

Mozilla/5.0 (Linux; Android 6.0.1; SM-G800F Build/MMB29K; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/69.0.3497.100 Mobile Safari/537.36

Agents with empty "browser" object:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/601.7.8 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/534.57.7 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.59.10 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/603.3.8 (KHTML, like Gecko)

Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16A366

Mozilla/5.0 (iPad; CPU OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16A366

Mozilla/5.0 (Linux; NetCast; U) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/537.31 SmartTV/6.0

Mozilla/5.0 (iPad; CPU OS 9_3_5 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13G36

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/605.1.15 (KHTML, like Gecko)

Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; tb-gmx/2.6.12; (gmx/1.5.0.0); rv:11.0) like Gecko

Mozilla/5.0 (iPhone; CPU iPhone OS 12_0_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Snapchat/10.43.0.25 (iPhone7,2; iOS 12.0.1; gzip)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/605.1.15 (KHTML, like Gecko)

Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.0.1) Gecko/2008070208

Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:2.2) Gecko/20110201

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)

Mozilla/5.0 (iPad; CPU OS 10_3_3 like Mac OS X) AppleWebKit/603.3.8 (KHTML, like Gecko) Mobile/14G60

Mozilla/5.0 (iPad; CPU OS 11_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E218

Mozilla/5.0 (iPhone; CPU iPhone OS 12_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) (Ecosia [email protected])

Mozilla/5.0 (iPad; CPU OS 11_2_6 like Mac OS X) AppleWebKit/604.5.6 (KHTML, like Gecko) Mobile/15D100

Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_3 like Mac OS X) AppleWebKit/603.3.8 (KHTML, like Gecko) Mobile/14G60

Mozilla/5.0 (iPhone; CPU iPhone OS 12_0_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16A405

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/600.8.9 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/603.1.30 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.78.2 (KHTML, like Gecko)

Mozilla/5.0 (Macintosh; Intel Mac OS X) AppleWebKit (KHTML, like Gecko)

Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; tb-webde/2.7.4; (webde/1.4.0.0); rv:11.0) like Gecko

swcd (unknown version) CFNetwork/974.2.1 Darwin/18.0.0
swcd (unknown version) CFNetwork/758.5.3 Darwin/15.6.0
swcd (unknown version) CFNetwork/889.9 Darwin/17.2.0
swcd (unknown version) CFNetwork/894 Darwin/17.4.0
swcd (unknown version) CFNetwork/897.15 Darwin/17.5.0
swcd (unknown version) CFNetwork/902.2 Darwin/17.7.0
swcd (unknown version) CFNetwork/901.1 Darwin/17.6.0
swcd (unknown version) CFNetwork/893.14.2 Darwin/17.3.0
swcd (unknown version) CFNetwork/975.0.3 Darwin/18.2.0
SafeBrowser/1.11.0.38 CFNetwork/974.2.1 Darwin/18.0.0

JuMiLo · 2018-11-02T18:02:22Z

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/601.4.4 (KHTML, like Gecko)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/601.7.7 (KHTML, like Gecko)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/604.3.5 (KHTML, like Gecko)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/605.1.15 (KHTML, like Gecko)
Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; (webde/1.5.0.0); rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; rv:11.0; BR-Safe-Browsing) like Gecko
Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; tb-gmx/2.6.9; (gmx/1.4.0.0); rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 6.1; Trident/7.0; tb-gmx/2.7.5; (gmx/1.4.0.0); rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; tb-webde/2.7.4; (web.de/1.3.0.0); rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; (webde/1.4.0.0); rv:11.0) like Gecko
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.1) Gecko/20100122 firefox/3.6.1
Mozilla/5.0 (iPad; CPU OS 10_3_2 like Mac OS X) AppleWebKit/603.2.4 (KHTML, like Gecko) Mobile/14F89
Mozilla/5.0 (iPad; CPU OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko)
Mozilla/5.0 (iPhone; CPU iPhone OS 10_2 like Mac OS X) AppleWebKit/602.3.12 (KHTML, like Gecko) Mobile/14C92
Mozilla/5.0 (iPhone; CPU iPhone OS 11_2_2 like Mac OS X) AppleWebKit/604.4.7 (KHTML, like Gecko) Mobile/15C202
Mozilla/5.0 (iPhone; CPU iPhone OS 11_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15F79
Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_5 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13G36
swcd (unknown version) CFNetwork/808.2.16 Darwin/16.3.0
swcd (unknown version) CFNetwork/811.4.18 Darwin/16.5.0

rvalitov · 2018-11-11T12:26:23Z

I made a test for Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.59.10 (KHTML, like Gecko). I get:

WhichBrowser\Parser Object
(
    [browser] => WhichBrowser\Model\Browser Object
        (
            [using] => 
            [family] => 
            [channel] => 
            [stock] => 1
            [hidden] => 
            [mode] => 
            [type] => 
            [name] => 
            [alias] => 
            [version] => 
        )

    [engine] => WhichBrowser\Model\Engine Object
        (
            [name] => Webkit
            [alias] => 
            [version] => WhichBrowser\Model\Version Object
                (
                    [value] => 534.59.10
                    [hidden] => 
                    [nickname] => 
                    [alias] => 
                    [details] => 
                    [builds] => 
                )

        )

    [os] => WhichBrowser\Model\Os Object
        (
            [family] => 
            [edition] => 
            [hidden] => 
            [name] => OS X
            [alias] => Mac OS X
            [version] => WhichBrowser\Model\Version Object
                (
                    [value] => 10.6.8
                    [hidden] => 
                    [nickname] => 
                    [alias] => 
                    [details] => 2
                    [builds] => 
                )

        )

    [device] => WhichBrowser\Model\Device Object
        (
            [manufacturer] => Apple
            [model] => Macintosh
            [series] => 
            [carrier] => 
            [identifier] => 
            [flag] => 
            [type] => desktop
            [subtype] => 
            [identified] => 1
            [generic] => 1
            [hidden] => 1
        )

    [camouflage] => 
    [features] => Array
        (
        )

    [cache:WhichBrowser\Parser:private] => 
    [expires:WhichBrowser\Parser:private] => 
    [cached] => 
)

It says stock, so it's not empty. What result do you expect?

JuMiLo · 2018-11-12T15:54:25Z

With empty I mean that no "browser->name" or "browser->version" is detected.

JuMiLo · 2018-11-12T15:55:11Z

Is there a way to add user agents manually?

etetlow · 2019-01-28T11:01:36Z

I can see myself quite plainly that the reason you're not getting any browser information back is that there's no browser information in most of the above user agent strings. You can't have what's not there. There was one that I thought would be caught, e.g. Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.1) Gecko/20100122 firefox/3.6.1 but wasn't, but all of the others that I tried that have any browser information came up fine.

jfarcas · 2019-01-29T17:53:45Z

Hi,

the following user agents doesn't get detected.

atc/1.0 watchOS/5.1.3 model/Watch3,4 hwp/t8004 build/16S5535a (6; dt:156)

ericmillsio · 2019-02-04T17:41:44Z

Outlook on iOS does not get detected. The value in parenthesis is the iOS outlook app version I believe.

Outlook-iOS/687.32416.prod.iphone (2.81.0)

maraouf · 2020-05-05T21:23:36Z

Some bots that are not added in the library that I had to add them manually

Would be nice to add them in the repo for future versions

    [ 'name' => 'Cron Job',                     'id'    => 'cron-job',      'regexp' => '/cron-job.org/u' ],
    [ 'name' => 'Google Stackdriver Monitoring - Uptime Checks', 'id'    => 'googlestackdrivermonitoring',      'regexp' => '/GoogleStackdriverMonitoring-UptimeChecks/u' ],
    [ 'name' => 'Byte Spider',                  'id'    => 'bytespider',      'regexp' => '/Bytespider/u' ],
    [ 'name' => 'Dot Bot',                      'id'    => 'dotbot',      'regexp' => '/DotBot\/([0-9.]*)/u' ],
    [ 'name' => 'Nimbostratus Bot',             'id'    => 'nimbostratus-bot',      'regexp' => '/Nimbostratus-Bot\/([0-9.]*)/u' ],
    [ 'name' => 'Uptime Bot',                   'id'    => 'uptimebot',      'regexp' => '/Uptimebot\/([0-9.]*)/u' ],
    [ 'name' => 'Netcraft Survey Agent',        'id'    => 'netcraftsurveyagent',      'regexp' => '/NetcraftSurveyAgent\/([0-9.]*)/u' ],
    [ 'name' => 'Domain Crawler',               'id'    => 'domaincrawler',      'regexp' => '/domaincrawler.com/u' ],
    [ 'name' => 'Zoominfo Bot',                 'id'    => 'zoominfobot',      'regexp' => '/zoominfobot/u' ],
    [ 'name' => 'FullStory Bot',                'id'    => 'fullstorybot',      'regexp' => '/FullStoryBot\/([0-9.]*)/u' ],
    [ 'name' => 'Semrush Bot',                  'id'    => 'semrushbot',      'regexp' => '/SemrushBot\/([0-9.]*)/u' ],
    [ 'name' => 'MJ12 Bot',                     'id'    => 'mj12bot',      'regexp' => '/MJ12bot/u' ],
    [ 'name' => 'Survey Bot',                   'id'    => 'surveybot',      'regexp' => '/SurveyBot\/([0-9.]*)/u' ],
    [ 'name' => 'BLEX Bot',                     'id'    => 'blexbot',      'regexp' => '/BLEXBot\/([0-9.]*)/u' ],
    [ 'name' => 'Mail.RU Bot',                  'id'    => 'mail\.ru_bot',      'regexp' => '/Mail.RU_Bot\/([0-9.]*)/u' ],
    [ 'name' => 'Researchscan',                 'id'    => 'researchscan',      'regexp' => '/Researchscan/u' ],
    [ 'name' => 'OrgProbe',                     'id'    => 'orgprobe',      'regexp' => '/OrgProbe\/([0-9.]*)/u' ],
    [ 'name' => 'Grapeshot Crawler',            'id'    => 'grapeshotcrawler',      'regexp' => '/GrapeshotCrawler\/([0-9.]*)/u' ],
    [ 'name' => 'SP Bot',                       'id'    => 'spbot',      'regexp' => '/spbot\/([0-9.]*)/u' ],
    [ 'name' => 'KomodiaBot',                   'id'    => 'komodiabot',      'regexp' => '/KomodiaBot\/([0-9.]*)/u' ],
    [ 'name' => 'WhatsApp',                     'id'    => 'whatsapp',      'regexp' => '/WhatsApp\/([0-9.]*)/u' ],
    [ 'name' => 'Bugsnag',                      'id'    => 'bugsnag',      'regexp' => '/Bugsnag/u' ],
    [ 'name' => 'SP Bot',                       'id'    => 'spbot',      'regexp' => '/spbot\/([0-9.]*)/u' ],
    [ 'name' => 'MASSCAN',                      'id'    => 'masscan',      'regexp' => '/masscan\/([0-9.]*)/u' ],
    [ 'name' => 'Cloudflare-AMP',               'id'    => 'cloudflare-amp',      'regexp' => '/Cloudflare-AMP\/([0-9.]*)/u' ],
    [ 'name' => 'aiHit Bot',                    'id'    => 'aihitbot',      'regexp' => '/aiHitBot\/([0-9.]*)/u' ],
    [ 'name' => 'Coccocbot Image',              'id'    => 'coccocbot-image',      'regexp' => '/coccocbot-image\/([0-9.]*)/u' ],

also there is a spelling mistake in CloudFlare, it is written CloudFare

[ 'name' => 'CloudFlare Always Online', 'id' => 'cloudflare', 'regexp' => '/CloudFlare-AlwaysOnline\/([0-9.]*)/u' ],

Xaraxia · 2020-05-12T04:39:02Z

Here's another one that is not detected:
"check_http/v2.3.1 (nagios-plugins 2.3.1)"

Not a big deal, since the only usual use case for this one is checking that a site is up, but FYI. That is the complete user-agent string. Browscap seems to be the only library that picks it up. http://thadafinser.github.io/UserAgentParserComparison/v4/user-agent-detail/dc/d5/dcd5e41b-7bc9-4a3b-b233-135826d90385.html

summercms · 2020-06-11T18:13:08Z

This is not my repo, but the last few comments are hijacking this issue.

You should open a separate issue and create a pr for any missing bots, crawlers, devices or browsers.

You can see some pr's I've opened as an example.

Added Nimbostratus bot it's quite common on the internet. As per request: WhichBrowser#100 (comment)

Added Majestic (MJ12) bot it's quite common seo bot on the internet. As per request: WhichBrowser#100 (comment)

This was referenced Jun 10, 2020

Correct Cloudflare bot name #458

Merged

Moz bot #477

Merged

summercms pushed a commit to summercms/Parser-PHP that referenced this issue Jun 11, 2020

Nimbostratus Bot

eafb524

Added Nimbostratus bot it's quite common on the internet. As per request: WhichBrowser#100 (comment)

This was referenced Jun 11, 2020

Nimbostratus Bot #478

Merged

Semrush Bot #479

Merged

summercms pushed a commit to summercms/Parser-PHP that referenced this issue Jun 11, 2020

Majestic (MJ12) Bot

bc9d37e

Added Majestic (MJ12) bot it's quite common seo bot on the internet. As per request: WhichBrowser#100 (comment)

This was referenced Jun 11, 2020

Majestic (MJ12) Bot - Seems to be issue adding this - tests are coming up blank ? #480

Closed

Netcraft Survey Bot #482

Merged

DomainCrawler Bot #483

Merged

Mail.ru Bot #484

Merged

Bytespider Bot #485

Merged

This was referenced Oct 24, 2020

Add Majestic MJ12 Bot a very common seo bot #596

Closed

Add Majestic MJ12 Bot a very common seo bot #597

Closed

Add Blex Bot to bots list #606

Closed

Update Netcraft Bots with more regex (there are 4 bot combinations) #608

Closed

summercms mentioned this issue Feb 1, 2021

Add ZoomInfo Bot #640

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Undetected user agents #100

Undetected user agents #100

JuMiLo commented Oct 18, 2018 •

edited

Loading

JuMiLo commented Nov 2, 2018

rvalitov commented Nov 11, 2018

JuMiLo commented Nov 12, 2018

JuMiLo commented Nov 12, 2018

etetlow commented Jan 28, 2019

jfarcas commented Jan 29, 2019

ericmillsio commented Feb 4, 2019 •

edited

Loading

maraouf commented May 5, 2020 •

edited

Loading

Xaraxia commented May 12, 2020

summercms commented Jun 11, 2020

Undetected user agents #100

Undetected user agents #100

Comments

JuMiLo commented Oct 18, 2018 • edited Loading

JuMiLo commented Nov 2, 2018

rvalitov commented Nov 11, 2018

JuMiLo commented Nov 12, 2018

JuMiLo commented Nov 12, 2018

etetlow commented Jan 28, 2019

jfarcas commented Jan 29, 2019

ericmillsio commented Feb 4, 2019 • edited Loading

maraouf commented May 5, 2020 • edited Loading

Xaraxia commented May 12, 2020

summercms commented Jun 11, 2020

JuMiLo commented Oct 18, 2018 •

edited

Loading

ericmillsio commented Feb 4, 2019 •

edited

Loading

maraouf commented May 5, 2020 •

edited

Loading