From 67e5b3fe1e9a9b7e3a6d25ec0573e9e86e27a790 Mon Sep 17 00:00:00 2001 From: RameezIbrahimOfficial Date: Tue, 15 Oct 2024 12:01:32 +0530 Subject: [PATCH 1/2] Added OSINT free and opensource APIs --- APIs.md | 429 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 429 insertions(+) create mode 100644 APIs.md diff --git a/APIs.md b/APIs.md new file mode 100644 index 0000000..1b60ba3 --- /dev/null +++ b/APIs.md @@ -0,0 +1,429 @@ +# OSINT Free and Open-Source APIs + +## **General OSINT APIs** + +### 1. [Shodan API](https://developer.shodan.io/) +- **Description**: Shodan is a search engine for internet-connected devices. It allows you to query IoT devices, servers, webcams, and more. +- **Free Tier**: Yes (limited) +- **Authentication**: API Key +- **Usage Limit**: 1,000 queries per month (free tier) + +### 2. [Have I Been Pwned](https://haveibeenpwned.com/API/v3) +- **Description**: Allows you to check if an email or domain has been exposed in a data breach. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 requests per month (free tier) + +### 3. [Amass API](https://github.com/OWASP/Amass) +- **Description**: Amass is a tool used for network mapping, subdomain enumeration, and DNS discovery. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited (depends on setup) + +### 4. [Censys](https://censys.io/) +- **Description**: Provides data on devices and servers exposed to the internet. You can search and query the data via the API. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 50 requests per day (free tier) + +### 5. [IPinfo API](https://ipinfo.io/) +- **Description**: Provides geolocation and IP data, including the organization, country, region, and city information. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 50,000 requests per month (free tier) + +--- + +## **Geolocation & IP APIs** + +### 6. [ipapi](https://ipapi.co/) +- **Description**: Provides geolocation data based on the IP address, offering insights like location, currency, and languages spoken. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 requests per month (free tier) + +### 7. [GeoIP2 by MaxMind](https://www.maxmind.com/en/geoip2-services) +- **Description**: Provides IP address geolocation services, including city, country, and organization information. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 requests per day + +### 8. [IPStack](https://ipstack.com/) +- **Description**: Provides geolocation information for an IP address, including city, region, country, and more. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 requests per month (free tier) + +### 9. [DB-IP](https://db-ip.com/) +- **Description**: Offers geolocation data for IP addresses, including location details such as country, city, and postal code. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 requests per day + +### 10. [WhatIsMyIP API](https://www.whatismyip.com/) +- **Description**: Allows you to find the public IP address of a user, as well as geolocation information. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +--- + +## **Domain & DNS APIs** + +### 11. [DNSdumpster](https://dnsdumpster.com/) +- **Description**: A free online resource that can be used to gather DNS information for domains, including subdomains and IP addresses. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +### 12. [WhoisXML API](https://whoisxmlapi.com/) +- **Description**: Provides access to WHOIS information for domain names, including registration details. +- **Free Tier**: Yes (limited access) +- **Authentication**: API Key +- **Usage Limit**: 500 queries per month + +### 13. [IPVoid](https://www.ipvoid.com/) +- **Description**: An IP reputation tool for checking if an IP address is flagged for malicious activities. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: 50 requests per day + +### 14. [VirusTotal API](https://www.virustotal.com/) +- **Description**: Allows you to scan URLs, domains, and IP addresses to check for malware, phishing, and other malicious activities. +- **Free Tier**: Yes (limited queries) +- **Authentication**: API Key +- **Usage Limit**: 4 requests per minute (free tier) + +### 15. [Robtex](https://www.robtex.com/) +- **Description**: Offers information about IP addresses, DNS records, and WHOIS data. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +--- + +## **Email & Social Media Intelligence** + +### 16. [Hunter.io](https://hunter.io/) +- **Description**: A tool for finding email addresses related to a specific domain name. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 50 requests per month (free tier) + +### 17. [EmailHippo](https://emailhippo.com/) +- **Description**: Email validation and verification service to detect if an email is real and valid. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: 25 requests per day + +### 18. [Clearbit](https://clearbit.com/) +- **Description**: Provides company and social data based on email addresses or domain names. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 50 requests per month + +### 19. [Social Search](https://www.social-search.com/) +- **Description**: Search for social media profiles associated with a specific email address or username. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +--- + +## **Security & Vulnerability APIs** + +### 20. [Exploit Database API](https://www.exploit-db.com/) +- **Description**: A database of exploits and vulnerabilities that provides access via API. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +### 21. [OSINT Framework](https://osintframework.com/) +- **Description**: An OSINT framework with links to multiple publicly available intelligence tools and APIs. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +### 22. [Open Threat Exchange (OTX)](https://otx.alienvault.com/) +- **Description**: Provides threat intelligence from the community, including indicators of compromise (IoCs). +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 500 queries per day + +### 23. [Shodan Security API](https://developer.shodan.io/) +- **Description**: Provides data about internet-connected devices, including vulnerabilities and security risks. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 queries per month + +### 24. [PunkSpider](https://punkspider.org/) +- **Description**: A security tool that scans the internet for vulnerabilities, particularly related to websites. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +--- + +## **Social Media & Web Scraping APIs** + +### 25. [Twitter API](https://developer.twitter.com/en/docs) +- **Description**: Allows interaction with Twitter data, including tweets, followers, and trends. +- **Free Tier**: Yes +- **Authentication**: OAuth +- **Usage Limit**: 500,000 requests per month (free tier) + +### 26. [Reddit API](https://www.reddit.com/dev/api/) +- **Description**: Provides programmatic access to Reddit posts, comments, and subreddits. +- **Free Tier**: Yes +- **Authentication**: OAuth +- **Usage Limit**: 60 requests per minute + +### 27. [Instagram Graph API](https://developers.facebook.com/docs/instagram-api) +- **Description**: Provides programmatic access to Instagram business accounts, media, and user profiles. +- **Free Tier**: Yes +- **Authentication**: OAuth +- **Usage Limit**: Varies + +### 28. [Scrapy Cloud](https://scrapinghub.com/) +- **Description**: A platform for running web scraping jobs at scale. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: Limited on + + free tier + +--- + +## **Other Useful OSINT APIs** + +### 29. [Google Safe Browsing](https://developers.google.com/safe-browsing/) +- **Description**: Provides real-time protection against phishing and malware by querying Google’s Safe Browsing lists. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 10,000 requests per day + +### 30. [Cyber Threat Intelligence API](https://www.recordedfuture.com/) +- **Description**: Provides access to real-time cyber threat intelligence data. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: Limited on free tier + +--- + +## **Miscellaneous OSINT APIs** + +### 31. [Wayback Machine API](https://archive.org/web/) +- **Description**: Access to archived web pages from the Wayback Machine. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +### 32. [WHOIS API](https://www.whoisxmlapi.com/) +- **Description**: Allows you to query WHOIS data about domains, including registration information. +- **Free Tier**: Yes (limited access) +- **Authentication**: API Key +- **Usage Limit**: 500 requests per month + +--- + +Absolutely! Here’s an extended list of **more free and open-source OSINT APIs**, bringing the total to **50+ APIs**. These APIs cover a broader range of open-source intelligence sources, including additional security, social media, geolocation, and other investigative tools. + +--- + +### 33. [ip-api](http://ip-api.com/) +- **Description**: A fast and accurate IP geolocation API for location and organization data based on IP address. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: 45 requests per minute (free tier) + +### 34. [GeoLite2 by MaxMind](https://www.maxmind.com/en/geoip2-databases) +- **Description**: Provides geolocation services with accuracy on city, country, and organization data for IP addresses. +- **Free Tier**: Yes (GeoLite2 version) +- **Authentication**: None +- **Usage Limit**: Unlimited (depends on setup) + +### 35. [IPQualityScore](https://www.ipqualityscore.com/) +- **Description**: Provides fraud prevention and IP geolocation data including proxy detection. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 requests per month (free tier) + +### 36. [IP2Location](https://www.ip2location.com/web-service) +- **Description**: Provides geolocation services, including city, region, country, and proxy detection based on IP addresses. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 requests per day (free tier) + +--- + +## **Email & Domain Intelligence (Continued)** + +### 37. [MxToolbox](https://mxtoolbox.com/api.aspx) +- **Description**: Provides DNS, SMTP, and blacklist checking tools for domain and email server analysis. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: 1,000 requests per month + +### 38. [Mailgun](https://www.mailgun.com/) +- **Description**: Email validation and delivery services that also allow for domain and email security analysis. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 100 emails per day (for free tier) + +### 39. [Mailtester](https://www.mailtester.com/) +- **Description**: A tool to validate the existence of email addresses in real-time to help identify spam traps. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Limited daily requests + +### 40. [Hunter.io Email Finder](https://hunter.io/) +- **Description**: Finds email addresses linked to domains or specific names for outreach and investigation. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 50 requests per month (free tier) + +--- + +## **Social Media & Web Scraping (Continued)** + +### 41. [Social Search API](https://social-search.com/) +- **Description**: A search engine for discovering social media profiles associated with a given username or email address. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +### 42. [Webhose.io](https://webhose.io/) +- **Description**: Web scraping tool that aggregates data from blogs, news sites, and forums across the web. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 requests per month (free tier) + +### 43. [Scrapy API](https://scrapy.org/) +- **Description**: A popular Python framework for web scraping, allowing you to crawl websites and gather data programmatically. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Depends on usage + +### 44. [Twitter Developer API](https://developer.twitter.com/en/docs) +- **Description**: Allows you to access Twitter data, including tweets, user profiles, and trends. +- **Free Tier**: Yes +- **Authentication**: OAuth +- **Usage Limit**: 500,000 requests per month + +### 45. [Facebook Graph API](https://developers.facebook.com/docs/graph-api/) +- **Description**: Allows interaction with Facebook data, such as user information, pages, groups, and posts. +- **Free Tier**: Yes +- **Authentication**: OAuth +- **Usage Limit**: Varies by app and endpoint + +### 46. [LinkedIn API](https://www.linkedin.com/developers/) +- **Description**: Provides programmatic access to LinkedIn user profiles, job postings, and company data. +- **Free Tier**: Yes +- **Authentication**: OAuth +- **Usage Limit**: Varies based on usage + +--- + +## **Cybersecurity & Threat Intelligence APIs (Continued)** + +### 47. [AbuseIPDB API](https://www.abuseipdb.com/) +- **Description**: A public repository of reports on IP addresses that have been involved in malicious activity. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 queries per day + +### 48. [AlienVault OTX](https://otx.alienvault.com/) +- **Description**: Provides access to a community-driven threat intelligence platform for threat indicators (IoCs). +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 500 queries per day + +### 49. [Vulners API](https://vulners.com/) +- **Description**: A vulnerability database that provides access to CVE data, software vulnerabilities, and other security risks. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: 100 queries per day (free tier) + +### 50. [HackerTarget](https://hackertarget.com/) +- **Description**: Provides various security tools for vulnerability scanning, including DNS lookup, reverse DNS, and domain analysis. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: 50 requests per day + +--- + +## **Publicly Available Datasets & Miscellaneous APIs** + +### 51. [The Commonsense API](https://www.commonsense.org/) +- **Description**: Provides access to data on non-profit organizations, government data, and more. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +### 52. [US Census Bureau API](https://www.census.gov/data/developers/) +- **Description**: Provides access to U.S. Census data, which can be useful for demographic analysis. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +### 53. [Bing Web Search API](https://www.microsoft.com/en-us/bing/apis/bing-web-search-api-v7) +- **Description**: Allows you to access web search data from Bing, including images, news, and web results. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 1,000 requests per month + +### 54. [Google Custom Search API](https://developers.google.com/custom-search/v1/overview) +- **Description**: Provides access to Google's search index, enabling you to query specific websites or web pages. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 100 queries per day (free tier) + +--- + +## **DNS & Network Analysis (Continued)** + +### 55. [DNSQuery API](https://www.dnsquery.io/) +- **Description**: Allows you to perform DNS queries and retrieve records like A, MX, NS, and TXT. +- **Free Tier**: Yes +- **Authentication**: API Key +- **Usage Limit**: 100 queries per day (free tier) + +### 56. [URLscan.io](https://urlscan.io/) +- **Description**: A service that provides screenshots and metadata of websites, helping to analyze their contents. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: 50 requests per month + +### 57. [Spyse](https://spyse.com/) +- **Description**: An internet-wide data search engine that allows you to query information related to IPs, domains, and certificates. +- **Free Tier**: Yes (limited access) +- **Authentication**: API Key +- **Usage Limit**: Varies by access + +--- + +## **Miscellaneous Open Data APIs** + +### 58. [Public APIs Directory](https://public-apis.xyz/) +- **Description**: A collection of free, public APIs across multiple domains, including OSINT, government, and social media. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +### 59. [OpenDataSoft](https://data.opendatasoft.com/) +- **Description**: A platform for accessing open datasets from a variety of domains, such as transportation, government, and environment. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Varies by dataset + +--- + +### 60. [DorkBot API](https://www.dorkbot.com/) +- **Description**: A tool that queries databases of known compromised sites, + + also known as "dorking," used for penetration testing. +- **Free Tier**: Yes +- **Authentication**: None +- **Usage Limit**: Unlimited + +--- \ No newline at end of file From 2e53028dd6625fe105320383ebd6290ee254cb6f Mon Sep 17 00:00:00 2001 From: RameezIbrahimOfficial Date: Tue, 15 Oct 2024 12:05:23 +0530 Subject: [PATCH 2/2] Removed Comments --- APIs.md | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/APIs.md b/APIs.md index 1b60ba3..7f1d335 100644 --- a/APIs.md +++ b/APIs.md @@ -226,10 +226,6 @@ --- -Absolutely! Here’s an extended list of **more free and open-source OSINT APIs**, bringing the total to **50+ APIs**. These APIs cover a broader range of open-source intelligence sources, including additional security, social media, geolocation, and other investigative tools. - ---- - ### 33. [ip-api](http://ip-api.com/) - **Description**: A fast and accurate IP geolocation API for location and organization data based on IP address. - **Free Tier**: Yes @@ -256,7 +252,7 @@ Absolutely! Here’s an extended list of **more free and open-source OSINT APIs* --- -## **Email & Domain Intelligence (Continued)** +## **Email & Domain Intelligence** ### 37. [MxToolbox](https://mxtoolbox.com/api.aspx) - **Description**: Provides DNS, SMTP, and blacklist checking tools for domain and email server analysis. @@ -284,7 +280,7 @@ Absolutely! Here’s an extended list of **more free and open-source OSINT APIs* --- -## **Social Media & Web Scraping (Continued)** +## **Social Media & Web Scraping** ### 41. [Social Search API](https://social-search.com/) - **Description**: A search engine for discovering social media profiles associated with a given username or email address. @@ -324,7 +320,7 @@ Absolutely! Here’s an extended list of **more free and open-source OSINT APIs* --- -## **Cybersecurity & Threat Intelligence APIs (Continued)** +## **Cybersecurity & Threat Intelligence APIs** ### 47. [AbuseIPDB API](https://www.abuseipdb.com/) - **Description**: A public repository of reports on IP addresses that have been involved in malicious activity. @@ -380,7 +376,7 @@ Absolutely! Here’s an extended list of **more free and open-source OSINT APIs* --- -## **DNS & Network Analysis (Continued)** +## **DNS & Network Analysis** ### 55. [DNSQuery API](https://www.dnsquery.io/) - **Description**: Allows you to perform DNS queries and retrieve records like A, MX, NS, and TXT.