I-TECH Changes

• Removed hosts that do not have a license or are meant for non-commercial purposes only.
• Added audio-streaming and video-streaming extensions

Run the following command to build the hosts file:

python3 updateHostsFile.py -a -e gambling porn fakenews audio-streaming video-streaming social

Push the change to GitHub

git push

Unified hosts file with base extensions

This repository consolidates several reputable hosts files, and merges them into a unified hosts file with duplicates removed. A variety of tailored hosts files are provided.

• Last updated: November 10 2017.
• Here's the raw hosts file with base extensions containing 42,839 entries.

List of all hosts file variants

The Non GitHub mirror is the link to use for some hosts file managers like Hostsman for Windows that don't work with Github download links.

Host file recipe	Readme	Raw hosts	Unique domains	Non Github mirror
Unified hosts = (adware + malware)	Readme	link	42,839	link
Unified hosts + fakenews	Readme	link	43,512	link
Unified hosts + gambling	Readme	link	44,369	link
Unified hosts + porn	Readme	link	50,703	link
Unified hosts + social	Readme	link	43,072	link
Unified hosts + fakenews + gambling	Readme	link	45,042	link
Unified hosts + fakenews + porn	Readme	link	51,376	link
Unified hosts + fakenews + social	Readme	link	43,745	link
Unified hosts + gambling + porn	Readme	link	52,233	link
Unified hosts + gambling + social	Readme	link	44,602	link
Unified hosts + porn + social	Readme	link	50,936	link
Unified hosts + fakenews + gambling + porn	Readme	link	52,906	link
Unified hosts + fakenews + gambling + social	Readme	link	45,275	link
Unified hosts + fakenews + porn + social	Readme	link	51,609	link
Unified hosts + gambling + porn + social	Readme	link	52,466	link
Unified hosts + fakenews + gambling + porn + social	Readme	link	53,139	link

Expectation: These unified hosts files should serve all devices, regardless of OS.

Sources of hosts data unified in this variant

Updated hosts files from the following locations are always unified and included:

Host file source	Description	Home page	Raw hosts	Update frequency	License
AdAway	AdAway is an open source ad blocker for Android using the hosts file.	link	raw	occasionally	CC BY 3.0
add.2o7Net	2o7Net tracking sites based on http://www.hostsfile.org/hosts.html content.	link	raw	occasionally	GPLv3+
add.Dead	Dead sites based on http://www.hostsfile.org/hosts.html content.	link	raw	occasionally	GPLv3+
add.Risk	Risk content sites based on http://www.hostsfile.org/hosts.html content.	link	raw	occasionally	GPLv3+
add.Spam	Spam sites based on http://www.hostsfile.org/hosts.html content.	link	raw	occasionally	GPLv3+
Mitchell Krog's - Badd Boyz Hosts	Sketchy domains and Bad Referrers from my Nginx and Apache Bad Bot and Spam Referrer Blockers	link	raw	weekly	'non-commercial with attribution'
KADhosts	Fraud/adware/scam websites.	link	raw	frequently	GPLv3
Malware Domain List	Malware Domain List is a non-commercial community project.	link	raw	weekly	'can be used for free by anyone'
MVPS hosts file	The purpose of this site is to provide the user with a high quality custom HOSTS file.	link	raw	monthly	CC BY-NC-SA 4.0
Dan Pollock - someonewhocares.org	How to make the internet not suck (as much).	link	raw	frequently	non-commercial with attribution
SpotifyAds	Spotify ads sources sites based on https://github.com/Xeroday/Spotify-Ad-Blocker content.	link	raw	occasionally	MIT
Steven Black's ad-hoc list	Additional sketch domains as I come across them.	link	raw	occasionally	MIT
tyzbit	Microsoft tracking domains. A fork of this repo providing additional data.	link	raw	rarely
UncheckyAds	Windows installers ads sources sites based on https://unchecky.com/ content.	link	raw	occasionally
yoyo.org	Blocking with ad server and tracking server hostnames.	link	raw	frequently

Extensions

The unified hosts file is extensible. You manage extensions by curating the extensions/ folder tree. See the fakenews, social, gambling, and porn extension folders.

Generate your own unified hosts file

To run unit tests, in the top level directory, just run:

python testUpdateHostsFile.py

Note if you are using Python 2, you must first install the mock library:

pip install mock

The updateHostsFile.py script, which is Python 2.7 and Python 3-compatible, will generate a unified hosts file based on the sources in the local data/ subfolder. The script will prompt you whether it should fetch updated versions (from locations defined by the update.json text file in each source's folder). Otherwise, it will use the hosts file that's already there.

Usage

Using Python 3:

python3 updateHostsFile.py [--auto] [--replace] [--ip nnn.nnn.nnn.nnn] [--extensions ext1 ext2 ext3]

Using Python 2.7:

python updateHostsFile.py [--auto] [--replace] [--ip nnn.nnn.nnn.nnn] [--extensions ext1 ext2 ext3]

Command line options:

--help, or -h: display help.

--auto, or -a: run the script without prompting. When --auto is invoked,

• Hosts data sources, including extensions, are updated.
• No extensions are included by default. Use the --extensions or -e flag to include any you want.
• Your active hosts file is not replaced unless you include the --replace flag.

--backup, or -b: Make a backup of existing hosts file(s) as you generate over them.

--extensions <ext1> <ext2> <ext3>, or -e <ext1> <ext2> <ext3>: the names of subfolders below the extensions folder containing additional category-specific hosts files to include in the amalgamation. Example: --extensions porn or -e social porn.

--flush-dns-cache, or -f: skip the prompt for flushing the DNS cache.
Only active when --replace is also active.

--ip nnn.nnn.nnn.nnn, or -i nnn.nnn.nnn.nnn: the IP address to use as the target. Default is 0.0.0.0.

--keepdomaincomments, or -k: false (default) or true, keep the comments that appear on the same line as domains. The default is false since some router-based implementations can't handle comments in-line with hosts.

--skipstatichosts, or -s: false (default) or true,

--noupdate, or -n: skip fetching updates from hosts data sources.

--output <subfolder>, or -o <subfolder>: place the generated source file in a subfolder. If the subfolder does not exist, it will be created.

--replace, or -r: trigger replacing your active hosts

--skipstatichosts, or -s: false (default) or true, omit the standard section at the top, containing lines like 127.0.0.1 localhost. This is useful for configuring proximate DNS services on the local network.

How do I control which sources are unified?

Add one or more additional sources, each in a subfolder of the data/ folder, and specify the url key in its update.json file.

Add one or more optional extensions, which originate from subfolders of the extensions/ folder. Again the url in update.json controls where this extension finds its updates.

Create an optional blacklist file. The contents of this file (containing a listing of additional domains in hosts file format) are appended to the unified hosts file during the update process. A sample blacklist is included, and may be modified as you desire.

• NOTE: The blacklist is not tracked by git, so any changes you make won't be overridden when you git pull this repo from origin in the future.

How do I include my own custom domain mappings?

If you have custom hosts records, place them in file myhosts. The contents of this file are prepended to the unified hosts file during the update process.

The myhosts file is not tracked by git, so any changes you make won't be overridden when you git pull this repo from origin in the future.

How do I prevent domains from being included?

The domains you list in the whitelist file are excluded from the final hosts file.

The whitelist uses partial matching. Therefore if you whitelist google-analytics.com, that domain and all its subdomains won't be merged into the final hosts file.

The whitelist is not tracked by git, so any changes you make won't be overridden when you git pull this repo from origin in the future.

What is a hosts file?

A hosts file, named hosts (with no file extension), is a plain-text file used by all operating systems to map hostnames to IP addresses.

In most operating systems, the hosts file is preferential to DNS. Therefore if a domain name is resolved by the hosts file, the request never leaves your computer.

Having a smart hosts file goes a long way towards blocking malware, adware, and other irritants.

For example, to nullify requests to some doubleclick.net servers, adding these lines to your hosts file will do it:

# block doubleClick's servers
0.0.0.0 ad.ae.doubleclick.net
0.0.0.0 ad.ar.doubleclick.net
0.0.0.0 ad.at.doubleclick.net
0.0.0.0 ad.au.doubleclick.net
0.0.0.0 ad.be.doubleclick.net
# etc...

We recommend using 0.0.0.0 instead of 127.0.0.1

Traditionally most host files use 127.0.0.1, the loopback address, to establish an IP connection to the local machine.

We prefer to use 0.0.0.0, which is defined as a non-routable meta-address used to designate an invalid, unknown, or non applicable target.

Using 0.0.0.0 is empirically faster, possibly because there's no wait for a timeout resolution. It also does not interfere with a web server that may be running on the local PC.

Why not use just 0 instead of 0.0.0.0?

We tried that. Using 0 doesn't work universally.

Location of your hosts file

To modify your current hosts file, look for it in the following places and modify it with a text editor.

Mac OS X, iOS, Android, Linux: /etc/hosts folder.

Windows: %SystemRoot%\system32\drivers\etc\hosts folder.

Updating hosts file on Windows

On Linux and Mac OS X, you can simply run the Python script, but on Windows, more work is required due to compatibility issues in implementing some of the functionality for Windows. It is preferable to run the batch file as follows:

updateHostsWindows.bat

This file MUST be run in command prompt with administrator privileges in the repository directory. In addition to updating the hosts file, it can also replace the existing hosts file, and reload the DNS cache. It goes without saying that in order for this to work, you must be connected to the internet.

To open a command prompt as administrator in the repository's directory, do the following:

Windows XP: Start -> Run -> cmd

Windows Vista, 7: Start Button -> type cmd -> right-click Command Prompt -> "Run as Administrator"

Windows 8: Start -> Swipe Up -> All Apps -> Windows System -> right-click Command Prompt -> "Run as Administrator"

Windows 10: Start Button -> type cmd -> right-click Command Prompt -> "Run as Administrator"

You can also refer to the "Third-Party Hosts Managers" section for further recommended solutions from third parties.

Reloading hosts file

Your operating system will cache DNS lookups. You can either reboot or run the following commands to manually flush your DNS cache once the new hosts file is in place.

Windows

Open a command prompt with administrator privileges and run this command:

ipconfig /flushdns

If you want to use a huge hosts file by merging hphosts (NOT INCLUDED HERE) you need to DISABLE and STOP Dnscache service before you replace hosts file in Windows Systems. You have been warned.

Before flushing the DNS cache, open a command prompt with administrator privileges and run this command:

sc config "Dnscache" start= disabled
sc stop "Dnscache"

Linux

Open a Terminal and run with root privileges:

Debian/Ubuntu sudo /etc/rc.d/init.d/nscd restart

Linux with systemd: sudo systemctl restart network.service

Fedora Linux: sudo systemctl restart NetworkManager.service

Arch Linux/Manjaro with Network Manager: sudo systemctl restart NetworkManager.service

Arch Linux/Manjaro with Wicd: sudo systemctl restart wicd.service

Others: Consult this wikipedia article.

Mac OS X

Open a Terminal and run:

sudo dscacheutil -flushcache;sudo killall -HUP mDNSResponder

Goals of this unified hosts file

The goals of this repo are to:

1. automatically combine high-quality lists of hosts,

2. provide easy extensions,

3. de-dupe the resultant combined list,

4. and keep the resultant file reasonably sized.

A high-quality source is defined here as one that is actively curated. A hosts source should be frequently updated by its maintainers with both additions and removals. The larger the hosts file, the higher the level of curation is expected.

For example, the (huge) hosts file from hosts-file.net is not included here because it is very large (780,000+ entries) and doesn't currently display a corresponding high level of curation activity.

It is expected that this unified hosts file will serve both desktop and mobile devices under a variety of operating systems.

Third-Party Hosts Managers

• Unified Hosts AutoUpdate (for Windows): The Unified Hosts AutUpdate package is purpose-built for this unified hosts project as well as in active development by community members. It's sophisticated enough to allow any novice the ability to install and uninstall the blacklist of their choosing to their local hosts file and keep it automatically up to date, while also being minimal enough to be able to be easily placed in a shared network location and deployed across an organization via group policies. And since it is in active development by community members, your bug reports, feature requests, and other feedback are most welcome.

Interesting Applications

• Block ads and malware via local BIND9 DNS server (for Debian, Raspbian & Ubuntu): Set up a local DNS server with a /etc/bind/named.conf.blocked file, sourced from here.

• Block ads, malware, and deploy parental controls via local DualServer DNS/DHCP server (for BSD, Windows & Linux): Set up a blacklist for everyone on your network using the power of the unified hosts reformatted for DualServer. And if you're on Windows, this project also maintains an update script to make updating DualServer's blacklist even easier.

• Blocking ads and malwares with unbound - Unbound is a validating, recursive, and caching DNS resolver.