hectorm / hblock

Improve your security and privacy by blocking ads, tracking and malware domains.

Improve your security and privacy by blocking ads, tracking and malware domains.

What is this for?

hBlock is a POSIX-compliant shell script that gets a list of domains that serve ads, tracking scripts and malware from multiple sources and creates a hosts file, among other formats, that prevents your system from connecting to them.

⚠️ hBlock by default replaces the hosts file of your system, consider making a backup first if you have entries you want to preserve.

Installation

hBlock is available in various package managers. Please check the PACKAGES.md file for an up-to-date list.

The latest available version can also be installed manually by running the following commands:

curl -o /tmp/hblock 'https://raw.githubusercontent.com/hectorm/hblock/v3.4.5/hblock' \
  && echo '2db6f52a14a3b87ff86b0fde3583f1a635ff580d26b36f8a103997a614314c6f  /tmp/hblock' | shasum -c \
  && sudo mv /tmp/hblock /usr/local/bin/hblock \
  && sudo chown 0:0 /usr/local/bin/hblock \
  && sudo chmod 755 /usr/local/bin/hblock

Additionally, a systemd timer can be set to regularly update the hosts file for new additions.

Usage

The default behavior of hBlock can be adjusted with multiple options. Use the --help option or check the hblock.1.md file for the full list.

Nightly builds

Nightly builds of the hosts file, among other formats, can be found on the hBlock website.

Temporarily disable hBlock

Sometimes you may need to temporarily disable hBlock, a quick option is to generate a hosts file without any blocked domains by running the following command:

hblock -S none -D none

I found a false positive, what should I do?

It's possible that sometimes one of the hBlock sources includes a domain that shouldn't be blocked, in those cases the best way to proceed is to temporarily add said domain to the allowlist and report the problem to the original blocklist author. This way it will also be automatically removed from hBlock within 24h.

To find which blocklist is behind a false positive you can clone the hMirror project and search for that domain. Although please note that not all hMirror blocklists are used by default in hBlock, an updated list of the sources used can be found in the SOURCES.md file.

hblock's Issues

Logo Proposal

Good day Sir @hectorm I am a graphic designer and i am interested in designing a logo for your good project. I will be doing it as a gift for free. I just need your permission first before i begin my design. Hoping for your positive feedback. Thanks and best regards! - Tobaloidee

Improvement : is there a quick way to turn hblock off and on?

Is there the option to tell
hblock on
hblock off

I have an issue when downloading from sourceforge.
I want to quickly test if it is not hblock blocking the server.

Will the original hosts file be put back?

feature request search function

any plans to add search functionality inside lists?
$ grep -i "analytics.google.com" /etc/hosts
$ hblock -W analytics.google.com

Youtube ads not blocked properly

I know it is not an issue of your program directly, but I am writing here to let you know, maybe there is some host file that handle better youtube ads like the default ad-blockers from opera/vivaldi

HowTo add hosts in correct way?

Hello, how should I add hosts entries in a correct way to /etc/hosts, so hblock want remove entries while running/generating that file. I tried by whitelist but thats not an working solution.
-- If I want declare/assign IP to host name (which host file is for) it will be removed from /etc/host in next hblock run/update. I cant whitelist it, as host declaration need to be in that exact file "/etc/host" as its where other programs looks for... and if I edit hblock script, it will be updated with Os update (arcolinux)... So, how to approach ?

Android - zip file invalid

After I installed curl and zip, I get this.

http://i.imgur.com/91JZSIn.png

Any ideas?

More sources

The script should use more sources, for example uBlock Origin also comes with EasyPrivacy and in my opinion also Fanboy’s Enhanced Tracking List‎ should be included
Also, I'd like to see an optional version with Fanboy’s Annoyance List and Anti-ThirdpartySocial list to get rid of social media plugins.

My other suggestions:

• Spam404‎: https://spam404bl.com/spam404scamlist.txt
• Basic tracking list by Disconnect‎: https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
• Malvertising filter list by Disconnect‎: https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
• Adblock Warning Removal List‎: https://easylist-downloads.adblockplus.org/antiadblockfilters.txt
• And maybe 'EU: Prebake - Filter Obtrusive Cookie Notices' for the users in EU: https://github.com/liamja/Prebake

hitelist

CoinBlockerLists moved to GitLab

https://gitlab.com/ZeroDot1/CoinBlockerLists

apprep.smartscreen.microsoft.com

.smartscreen.microsoft.com is used for Windows Defender Smartscreen which is a security feature and not telemetry.

Please unblock it.

https://docs.microsoft.com/en-us/windows/privacy/windows-endpoints-1903-non-enterprise-editions

Curl error for hblock v215 on kali linux

Hi Mate,
I have been using 2.1.3 and the previous builds as well. However, today when I updated to 2.1.5, since then I am facing the following error: curl: (3) URL using bad/illegal format or missing URL. Error snap below. My system is:

5.4.0-kali3-amd64 Debian 5.4.13-1kali1 (2020-01-20) x86_64 GNU/Linux

Please let me know if I am missing anything or is this actually a bug?

/usr/local/bin/hblock

!/bin/sh
# Version:    2.1.5

Danke mate. :)

distinguish "vortex-win.data.microsoft.com"

While v10c.vortex-win.data.microsoft.com is for Diagnostics data Telemetry, the domain de.vortex-win.data.microsoft.com is for Microsoft Defender Advanced Threat Protection - while the prefix (in this case ".de") is country specific.

So please remove

• vortex-win.data.microsoft.com

from your list.

Source: https://docs.microsoft.com/en-us/windows/privacy/configure-windows-diagnostic-data-in-your-organization#endpoints

Possible false positive: addevent.com www.addevent.com

I believe addevent.com www.addevent.com may be false positives.
It is a service for sending calendar events to people, and they can accept the invite from Google Calendar/iCal etc.

I attended a virtual conference and this service is how they were sending out invites. I dont believe it is spyware or adware.

hblock via Cmder terminates on '| sort -u'

hblock terminates at the "Sort entries" command using Cmder on Windows (10).

λ sh hblock -O "%SystemRoot%\System32\drivers\etc\hosts"
--- 8< ---
   - Sort entries
-uThe operation completed successfully.

Of course, nothing actually completed successfully. (See below for sort behavoir.)

On line 217, changing | sort -u to | sort | uniq allows hblock to actually complete successfully. :)

λ sh hblock -O "%SystemRoot%\System32\drivers\etc\hosts"
--- 8< ---
   - Sort entries
   - Add new destination
 + Generating hosts file...
 + 50218 hosts added!

Making the same edit on a Linux VM (CentOS-7.2.1511) showed identical write and sort behavior between | sort -u and | sort | uniq.

Comparing the output files of each with diff only shows a difference in commented time stamps.

xD

.
edit

Sort as provided by Cmder is a Windows command, hence the incompatibility.
https://technet.microsoft.com/en-us/library/bb491004.aspx

λ cat sortme
XX          
bb          
cc          
aa          
bb          
cc          
xX          
cc          
xx          

λ cat sortme | sort
aa                 
bb                 
bb                 
cc                 
cc                 
cc                 
xx                 
xX                 
XX                 

λ cat sortme | sort -u
-uThe operation completed successfully.
cat: write error: No space left on device

λ cat sortme | uniq
XX                 
bb                 
cc                 
aa                 
bb                 
cc                 
xX                 
cc                 
xx                 

λ cat sortme | sort | uniq
aa
bb
cc
xx
xX
XX

Just to show there is really space left on device :) sort in Cmder is being silly.

λ df -h | head -1 && df -h | tail -1
Filesystem                                               Size  Used Avail Use% Mounted on
N:/Users/Username/Downloads/cmder/vendor/git-for-windows  932G  294G  638G  32% /

Solus

Tried in solus
Hblock edits the hosts file and adds the blocklists
But doesn't create the hblock.d folder thus no whitelist or blacklist file
I manually created and tried adding one domain from blocklist to whitelist
Then refreshed the hosts file
Although in terminal it shows 1less domain is added, the whitelist domain isn't removed from blocklist
Hope to see a working version soon 😖

Please add antipopads list to source.

https://github.com/Yhonay/antipopads

[Help] Can WHITELIST be fetched via url?

I want to fetch whitelist from url, and use with hBlock. Is there any way to do that?

Thanks for this great script, btw! :)

forms.gle - False Positive

This domain is just Google Forms.

Windows version

This script worked perfectly on Linux and as far as I'm aware, no one has developed a similar solution for Windows. Windows only has a freeware application called HostsMan that performs poorly.

StopForumSpam blocks YouTube.com and other legit domains

Hello,
hBlock list is blocking youtube.com as it exists in the StopForumSpam list.
It seems to have a lot of false-positives, it blocks some international yahoo domains, and the official nyc.gov website and firefox.com. Judging by a quick glance.
It seems like this list is troublesome and it would be better to remove it as a whole.

Whitelist and blacklist from files

Maybe you could add option to load whitelisted and blacklisted domains from files.
This way we would be able to share them between many computers.

For default it could be $HOME/.hblock_whitelist and $HOME/.hblock_blacklist.
But I'm not sure how to make it work with systemd service files.

Making a time-stamped backup of /etc/hosts before overwriting it.

The title pretty sums it all, how about making a time-stamped backup of the hosts file before overwriting it?

False Positive: reddit.app.link

Domain: reddit.app.link
This domain is used for opening reddit.com browser links in the mobile app.

false positive: ssl.bblck.me

Hello,
You are blocking ssl.bblck.me which is hosting another blocklist
https://ssl.bblck.me/blacklists/hosts-file.txt
https://ssl.bblck.me/blacklists/domain-list.txt
Thanks

No hash available for specified version

I know the readme provides a hash in the install section script but it would be good to host a copy of this at the same place as the script (e.g. https://raw.githubusercontent.com/hectorm/hblock/v2.1.4/hblock.sha512 or something like that). Lack of a hash available through a URL makes this project hard to use in automated scripts (as someone has to check the readme when there's an update instead of being able to bump the version number in the URL and have it find the new hash also). I'm trying to write an Ansible role to install and configure this currently and this is the issue I have ran into, most people wouldn't deploy anything on their server without at least a hash check so this would be a really good feature.

Whitelist common false-positives

It might be a good idea to add a list of commonly whitelisted domains such as this to lower the probability of people having problems.

Found the list at https://firebog.net/ under "Whitelisting Suggestions".

Whitelist

Is the whitelist consistent or is it a once-per-command flag?
If there isn't a way to make the whitelist permanent by default, is there a way to change it without having to go into the script and add domains manually every time?

anyone has issue with Android Emulator

I am having issue with Android emulator, basically my emulator don't have any access to internet.

thanks!

is there any way to unblock everything that I blocked ?

is there any way to unblock everything that I blocked ? I tried the --help but it did'nt show anything? would be nice to add it

improvement: latest version

Please consider my idea to add 'latest version' release for semver compatibility, so that not to add versions manually.
eg.

curl -Ls -o /dev/null -w %{url_effective} https://github.com/docker/compose/releases/latest

returns latest version v<sem_version> automatically

curl -fsSLI -o /dev/null -w %{url_effective} https://github.com/hectorm/hblock/releases/latest

question: does hblock remove duplicates?

does Hblock remove duplicates?

does hblock check if remote hosts file size has changed since previous update?

does hblock check to see if remote hosts files on remote server have changed before downloading, thus preventing the download of duplicates and speeding up processing? if not, please consider this a feature request.

One must maintain copies of the 1. files or 2. hashes on the system and compare the hash or whatever, diversion.ch does this effectively.

this keeps popping up when I try to watch a video hosted on openload

Please add these, I don't think I go through a day without them popping up
www.simple2date.com
http://34.250.229.112/

financialprovider-logos.platform.intuit.com

False positive: This domain is used for showing logos of financial providers on the Mint platform. With this domain blocked, the logo doesn't show up.

Domains for third-party requests

Hi there, I noticed that your blocklists contain domains from rules like this: ||example.com^$third-party.
Why block them?

Geoffrey Frogeye's block list

could you add that list?

https://hostfiles.frogeye.fr/

Version 2.1.3

I just dropped from 150.000 blocked domains to 116.000 domains on ArcoLinux

I suppose you know why that is

Support from hostsVN - advertising block of Vietnam

Hello!

I am from Vietnam, I have a project hostsVN for the purpose of blocking ads and tracking from advertisers from Vietnam. Wishing to support ad blocking projects that support blocking domains in Vietnam, so I created this issue hoping you will merge with hostsVN. The project has been maintained for more than 1 year and continuously updates new domain names.

Github: https://github.com/bigdargon/hostsVN/

Some sources use hostsVN:

• https://abpvn.com/android/
• https://github.com/AdguardTeam/AdGuardSDNSFilter/blob/2437fd7f0b9586ef53a417dd63a59deac43452b0/Filters/filter.template#L92
• https://github.com/StevenBlack/hosts/releases/tag/2.5.0
• in nextdns lists

The hosts file I categorized only includes domain name blocking ads and tracking in Vietnam. I hope you just need to merge this file. Link https://raw.githubusercontent.com/bigdargon/hostsVN/master/option/hosts-VN

Thanks you!

Download List link not available

The link:
https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt
is not working - not available...
I commented it out but am wondering if there is a alternative/replacement link..?

socialize.us1.gigya.com

Hi, you are blocking the following domains:

socialize.us1.gigya.com

These domains break logging into iRobot app on Android, and starting iRoomba vacuum from Google Home

Installation error: curl: (6) Could not resolve host: raw.githubusercontent.com

~$ curl -o /tmp/hblock 'https://raw.githubusercontent.com/hectorm/hblock/v2.1.6/hblock' \

&& echo '9e22c32c8ae4d93df18a41f08d31e8668ef4342fda82ca91ec4aa47f718fdadc /tmp/hblock' | shasum -c
&& sudo mv /tmp/hblock /usr/local/bin/hblock
&& sudo chown root:root /usr/local/bin/hblock
&& sudo chmod 755 /usr/local/bin/hblock
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:14 --:--:-- 0curl: (6) Could not resolve host: raw.githubusercontent.com

I tried doing it 4-5 times, different connections, restart, clean installation. Nothing works..

Problem making package

I'm maintaining your package for the Arch User Repository and have recently come across a problem that was successfully fixed with the help of the community, but a member has advised me to tell you about problems in your Makefile. I am by no means an expert on this (and don't fully understand what they suggest), so I will point you to this forum topic: https://bbs.archlinux.org/viewtopic.php?id=244994

telecommand.telemetry.microsoft.com

telecommand.telemetry.microsoft.com is used by Windows Error Reporting and isn't telemetry.

So please unblock it

https://docs.microsoft.com/en-us/windows/privacy/windows-endpoints-1903-non-enterprise-editions

edge.activity.windows.com

edge.activity.windows.com
Edge browser uses the activity.windows.com domain to sync browser data
It should not be disabled

Custom blacklisting sources

Hello, thank you for nice script to make our lives better.

Would be nice to have possibility to specify custom blacklisting sources that would be used instead of build in.
I would like to use https://github.com/StevenBlack/hosts that already have many concatenated by type and ready to use.

Req: Option, multiple hosts per line;

An option to use a select number of hosts per line would be nice, I use 20 hosts per line on most machines, this reduces hosts file size by 25%, and helps hosts file load times and responsiveness further as well. It shrunk my file from 45 megabytes to 35.

exampe:

0.0.0.0 google.com google.ca google.co.uk microsoft.com twitter.com facebook.com

s.youtube.com

This domains has been breaking Youtube recommendations and saving items to history

Blocked google icons

When I use this filter, some of the icons are blocked in google services, namely google calendar / meet. I did not check the rest.
https://raw.githubusercontent.com/hectorm/hmirror/master/data/lightswitch05-ads-and-tracking/list.txt

Can't download the zip, Windows 10 detects virus

The windows version can't be downloaded on Windows 10, because Windows complains about virusses, probably a false positive.

lnkd.in non malicious

Hi
For reference, the domain lnkd.in is not malicious and is just linkedin's URL shortener service
I have reported this to phishing army author and only opening this issue for rerefence
If you do not need to whitelist on your end please feel free to close this issue