javanxd / ya-pihole-list

A bunch of useful pi-hole adlists and a auto updater.

Hi there 👋

ya-pihole-list's Issues

Remove dbl.oisd.nl from your master list

I don't know if this is a temporary issue, or permanent, but there appears to be an issue with dbl.oisd.nl. May want to remove that from your master list.

Adlist:

# Ads, Mobile Ads, Phishing, Malvertising, Malware, Spyware, Ransomware, CryptoJacking and some Telemetry, Analytics, Tracking
https://dbl.oisd.nl/

Log:

  [i] Target: https://dbl.oisd.nl/
  [✗] Status: https://dbl.oisd.nl/ (410)
  [✗] List download failed: no cached list available

Updated Pihole adlist format

The pihole adlist format now expects 10 columns and the INSERT breaks on the NULL values that get autofilled.
Line 73 in adlists-updater.sh needs to be updated to read:
echo "${rowid},\"${domain}\",1,${timestamp},${timestamp},\"Added by Updater\",,0,0,0" >> "${tmpFile}"

Malware Domains no longer in operation

https://mirror1.malwaredomains.com/files/justdomains 404s. The project is no longer in operation.

According to https://riskanalytics.com/community/:

We have merged AutoShun and Malware Domains, and have integrated those technologies into ShadowNet. Through ShadowNet we are able to identify the latest threat intel gathered from around the globe, share learnings and best practices from our network of clients and partners and continually build those learnings into our products, resulting in the best, most powerful threat intel for your organization.

As of November 30th, 2020, both AutoShun and Malware Domains feeds will be shut off, and subscribers will no longer receive intel from those sources.
We will continue to make a free, non-commercial version of our threat intel, ShadowNet, while the enhanced full-threat intel feed will be licensed to our commercial partners.

It should probably be removed from the list.

updatescript updates whole system

Hi,

While having an up to date system is important, I really don't think it should be this scripts responsibility to perform the updates. Could you remove those lines, or at least make it overridable by setting some environment variable?

Add check for sqlite3 program [Feature Request]

Was running the script, but it wasn't updating the adlist. Starting debugging the logs/script and realized it was calling sqlite3 program, which is not installed on the system. Maybe add a check near the beginning for sqlite3 and error out if it is missing? It's not required to run Pi-hole.

Insert Query Error

There appears to be an issue with the insert query:

Info: Download latest adlists list to /home/pi/adlists.list.updater
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1620 100 1620 0 0 189 0 0:00:08 0:00:08 --:--:-- 363
Info: Adding content of /home/pi/adlists.list.updater into gravity database
Warning: Some warnings in table adlist in database /etc/pihole/gravity.db\n /ho me/pi/adlists.list.updater.tmp:1: expected 6 columns but found 5 - filling the r est with NULL
/home/pi/adlists.list.updater.tmp:1: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:2: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:2: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:3: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:3: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:4: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:4: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:5: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:5: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:6: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:6: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:7: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:7: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:8: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:8: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:9: expected 6 columns but found 5 - filling th e rest with NULL
/home/pi/adlists.list.updater.tmp:9: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:10: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:10: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:11: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:11: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:12: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:12: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:13: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:13: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:14: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:14: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:15: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:15: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:16: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:16: INSERT failed: datatype mismatch
/home/pi/adlists.list.updater.tmp:17: expected 6 columns but found 5 - filling t he rest with NULL
/home/pi/adlists.list.updater.tmp:17: INSERT failed: datatype mismatch
Info: Update Gravity because of the changes to the gravity.db
[i] Neutrino emissions detected...

Changes to adlists.list.updater are ignored

I added a new blocklist to the file adlists.list.updater:

#DoH
https://raw.githubusercontent.com/Sekhan/TheGreatWall/master/TheGreatWall.txt

But when I run adlists-updater.sh I do not see that list appear as a target in the script output. The last target that appears is the last line in your adlists.list.updater file:
Target: https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/dnscrypt/spy.txt

What do I need to do to make the script recognize my edits to the adlists.list.updater file?

Add YouTube ad-blocking

Hi,

Thanks for your addlist + updater.
Fantastic job.
Now my question, is it possible to add a YouTube ad-blocking list?

Thanks in advanced.
Jan.