Need an update mechanism about pi-hole

[dschaper]

That's part of what I'm working on with the .deb package. With a pihole repository, then an install would just be another apt-get and when updates are ready the usual apt-get update/upgrade should pull in the updates. Debian packages are just a little more involved in creating, plus we'd need to make a repository (repo.pihole.net?) to host. Any help from other DD's or people with packaging experience would be great, I'm used to supporting Arch. But it's on the list! lol...

[dschaper]

That being said, I don't remember if I asked yesterday, so forgive me if I did, but can you cut a 2.1 release and put up a tarball? It would help with a baseline for the package and reduce the patch count.

[jacobsalmela]

You did ask, but I haven't actually made a tarball before; I mean I know what tools to use, but I'm not quite sure how to package it up. Do I just archive all the necessary files and then use a script to put them in the right place? I'm a bit of n00b in this department.

[dschaper]

No problem, when you made your 2.0 release it created the zip and the tar.gz for you. You just need to make a new release and tag it. https://help.github.com/articles/creating-releases/

I just tested it, and the tarballs are auto-created. Just set the version to 'v2.1' or something like that, the release number prepended with a v.

[jacobsalmela]

Done.

[dschaper]

Awesome, thanks!

[DonLexos]

Following this, what would be the best way to update from say a 1 month old installation? Complete reinstall or is there a smart method (since functions have moved, and maybe even helper files have been added)

[dschaper]

Well, you wouldn't lose anything by doing a fresh install, may hold off a few days though, there are some changes coming. I'm working on a .deb that would eliminate the upgrade issues, but we need to set a good baseline to get a good .deb that isn't going to change locations with each new release.

[DonLexos]

No rush, it all works fine, but I've seen many new commits and wondered what the best path forward would be for "an early adopter" (although that is a bit overstating it). There have been many changes recently so I figured I'd give it a shot. Not sure f.e. if the list selection is still in the script or in a configuration file (which would eliminate any issues on an update in the future, provided there is a system like apt-get update).

[dschaper]

The list that comes as a default with gravity.sh hasn't changed much, so that's why I said you wouldn't lose much doing a fresh install, you'd basically get what you already have. There have been a lot of bug fixes and expansions to support other platforms and a lot of skeleton framework to set up some pretty cool changes coming up, so it's kind of a transition period. Functionally there isn't much different right now, but there will be some changes soon.

Jacob is doing a lot with the web interface that will allow you to customize your blocking in a very user friendly way. Gravity.sh is still the workhorse and all that has changed there is internal structure. There may be a little speed up, but I'm not sure.

So if it's working, hold on a bit and see what's in store...

[DonLexos]

HI,

I jumped and did a fresh installation earlier today (with same IP setup etc) and all works fine, however it it a bit unclear to me. Assume there is a new version, can I simply update gravity.sh or are the more files that need to be changed? I am not very familiar with the github system but am able to SSH into the pi and change it on that level (and restart services if required).

Thanks!

[jacobsalmela]

It's possible new files could be added or modified (see #174, which was just merged). So in order to get these files you can basically reinstall using curl -L install.pi-hole.net | bash but it's not a true "update" as there isn't logic for checking what files exist or how they compare to the new ones. This would basically just re-install everything.

[PromoFaux]

I have a cunning plan...

How about adding a version number txt file to the repo?

when gravity.sh runs (which it does every night) It checks the currently downloaded version file against the one in the repo. If the one in the repo is newer then run install.sh (which can then be expanded to run silently if pi-hole is already installed)

[jacobsalmela]

But doesn't it seem a bit excessive to try to apt-get update and to try and install everything again?

I was hoping for something more along the lines of curl -L update.pi-hole.net | bash. I have been thinking about how updateDashboard.sh use git to keep things synchronized. Something like that would be cool. Then it could just be a cron job.

[PromoFaux]

Could updatedashboard be integrated into the existing install.sh I wonder?

apt-get update could easily be skipped if install.sh detects it's an existing installation. I don't mind tackling it tomorrow, if you like? I know my way around the install script now!

Then it could just be a cron job.

No need if you defer the work to gravity.sh, as that is already on a cron job...

[jacobsalmela]

But the latest Web interface is always downloaded with the installer; updateDashboard.sh just keeps it up to date with git. I'm not sure I understand what you mean by integrating it into the installer.

[PromoFaux]

Basically, the part in the installer that downloads master.zip can be replaced with the same logic from updateDashboard.sh

if ! is_repo; then
    make_repo
fi
update_repo

If you look over the changes I've suggested for the install script, you'll see I've been working on making it a bit more intelligent. (which will essentially turn it into a dual purpose installer/updater)

[jacobsalmela]

So would you use git or some other method to get the updated files from master?

[PromoFaux]

For the Admin Dashboard? git. Same as updateDashboard.sh

For the rest of the files, no reason why they too could not be checked out to a local repo and distributed to where they need to be, rather than using curl as it is right now.

[jacobsalmela]

So wouldn't we need to restructure the repo to reflect the file structure of the OS so the files end up in the right place?

[dschaper]

It would add git to the dependencies, but I don't think it's all that big of a package. There's a nice StackExchange post with a function on evaluating the status of the local repo and deciding on the need to fetch/pull or stay.

[PromoFaux]

@jacobsalmela - I shouldn't have thought so, the script can handle copying file out to their relevant directories!

@dschaper - Apparently It's already a dependency

[dschaper]

@PromoFaux Well, I learn something new everyday! Lol...

[jacobsalmela]

Yeah, I added that back when I merged the updateDashboard.sh script.

[AzureMarker]

Does the new install script fix this?

[PromoFaux]

Ah, forgot the issue was open still!

Yeah, as far as I'm concerned the install script safely acts as an update mechanism too.

[kmr2319]

Would love for an "update" mechanism. I have some additional customizations to my dnsmasq.conf that get overwritten every time I "reinstall". This happened as recently as Jan 28th.

I like the git idea better than the deb package - only because I'm a latest and greatest kind of guy...
These projects use a git sync type of update if I'm not mistaken:
https://github.com/drzoidberg33/plexpy
https://github.com/SiCKRAGETV/SickRage
https://github.com/RuudBurger/CouchPotatoServer
As I'm sure others do too...

[dschaper]

Version 2.5 of Pi-hole now uses a config file in /etc/dnsmasq.d/01-pihole.conf just so we specifically don't stomp on existing installations that use dnsmasq.conf for their own internal use.

[dschaper]

Also, you should always be able to get the latest version via curl or git but we're also working on .deb packaging for users that would like to get their updates via apt-get. That part is taking a lot longer than I expected and that's my fault, Debian has quite the stringent ruleset when it comes to making packages that are compliant...

[PromoFaux]

@kmr2319 This is now fixed from V2.5.1 onwards.

The logic is now that if we detect an existing dnsmasq.conf that doesn't contain any pi-hole modifications, then we will leave it alone.

If it happens to contain pi-hole modifications AND your own (or just pi-hole modifications) it will be backed up to dnsmasq.conf.orig and overwritten with the default dnsmasq.conf from a fresh install of dnsmasq.conf. The first time you run the install script to update, you may need to manually fix your dnsmasq.confone last time, but every subsequent run will leave it alone!

[kmr2319]

Fantastic! I had updated to 2.5 just after posting my comment, saw that the dnsmasq.conf was overwritten and the new 01-pihole.conf was created. Dropped my few config items into their own conf file in /etc/dnsmasq.d. Saw that 2.5.1 was out there and updated again and my changes are still available!!! Awesome.

However, it would be nice if there was a separate upgrade script, so existing users don't have to go through the install script and be prompted for any input. Just grab the latest code and recycle....

Thanks again!

[DonLexos]

Agree with @kmr2319 on the update part. Ofcourse the script checks for all kinds of files in existance and makes sure settings are resolved, but dedicated IP check, ip4/6 selection and more can be skipped in general on an upgrade imho aswell. If you want to change it you could always just run the installer instead of the upgrade. :-)

[PromoFaux]

The general idea was that if they wanted to change things, they could.. but it's a good point actually, just an "update.sh" that ships with it, so you don't even need to curl it, just run it! I'll get on that later!

[DonLexos]

if that would exist it could even be CRON'd to auto update weekly, provided it only gets stuff from master

[PromoFaux]

Autoupdate COULD be set up, but not by default. We could provide an option to run it automatically, or provide instructions on how it can be run through cron, but if we start updating automatically out of the box, we're gonna have unhappy users somewhere along the line...

[DonLexos]

Yes I am aware of that, not by default, but same as earlier to purge the pihole.log daily (which is now default I believe regardless if any statistics are saved) to keep the system quick and lean.

[jacobsalmela]

I have often thought of a

curl -L update.pi-hole.net | bash

I wouldn't want to do this until we do a revamp of some code, which would prevent us from repeating code or missing out on any hardcoded variables.

As as example:

1. /etc/phole/pihole.vars would be a bash script that would export variables.
2. /usr/local/bin/updatePihole would source /etc/phole/pihole.vars to get all the folder name, IP's, etc.

Then /usr/local/bin/updatePihole could just be added as a cron job without the need to be piping stuff to bash over the Internet, which I know doesn't sit well with everyone.

Also, we could make a separate update script now, but we would probably end up using a lot of the same code, which doesn't make sense to have it in two places.

The git method helps a lot right now, and as @dschaper said, the .deb would be ideal and make it more portable for updating via apt-get. So there are a couple of different approaches that will need some thought.

[dschaper]

Thanks ;)

[jacobsalmela]

Oops. I though I could assign more than one person...I was trying to assign myself and @PromoFaux

[dschaper]

You should be able, it should just be a toggle...

[dschaper]

Huh, I guess you cant? That's odd?

[jacobsalmela]

Yeah, I thought it would be like Labels where you can select multiple ones...

[AzureMarker]

The hardest part with update.pi-hole.net is that we would have to handle every version case, especially if we change locations of files or add new ones. @PromoFaux suggested bundling an update script with pi-hole, which would do some update prep and then download a new updater that would finish the job. I'm thinking that the first one could get it to a stable state (all files are either deleted or in a specific location) and then the second just does a headless install over it.

I think we also need a purge script to remove any Pi-Hole specific files, even from old installations. Something using mlocate would work well but I think it would require a lot of user prompts (one for each deletion) to ensure that we dont "purge" anything that we shouldnt

[AzureMarker]

As long as we keep track of what files we create, we shouldn't have a problem of not deleting our files.

[diginc]

Hi, I would endorse the Debian approach since #77 solves lots of problems being homegrown right now.

https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html

removing orphaned files new debian versions don't have

6.6 Details of unpack phase of installation or upgrade
6. Any files which were in the old version of the package but not in the new are removed.
11. The backup files made during installation, above, are deleted.

Uninstall + cleanup

6.8 Details of removal and/or configuration purging
7. The package's file list is removed.

I can't find where it says it prompts to replace/preserve/merge user modified files in the package file list but I know I've experienced debians stopping mid install to ask that to me too.

[PromoFaux]

No longer needed!