Unclear from the documentation, will syncoid remove snapshots? about sanoid

[devZer0]

i also came across this and think it`s a little bit irritating - you tell:

"Sanoid is a policy-driven snapshot management tool for ZFS filesystems."
and then you tell
"Sanoid also includes a replication tool, syncoid, which facilitates the asynchronous incremental replication of ZFS filesystems."

i would expect syncoid does replication - if some snapshot is added on the source, syncoid will sync it to the target, too.

so, if syncoid is for replication i`m quite surprised that it does only add but never remove a snapshot which is being removed at the source.

i also tried zrep, but syncoid seems to be more reliable

[ljwobker]

I'm struggling through what I think is a similar use case: I want to ultimately run sanoid on the source machine and build up some set of periodic snapshots on the source machine, and syncoid those over to the target machine. But on the target machine, I want to have a different set of retention policies for that dataset. For the sake of example, let's say that on the source I want to have 24 hourlies, 30 dailies, 12 monthlies, and 3 yearlies. But on the target, I want to only keep 3 monthlies and 3 yearlies.

Is the "right" approach here to syncoid everything over to target, and then run sanoid with a different set of templates/configs? Or do we run syncoid on the target system with some sort of config that allows it to keep a different set of retention policies?

Sorry if this is basic or has been answered somewhere, but I couldn't find it and AndrewJDR's comment above about adding it to the README led me to believe I might not be the only one who could benefit from the explanation... ;-)

[jimsalterjrs]

That's not syncoid's job. If you want automated snapshots removed on the target, run Sanoid on the target, with a policy that tells it what snapshots to prune and when, and not to take snapshots for itself.

There's an example policy template for replication targets - the "backup" template - in the default conf files included here on Git.

[Vilbrekin]

It would be a nice feature to add. Indeed my backp server is accumulating all the 'frequent' ones, generating a lot of noise. Would be really nice to have syncoid delete them once deleted on the master (or simply using replication streams).

[AndrewJDR]

Correct me if I'm wrong, but you seem to advocate for a separate policy on the destination that prunes older snapshots, but I find this problematic, since the destination doesn't have any awareness of whether the sources replicating to it are still reliably replicating every day. If a source stops replicating for some reason, and the destination is just blindly pruning old snapshots, it might prune away the only snapshot it had in common with the source, creating a very unpleasant situation once the source starts replicating again.

Likewise if the replication and snapshotting processes are unaware of each other on the source, and replication has been failing for a long period of time, the source might expire the only snapshot it has in common with the destination, creating a similar problem.

Unless I'm missing something here (quite possible!) for a truly robust replication solution that includes pruning, it feels to me like syncoid does need to be involved in the process of pruning on the source and destination, so that it can be confirmed that a common snapshot exists on both the source and destination before pruning a given snapshot on either the source or destination.

[jimsalterjrs]

You do not set it, and cannot control it. Syncoid removes stale sync snapshots after successfully replicating to a new sync snapshot. It will not remove them any other way than after a new successful replication to a new sync snapshot.

As long as you don't personally manually delete the syncoid snapshot, you'll never lose your common snapshot for replication.

Note that this CAN present problems in an A->B, A->C replication scheme with three or more machines, since the syncoid machine C WILL replicate but WON'T remove sync snapshots created by B, and vice versa. If you do three+ way replication like this, you will either need to come up with your own method of removing foreign sync snapshots, or use --no-sync-snap to tell syncoid not to use sync snapshots - in which case you need to keep enough of a handle on your snapshot management that you'll know of problems before they've gone on for long enough that you no longer have a common snapshot.

Basically if you want multiple-way replication, you really really need to set up Nagios or other monitoring agent to call sanoid --monitor-snapshots to check for replication health, and send you timely alarms if you stop getting replication.

[jimsalterjrs]

Syncoid only removes snapshots that syncoid itself has created, from the
same machine on which syncoid being run.

For example on a machine named MYBOX you do syncoid
root@OTHERBOX:pool/dataset pool/dataset. Syncoid creates a snapshot on
OTHERBOX named pool/dataset@syncoid_MYBOX-yyyy-mm-dd-hh:mm:ss, and
replicates up to that snapshot into the local pool on MYBOX.

The next time you run the same command, Syncoid will create a new
snapshot named @syncoid_MYBOX(etc) and replicated it in, then once
successfully replicated in, will remove any priorly existing snapshots
named syncoid_MYBOX(etc) on OTHERBOX and on MYBOX.

Those are the only snapshots Syncoid will ever remove. So if you
don't name your snapshots @syncoid_[machinename]-[datestamp], they're
safe. =)

On 02/17/2016 04:26 AM, Silviu Vulcan wrote:

I have a process on a server that takes regular snapshots and rotates
them (deleting old ones). Will syncoid propagate such deletions ?

Thank you!

—
Reply to this email directly or view it on GitHub
#25.

[jimsalterjrs]

i would expect syncoid does replication - if some snapshot is added on the source, syncoid will sync it to the target, too.

This is correct.

so, if syncoid is for replication i`m quite surprised that it does only add but never remove a snapshot which is being removed at the source.

This would be particularly obnoxious behavior. Backup servers frequently have much greater amounts of storage available than production servers. Not automatically deleting snapshots on the target that are no longer present on the source enables different retention policies on source and target, allowing for things like fast, solid-state production pools with a few days' snapshots, and massive, spinning rust backup pools with months or even years' worth of snapshots.

[jimsalterjrs]

Yes, you're missing something - unless you explicitly tell it not to, syncoid creates its own snapshots, which it manages itself, removing old syncoid snapshots only after successful replication completes. This ensures never losing a common snapshot between the replication host. Sanoid does not remove syncoid snapshots.

[AndrewJDR]

Thanks, I'm glad I was missing something.
Where do I set the policy for when syncoid decides to expire snapshots on source and dest? Does it use the same pruning configuration as sanoid (from the conf file)?

[AndrewJDR]

I don't need multiple-way replication, so that's not an issue for me.

Regarding what you said here:

You do not set it, and cannot control it. Syncoid removes stale sync snapshots after successfully replicating to a new sync snapshot. It will not remove them any other way than after a new successful replication to a new sync snapshot.

What do we mean by "stale" in this case? All but the most recent in-common snapshot? I want to understand better how syncoid decides what's stale or not / what to expire or not.

[AndrewJDR]

Okay, I think I understand after having used syncoid/sanoid more.

I've used a different zfs replication scheme before (freenas) that only replicated the snapshots created by the snapshotting script, and so the way syncoid creates and replicates its own set of snapshots (separate from sanoid) is different from that, so I had no intuition about it. But at the end of the day I far prefer the syncoid method, since it does a better job of ensuring a common snapshot always exists between sources and destinations.

If I get some time, I'd like to write up a short paragraph for the README and submit it as a PR, just to explain the reasoning behind separate syncoid snapshots (i.e. to ensure at least one common snapshot is always maintained between sources and destinations), and the behavior of them - that they get automatically pruned to the minimum number of snapshots possible in order to maintain one common snapshot between source and dests. I think it would be helpful for people coming from other replication schemes that work differently.