How to backup all streams of a NATS Streaming Server? about natscli

[ripienaar]

We do not have a way to do that, each stream goes into back up state independently, message ingest does not pause during snapshots so you'd never get a exact point in time snapshot.

You can shut one of the replicas assuming all is R3 or greater and back up its disk - I know not ideal.

[hossainemruz]

Thank you @ripienaar for your response. Do you have any plan to support the workaround? For example, if the stream name is a wildcard, then the backup command will automatically find the respective streams and backup them. Probably, in a single *.tgz file but one file for each stream is also fine.

[ripienaar]

for stream in $(nats s ls -j|jq -r '.[]')
do
  nats stream backup ....
done

[hossainemruz]

Yes. That's what I am planning for now. However, it would have been great if nats has built-in support for this use-case. We can contribute the feature if you are willing to accept.

[ripienaar]

yes, should be a easy thing to add to the stream backup command, I'd then take the supplied directory and do dir/<stream name>/backup files... so you get a series of stream backups in one dir. Today each backup has to be in a dir of its own

[hossainemruz]

Let us know if you have bandwidth to do it or need any help from us. It's not an emergency.

Btw, thank you for the amazing works.

[ripienaar]

We'd love a contribution if you want to give it a go

[hossainemruz]

Sure. Will do ASAP.

[joelhoisko]

Maybe an offtopic comment, but just so that I'm not going insane: this issue is about backing up streams in JetStream and not NATS Streaming Server (the precursor to JetStream) right? I'm asking, because I've been searching for information about backups/restores for NATS Streaming Server specifically and to my understanding the NATS Streaming Server doesn't have a concept of streams nor does it really work with the nats CLI tool either. Or am I missing something here?

[ripienaar]

This issue relates to JetStream, not Streaming Server indeed. Given that Streaming Server is retiring, yes we did not add extensive support for it in the nats cli

[ripienaar]

We now have nats account backup that does this.