restic / rest-server

Rest Server is a high performance HTTP server that implements restic's REST backend API.

Introduction

restic is a backup program that is fast, efficient and secure. It supports the three major operating systems (Linux, macOS, Windows) and a few smaller ones (FreeBSD, OpenBSD).

For detailed usage and installation instructions check out the documentation.

You can ask questions in our Discourse forum.

Quick start

Once you've installed restic, start off with creating a repository for your backups:

$ restic init --repo /tmp/backup
enter password for new backend:
enter password again:
created restic backend 085b3c76b9 at /tmp/backup
Please note that knowledge of your password is required to access the repository.
Losing your password means that your data is irrecoverably lost.

and add some data:

$ restic --repo /tmp/backup backup ~/work
enter password for repository:
scan [/home/user/work]
scanned 764 directories, 1816 files in 0:00
[0:29] 100.00%  54.732 MiB/s  1.582 GiB / 1.582 GiB  2580 / 2580 items  0 errors  ETA 0:00
duration: 0:29, 54.47MiB/s
snapshot 40dc1520 saved

Next you can either use restic restore to restore files or use restic mount to mount the repository via fuse and browse the files from previous snapshots.

For more options check out the online documentation.

Backends

Saving a backup on the same machine is nice but not a real backup strategy. Therefore, restic supports the following backends for storing backups natively:

• Local directory
• sftp server (via SSH)
• HTTP REST server (protocol, rest-server)
• Amazon S3 (either from Amazon or using the Minio server)
• OpenStack Swift
• BackBlaze B2
• Microsoft Azure Blob Storage
• Google Cloud Storage
• And many other services via the rclone Backend

Design Principles

Restic is a program that does backups right and was designed with the following principles in mind:

• Easy: Doing backups should be a frictionless process, otherwise you might be tempted to skip it. Restic should be easy to configure and use, so that, in the event of a data loss, you can just restore it. Likewise, restoring data should not be complicated.

• Fast: Backing up your data with restic should only be limited by your network or hard disk bandwidth so that you can backup your files every day. Nobody does backups if it takes too much time. Restoring backups should only transfer data that is needed for the files that are to be restored, so that this process is also fast.

• Verifiable: Much more important than backup is restore, so restic enables you to easily verify that all data can be restored.

• Secure: Restic uses cryptography to guarantee confidentiality and integrity of your data. The location the backup data is stored is assumed not to be a trusted environment (e.g. a shared space where others like system administrators are able to access your backups). Restic is built to secure your data against such attackers.

• Efficient: With the growth of data, additional snapshots should only take the storage of the actual increment. Even more, duplicate data should be de-duplicated before it is actually written to the storage back end to save precious backup space.

Reproducible Builds

The binaries released with each restic version starting at 0.6.1 are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that are contained in the builder repository.

News

You can follow the restic project on Mastodon @resticbackup or subscribe to the project blog.

License

Restic is licensed under BSD 2-Clause License. You can find the complete text in LICENSE.

Sponsorship

Backend integration tests for Google Cloud Storage and Microsoft Azure Blob Storage are sponsored by AppsCode!

rest-server's Issues

Create issue and PR templates

Create templates for issues (bug reports and feature requests) as well as PRs.

rest-server with non-local (Google Cloud, B2, S3..) backend

Just starting with restic (which actually seems to be "too good to be true"! :-) so maybe I missed a feature. Otherwise: Feature request!?

Having been to all kinds of bad situations, I consider the threat model in restic/restic#784 very real:
"Servers-to-be-backuped" (STBB) are taken over by an adversary trying to delete backups for extortion purposes.
I'd extend it to "admin fat fingers" (accidentally deleting backups from the STBB). So I like the idea of --append-only with a dedicated rest-server (RS) that doesn't share admin credentials with STBBs.

I would like to let RS store its data on the cloud backends (remote storage) as implemented by restic's client itself: Google Cloud Storage, Backbaze B2, AWS S3, SFTP... (personally interested in GCS and B2)
Why?

• I don't have an RS with adequate storage.
• I don't want to maintain libraries and credentials for remote storage on all STBB.
• I would like to switch remote storage without reconfiguring a dozen STBBs but only one RS.

Is this possible with rest-server?

Philipp

go mod vendor Problems with goji

make failed:

go: inconsistent vendoring in /opt/rest-server:
        github.com/beorn7/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/goji/[email protected]+incompatible: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/golang/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/gorilla/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/inconshreveable/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/matttproud/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/miolini/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/prometheus/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/prometheus/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/prometheus/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/prometheus/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/spf13/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        github.com/spf13/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        [email protected]+incompatible: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        golang.org/x/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        [email protected]+incompatible: is replaced in go.mod, but not marked as replaced in vendor/modules.txt
        github.com/gorilla/[email protected]: is replaced in go.mod, but not marked as replaced in vendor/modules.txt

run 'go mod vendor' to sync, or use -mod=mod or -mod=readonly to ignore the vendor directory
Makefile:15: recipe for target 'rest-server' failed
make: *** [rest-server] Error 1

go mod vendor failed also
go: github.com/goji/[email protected]+incompatible used for two different module paths (github.com/goji/goji and goji.io)

rest-server log POST request size is zero

POST requests are logged with a size of "0"
GET requests are logged with the correct size

Is it possible to log the request size for POST requests too?
Want to do some stats based on access logs.

Tested with rest-server 0.9.6

BR Joerg

TLS Client Authentication

Feature request. Alternative to htpasswd auth.

unexpected HTTP response code 500

Hey,

i ve Problem.

I installed the restic-server and started it on port 8000.
An init and backup doesnt be work.

debug-output:

CheckConfig()
stat /tmp/restic/ovm4154/config: no such file or directory
POST /ovm4154/keys/93bcd6be459943629ee0e7f21795c9f00d9d2cab7813c19e6a8284e2be977b18
SaveBlob()
open /tmp/restic/ovm4154/keys/93bcd6be459943629ee0e7f21795c9f00d9d2cab7813c19e6a8284e2be977b18: no such file or directory
GET /ovm4154/keys/
ListBlobs()
GET /ovm4154/keys/9212e47add82a13c56944d9c627e06f1f329bc0f40ffa72bda9bd94c20ac8aee
GetBlob()
GET /ovm4154/config
GetConfig()
GET /ovm4154/locks/
ListBlobs()
POST /ovm4154/locks/b2324dbd1842a8ebdef807865be7d69adc44f5a5b16218488387802d0ad5c664
SaveBlob()
GET /ovm4154/locks/
ListBlobs()
GET /ovm4154/locks/b2324dbd1842a8ebdef807865be7d69adc44f5a5b16218488387802d0ad5c664
GetBlob()
GET /ovm4154/index/
ListBlobs()
GET /ovm4154/snapshots/
ListBlobs()
POST /ovm4154/data/8b721003b31308f0bc931d242cb747e11ac1667454e57ab190416c777f7424c2
SaveBlob()
open /tmp/restic/ovm4154/data/8b/8b721003b31308f0bc931d242cb747e11ac1667454e57ab190416c777f7424c2: no such file or directory
POST /ovm4154/data/d9eb46fdbf9083fca170e23c73516b333b225bbd940ffc89aa4a16057d3bfe72
SaveBlob()
open /tmp/restic/ovm4154/data/d9/d9eb46fdbf9083fca170e23c73516b333b225bbd940ffc89aa4a16057d3bfe72: no such file or directory
POST /ovm4154/data/0e6ff305af31380f304b46d1ce97b6f4e106521147f3390f8308a077fab81d29
SaveBlob()
open /tmp/restic/ovm4154/data/0e/0e6ff305af31380f304b46d1ce97b6f4e106521147f3390f8308a077fab81d29: no such file or directory

Do you have any idea?

greetings
Sam

SELinux Issues

After upgrading to latest Redhat 7.5 release I cannot start my restic/rest-server docker container. Seems to be some SE Linux issues. Maybe I'm missing something in my docker setup.

docker-compose up
Starting rest_srv ... done
Attaching to rest_srv
rest_srv | Error relocating /lib/ld-musl-x86_64.so.1: RELRO protection failed: Permission denied
rest_srv | Error relocating /entrypoint.sh: RELRO protection failed: Permission denied
rest_srv exited with code 127

tail audit.log
type=PROCTITLE msg=audit(1527685116.086:3277): proctitle=2F7573722F7362696E2F69707461626C6573002D2D77616974002D74006E6174002D4900444F434B45525F504F5354524F5554494E47002D73003132372E302E302E3131002D7000746370002D2D73706F7274003333303231002D6A00534E4154002D2D746F2D736F75726365003A3533
type=AVC msg=audit(1527685116.187:3278): avc: denied { read } for pid=16615 comm="entrypoint.sh" path="/lib/ld-musl-x86_64.so.1" dev="dm-2" ino=669806 **scontext=system_u:system_r:container_t:**s0:c462,c865 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file
type=SYSCALL msg=audit(1527685116.187:3278): arch=c000003e syscall=10 success=no exit=-13 a0=7f5f76db6000 a1=1000 a2=1 a3=7f5f76b401bb items=0 ppid=16596 pid=16615 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="entrypoint.sh" exe="/bin/busybox" subj=system_u:system_r:container_t:s0:c462,c865 key=(null)
type=PROCTITLE msg=audit(1527685116.187:3278): proctitle=2F62696E2F7368002F656E747279706F696E742E7368
type=AVC msg=audit(1527685116.187:3279): avc: denied { read } for pid=16615 comm="entrypoint.sh" path="/bin/busybox" dev="dm-2" ino=266240 scontext=system_u:system_r:container_t:s0:c462,c865 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file
type=SYSCALL msg=audit(1527685116.187:3279): arch=c000003e syscall=10 success=no exit=-13 a0=56308595f000 a1=4000 a2=1 a3=5630856a22d4 items=0 ppid=16596 pid=16615 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="entrypoint.sh" exe="/bin/busybox" subj=system_u:system_r:container_t:s0:c462,c865 key=(null)
type=PROCTITLE msg=audit(1527685116.187:3279): proctitle=2F62696E2F7368002F656E747279706F696E742E7368

Looks like the restic docker container files need to have container_t context?

Audit2allow gives me this but Not sure how to change context on the docker image filesystem.

module test_rule 1.0;

require {
type unlabeled_t;
type container_t;
class file { execute_no_trans open read };
}

#============= container_t ==============

#!!!! The file '/usr/bin/rest-server' is mislabeled on your system.
#!!!! Fix with $ restorecon -R -v /usr/bin/rest-server
allow container_t unlabeled_t:file execute_no_trans;

#!!!! This avc is allowed in the current policy
allow container_t unlabeled_t:file { open read };

What is in a name?

Why do we stick to the name rest-server?

And what would be a better name?

Add docker port mapping

In readme guide please add: -p 80:80

Start server

docker run --name myserver -p 80:80 -v /my/data:/data restic/rest-server

If you don't forward the container port then restic cannot communicate with the container on the rest server.

Why?

Was hoping to get some more information (here or in the README) as to why I or anybody would want to use this over the SFTP backend. Does it have better performance characteristics? If so, why? What about compared to OpenSSH-HPN? What does this give you that SFTP doesn't?

Require a htpasswd by default

Currently rest-server fails open (does not require auth) if a .htpasswd file does not exist or cannot be opened. This is a pretty bad default from a security point of view.

I suggest that we require the .htpasswd file to be present, unless a -no-auth flag is set by the user to explicitly tell rest-server it should be wide open.

The downside of adding this is that it would break deployments for existing users that rely on their server being open. I'm not sure if this is a good idea, because it might break unattended backups for users and they might not be monitoring them.

PR #59 contains some comments on this.

Private repos

The new private repositories feature add in 6c846f8 and included in release v0.9.7 does not appear to be documented in the README or elsewhere.

dns issue?

restic won't connect to the rest-server via domain name, only via ip.
I get this error:

restic -r rest:http://mini.local:8000 backup Documents/
unable to open config file: client.Head: Head http://mini.local:8000/config: dial tcp: lookup mini.local on 192.168.20.1:53: no such host
Is there a repository at the following location?
rest:http://mini.local:8000

./rest-server --path /Users/phil/backups/restic
rest-server 0.9.4 (v0.9.4-0-g0a0ed9c) compiled with go1.8.3 on darwin/amd64
Data directory: /Users/phil/backups/restic
Authentication disabled
Starting server on :8000

ping mini.local works fine.
This is on a local network - the server and clients are macs (10.12.6).
192.168.20.1 is my router.
Any ideas?

Make configuration not global

I'm using this as a library and would like to start REST servers for different data directories. Currently, the Config is global and cannot be customized per-mux.

I will submit a PR for this. Do you have any design suggestions for making this change?

SynoCommunity Package

I'd like to build an package for SynoCommunity and therefore for Synology NAS systems.

I saw references in the last issues (#24 (comment) & #25).

What needs to be done?

• Base Package
• Some sort of UI to set the Repo-path, the SSL-certs and all other parameters

---

Related links:

• SynoCommunity
• SynoCommunity's Buildsystem
• Developers HowTo

Integrate with syslog

Hi,

The title is pretty self-explanatory: would it be possible to integrate with syslog?

Our logs are growing pretty fast so we need rotation, among other nice features of syslog :)

Publish Docker Image

As #14 was merged, someone from restic team should publish the image to http://hub.docker.com/r/restic

Request path sanitization issues

There seems to be a small request path sanitization issue:

bitsie:~ weingart$ curl -v --path-as-is http://toby:toby@localhost:8000/data/../ && echo ''
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8000 (#0)
* Server auth using Basic with user 'toby'
> GET /data/../ HTTP/1.1
> Host: localhost:8000
> Authorization: Basic dG9ieTp0b2J5
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Thu, 27 Jul 2017 05:10:41 GMT
< Content-Length: 64
< Content-Type: text/plain; charset=utf-8
< 
* Connection #0 to host localhost left intact
[".htpasswd","config","data","index","keys","locks","snapshots"]
bitsie:~ weingart$

I've not tried the other endpoints, nor tried other methods of sending bogus paths.

File open issue

Hi,

while trying to backup via restic I get the following error on a synology NAS (arm device)

POST /data/84da53f26249f5774d99ccafecf244e770a6b37da22faa7ffbb817d8ff641224
SaveBlob()
open /volume1/Public/data/84/84da53f26249f5774d99ccafecf244e770a6b37da22faa7ffbb817d8ff641224: no such file or directory

I traced it back to handlers.go:239

tf, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_EXCL, 0600)

and reason could either be that the folder is not created (it isn't there) or that the permissions can't be set.

Did anyone have a similar issue?

Koen

Question: max body size on a reverse proxy

What is the minimum required body size for rest-server?

Thanks!

Persist changelog/unreleased/ folder

Add a .gitkeep file to make it persist even when it's empty.

Feature request: limit repo size

I'd like to host a REST server for multiple users and I don't want them to use up all of my server's disk space.
So it would be nice to be able limit the overall disk space the repos take, i.e. the recursive size of --path.
Furthermore, I'd optimally be able to set different repo sizes for different users.

Compare with minio

Following #9 i'd like to know how it compare to minio ?
I see that it's dead simple, which is a killer feature, but maybe it's also more efficient ?

Further prometheus metrics

Hi!

Great work. This is exactly how backups should be.

What I'd wish: Expose not only rather technical data in the '/metrics' endpoint but also information about the backup status. I could then easily setup alarms (e.g. "latest backup is older than 3 days") and be assured everything is fine.

Best wishes
Adrian Schneider

Ability to specify path of TLS cert and key

It would be nice if I'll be able to specify TLS cert and key path.
i.e. ./rest-server --tls-cert /etc/restic/cert --tls-key /etc/restic/key --tls

Show a webpage when visited from a browser.

Docker image ID

da93e5693693

What should rest-server do differently?

Health checks! Some way to somehow query the server is not dead nor a zombie. Active gophers only.

What are you trying to do? What is your use case?

Discovering problems early.

Did rest-server help you today? Did it make you happy in any way?

I'll opt for a wonderful image instead, by Tadeusz Lakota:

Please put compiled binary files for rest-server in Releases

Hi!

Thank you for restic, it is awesome. I'm now writing deploy with ansible for integrating restic backup in our infrastructure and it will be great to just download already compiled binary file of rest-server instead of compiling it.

Ivan Luckyanov

Build separate binaries for various arm & mips architectures

See https://golang.org/doc/install/source#environment

Specifically $GOARM & $GOMIPS variables.

Set --append-only and --max-size per user

It would be nice if I could set the --append-only option per user, to be able to backup multiple hosts, where some are trusted and should be allowed to change data afterwards (e.g. forget old snapshots) and some are not, using a single rest-server instance with a shared --path and a shared .htpasswd file.

My current solution consists of two different instances - one with the --append-only flag and one without - using two different remotes(/hostnames, using lighttpd as a proxy), two different paths and two different htpasswd files, which isn't that great.

Support subdirectories with --private-repos

What should rest-server do differently?

Currently only one level of username is supported with the --private-repos flag, e.g. http://server/username is supported but not http://server/username/foo, http://server/username/foo/bar, etc.

Rest-server should allow at least one additional level, e.g. http://server/username/foo/bar.

We should at the same time consider if there's a point in, and without too much complexity, possible to, make it accept any number of subdirectories, e.g. http://server/username/foo/bar/star/har.

What are you trying to do? What is your use case?

Example use case: #76 (comment)

TLS hot reload

Hello,
thank you for rest-server! I'm using it with letsencrypt-issued TLS certificates and I'm interested to know if hot-reload of certificates is planned or something that rest-server could get? For example upon SIGHUP (#80) or automatically e.g. with https://github.com/dyson/certman.

server response unexpected: 500 Internal Server Error (500)

Versions

restic version : 0.7.0
rest-server version : 0.9.3

Steps to reproduce:

Step1. Create directory rest-test

root@backup-server-restic:/backup# mkdir rest-test

Step2. Initialize it as local repo

root@backup-server-restic:/backup# /usr/local/bin/restic -r /backup/rest-test init                             
enter password for new backend:                      
enter password again:                                
created restic backend f8ff0dbe7b at /backup/rest-test                                                    

Please note that knowledge of your password is required to access                                         
the repository. Losing your password means that your data is                                              
irrecoverably lost. 

Step3. Run rest-server

root@backup-server-restic:/backup# /usr/local/bin/rest-server --path /backup/rest-test/ --listen ":8002" --debug
rest-server 0.9.3 compiled with go1.8.1 on linux/amd64
Data directory: /backup/rest-test/
Authentication disabled
Starting server on :8002

Step4. Try to create snapshot of /tmp directory using rest-server api, get 500 internal server error

Output from client:

root@backup-server-restic:/backup/rest-local/tmp# /usr/local/bin/restic -r rest:http://127.0.0.1:8002/ backup /tmp/
enter password for repository: 
scan [/tmp]
scanned 9 directories, 0 files in 0:00
[0:00]   0B/s  0B / 0B  9 / 9 items  0 errors  ETA 0:00 
duration: 0:00, 0.00MiB/s
server response unexpected: 500 Internal Server Error (500)
restic/backend/rest.(*restBackend).Save
        src/restic/backend/rest/rest.go:134
restic/repository.(*Repository).savePacker
        src/restic/repository/packer_manager.go:122
restic/repository.(*Repository).Flush
        src/restic/repository/repository.go:248
restic/archiver.(*Archiver).Snapshot
        src/restic/archiver/archiver.go:733
main.runBackup
        src/cmds/restic/cmd_backup.go:493
main.glob..func2
        src/cmds/restic/cmd_backup.go:39
github.com/spf13/cobra.(*Command).execute
        src/github.com/spf13/cobra/command.go:631
github.com/spf13/cobra.(*Command).ExecuteC
        src/github.com/spf13/cobra/command.go:710
github.com/spf13/cobra.(*Command).Execute
        src/github.com/spf13/cobra/command.go:669
main.main
        src/cmds/restic/main.go:63
runtime.main
        /usr/local/go/src/runtime/proc.go:185
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:2197

Output from server:

root@backup-server-restic:/backup# /usr/local/bin/rest-server --path /backup/rest-test/ --listen ":8002" --debug                                                                                                          
rest-server 0.9.3 compiled with go1.8.1 on linux/amd64                                                    
Data directory: /backup/rest-test/                   
Authentication disabled                              
Starting server on :8002                             
HEAD /config                                         
CheckConfig()                                        
GET /keys/                                           
ListBlobs()                                          
GET /keys/5751c09eaf4fdfa0c96c627e8dce0ba3871ae9c50fc532f0a13572d4fca58b76                                
GetBlob()                                            
GET /config                                          
GetConfig()                                          
GET /locks/                                          
ListBlobs()                                          
POST /locks/b7fa70e3bfaf7423e44342e77a6cf48d1012678b7e8cecaaf53a1d5d697eb445                              
SaveBlob()                                           
GET /locks/                                          
ListBlobs()                                          
GET /locks/b7fa70e3bfaf7423e44342e77a6cf48d1012678b7e8cecaaf53a1d5d697eb445                               
GetBlob()                                            
GET /index/                                          
ListBlobs()                                          
GET /snapshots/                                      
ListBlobs()                                          
POST /data/b11f8ad215c01849238d2e462b4ace65fe7023a947335d1dfd313dd66e5d2a13                               
SaveBlob()                                           
open /backup/rest-test/data/b1/b11f8ad215c01849238d2e462b4ace65fe7023a947335d1dfd313dd66e5d2a13: no such file or directory                                                                                           
DELETE /locks/b7fa70e3bfaf7423e44342e77a6cf48d1012678b7e8cecaaf53a1d5d697eb445                            
DeleteBlob()  

Comments

When i initialize rest repository on step2 via rest-server api, everything works good. I only see problem when initialize it locally and then try work with it using rest-server api.

Release new version

It's been a long time since the last release. There have been some improvements regarding metrics and a security fix for append-only mode.

It would be nice it these are available in a released version.

Docker create_user script error when reading password as argument

There is a bug in the docker/create_user script (using both username and password as arguments): htpasswd requires -b argument to indicate that password is passed from the command line rather than prompting for it.

Don't do mkdirAll, if a directory is missing

Been trying to figure out why something keeps recreating my restic repo...and blocking the system from mounting my restic repo.

Was adding all kinds of extra tests to ensure that services wait for the volume to appear, which can take a while (sometimes hours) after bouncing raid box. Turns out the problem is on the other side of the bounce, where volume has disappeared, but rest-server refuses to accept that, and makes the mountpoint non-empty and blocks it from ever coming back.

The mkdirAll was the fix for issue #40

WIsh I had snapped out that new NAS back when I thought I could afford one....

Handle SIGHUP

This is related to https://forum.restic.net/t/rest-server-crashes-or-exits-unexpectedly/1073/4

version: 0.9.7 or dfe9755 (master)

Steps to reproduce

• Start rest-server
• kill -HUP <pid>

What happens

rest-server exits.

What I expect

At least not to die, (ignoring the signal?). Reloading log files would be nice, and that seems to be the usual behaviour for this signal.

Some background, I discovered this because rest-server was unexpectedly dying, and logfiles would be empty. It appears that synology, the box we're running it on, generated a logrotate configuration for it and was killing it every so often. That wasn't fun.

I deactivated it for now, but handling SIGHUP would be nice. Rotating log files can be useful :)

Release files for 0.9.8

It seams 0.9.8 has been released but without any release files.

https://github.com/restic/rest-server/releases/tag/v0.9.8

rest: Server User / Password

Hello @ALL,

from my Point, it's not a good solution to send the Passwort / User with the URL.
Everyone can see the data in the Process list.

is there any other option ?
like export as variable ... or entering after starting restic ?

Thanks a lot :-),
René

Scoop for rest-server

Installing and managing restic on Windows is easier with Scoop. However, cannot find it for rest-server. Any interest?

unable to list pack f8d9161a: unexpected HTTP response code 404

Sometimes the rest-server returns a 404 response code instead of the pack.

The file in question definitely exists:

sh-4.2# ls data/main/data/f8 | grep f8d9
f8d9161a32b18bf4dc2a86d31b20881d820c8373fcc4d8031e0a1001664506a6

It seems that the server fails to return the file properly at times.

The server is started within systemd, could this be an issue with some limit?

Rename --cpuprofile to --cpu-profile

The --cpuprofile flag is inconsistently named, it should be --cpu-profile as it's actually two words (like --append-mode and --private-repos).

I realize people aren't always happy with renames, but we're still at a pre-1.0.0 version and it's better to do it now than later if it's going to be done. Also, this is a flag that probably almost noone ever uses anyway, so it should be fine.

Anyone against?

Mode in which authenticated users can only access their own directory

Hallo,

rest-server should have a mode in which each individual user can only access it's private directory. For instance "userA" should only be able to access the subdirectory "userA".

Mebus

Fork and Die

Hi, there way to let this daemon fork and die.
I see you use systemd, that make it for you.

Use case:
Some NAS not use systemd, then fork and die is required for correct working.
It could it be implemented as command line parameter.
I found a go lib that allow without big effort Fork and Die.

Support LDAP Authentication

The .htpasswd auth is difficult to scale or manage beyond a few users. Supporting LDAP would allow for large number of users in both *nix and Windows environments.

Repository URLs for --private-repos in README

The README currently states rest:https://foo:pass@host:8000/foo/bar as a valid URL for user foo when using --private-repos. But when using such a URL rest-server answers with a 500 Internal Server Error (ignored by restic) first and with a 404 Not Found second on restic init.

We could just fix this error in the documentation or enable such multiple repos for one user. There are some corner cases where this may be interesting to have.

Implementing it for some other separator (e.g. '+') would be pretty easy. I thought I need this, so I already implemented it in https://github.com/moho1/rest-server/tree/multiple_private_repos , but then I found out I don't need it, so I am not opening a pull request here but leave it as discussion. I can happily open a pull request if this is wanted this way.

Implementing it with '/' as separator would be much nicer, but more complex.

Make importable

Hi, me again. 😄 I'm this 👌 close to having this implemented as a Caddy plugin, which will allow you to add the restic directive to your Caddyfile, so you can use HTTPS to transfer files instead of HTTP, without having to worry about certificates.

My motivation is to avoid relying on SFTP which gets slower and slower forever during the transfer, until the transfer takes weeks instead of hours. My hope and expectation is that HTTPS won't do this because it doesn't have the crazy flow control that SSH + SFTP layers on top of TCP's. Or something.

Anyway, one benefit of this being a Caddy plugin is that Caddy can just run indefinitely and keep the
server going (instead of having to restart it to reload renewed certs), and also serve other websites on the same port too. (One alternative you could do is use autocert into your program, to get this feature without integrating with Caddy. Autocert requires only a couple lines of code.)

However, in order for me to use this as a library, I need to be able to import it. Specifically, I think I need access to the setupMux() function. And if the mux and handlers and stuff could be moved into a non-main package, then I could import it.

So, in summary, I'm requesting:

1. Move as much as possible out of the main package so it can be imported
2. Export setupMux() so it can be used by other packages in their own handlers

I think I could submit a PR if you're OK with this.

Can't access metrics when private repos is enabled

When having --private-repos and --prometheus, the /metrics can't be accessed.

The fix would be to add a isMetricsPath() like this:

diff --git a/handlers.go b/handlers.go
index de3f328..04eb5c2 100644
--- a/handlers.go
+++ b/handlers.go
@@ -132,6 +132,14 @@ func isUserPath(username, path string) bool {
        return len(path) == len(prefix) || path[len(prefix)] == '/'
 }
 
+func isMetricsPath(path string) bool {
+       prefix := "/metrics"
+       if !strings.HasPrefix(path, prefix) {
+               return false
+       }
+       return len(path) == len(prefix) || path[len(prefix)] == '/'
+}
+
 // AuthHandler wraps h with a http.HandlerFunc that performs basic authentication against the user/passwords pairs
 // stored in f and returns the http.HandlerFunc.
 func AuthHandler(f *HtpasswdFile, h http.Handler) http.HandlerFunc {
@@ -141,7 +149,7 @@ func AuthHandler(f *HtpasswdFile, h http.Handler) http.HandlerFunc {
                        http.Error(w, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
                        return
                }
-               if Config.PrivateRepos && !isUserPath(username, r.URL.Path) {
+               if Config.PrivateRepos && !isUserPath(username, r.URL.Path) && !isMetricsPath(r.URL.Path) {
                        http.Error(w, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
                        return
                }

I will submit a PR with that patch.

Reports and statistics about usage

Hello,

it would be nice to have at least a CLI utility to see some metrics, for example I want to have a list of all repos with the date of the last backup/snapshot and the total size it takes on disk repo by repo. Basically a bit like what we can do client by client on client side, but do it on server side with a more global view.

This is crucial info to supervise that we could add to our monitoring station, as backups need to be done frequently and we need to know when it is not the case for whatever reason.

It could be non-specific to rest-server backend but since that's what I use that's what I'm interested in :)

The project does not have a Changelog.

This project currently does not have a Changelog file, but should have one.

Mebus

Auto-generate subfolders

In the README it's explained that in order to use previously local repos, moving them to your rest-server, you have to run this one-liner: for i in {0..255}; do mkdir -p $(printf "data/%02x" $i); done.

Would it not make sense to have rest-server simply create those directories automatically when needed? I imagine there's a way to check it being needed or not just once during a backup run or similar.