Migration of already existing repositories leads to 404 about rest-server

[maxkratz]

Update:

The source repository was missing the folder locks (but was working just fine!) Maybe Minio S3 deletes empty folders? I don't know ...

However, reading this thread, I was tempted to create the locks folder manually on the NFS share by running this command: $ mkdir -p locks

This resolved my issue and the repository can now be used by restic accessing via rest-server.

My conclusion at this point is that my restic client refuses to work with rest-server if the folder locks is missing in the repo. This is not the case if using Minio S3 as backend.

[rawtaz]

A short summary is that restic expects the repository to be what it should be. If you or something else deletes parts of the repository that init created, then you may run into issues. In this case the problem was that the copy of the repository wasn't complete.

[maxkratz]

In this case the problem was that the copy of the repository wasn't complete.

Then I'm asking the question of how restic is able to use the repository when accessing via S3. All of my repositories were in use via S3 until yesterday and restic never complained about a missing locks folder.
(I still have the S3 buckets on the original storage and can see that the folder is missing in every repository.)

[maxkratz]

I've created a minimal working example to reproduce this: restic does not create a lock folder if it uses a S3 backend for the repository. I used the newest release from GitHub and ran restic init as well as restic --verbose backup ..

Should I open a new issue at the main repo?

[maxkratz]

Update: I found a similar issue and posted a comment.

[MichaelEischer]

The source repository was missing the folder locks (but was working just fine!) Maybe Minio S3 deletes empty folders?

S3 uses a flat namespace, it has no concept of folders. Files in a "folder" are just files with a corresponding prefix. It is possible to simulate folders with empty placeholder files. But restic doesn't create those files. Thus, copying a repository from S3 won't include these folders.

restic ignores empty folders (see https://github.com/restic/restic/blob/bfc9c6c9712054366ae6cf1dd66839bb17359e44/internal/backend/local/local.go#L258 ), thus there's no reason why the rest-server shouldn't do the same.

@maxkratz the issue in restic is specific for the 'local' backend. It is unrelated to the 'rest' backend.

[maxkratz]

S3 uses a flat namespace, it has no concept of folders. Files in a "folder" are just files with a corresponding prefix. It is possible to simulate folders with empty placeholder files. But restic doesn't create those files. Thus, copying a repository from S3 won't include these folders.

restic ignores empty folders (see https://github.com/restic/restic/blob/bfc9c6c9712054366ae6cf1dd66839bb17359e44/internal/backend/local/local.go#L258 ), thus there's no reason why the rest-server shouldn't do the same.

Thank you for your explanation. I've used an older version of Minio S3 as my restic backend that was able to use a "local"/"file system" backend and wrote all files and folders directly to disk (plus additional metadata of course).

@maxkratz the issue in restic is specific for the 'local' backend. It is unrelated to the 'rest' backend.

You are right. Does this mean I should open another issue in the restic repository regarding the rest backend to state the 404 error if the locks folder is missing? If I understand the discussion in the linked issue correctly, this bug was alreday fixed for the local backend.