moby / docker-ci-zap

run file:

.\docker-ci-zap.exe -folder "C:\ProgramData\docker"

docker-ci-zap's Issues

Please add this to chocolatey

This shall be either inbox with docker or be easily accessible for admins to perform maintenance of docker hosts.

Tool throws exception

upon running:
.\docker-ci-zap.exe -folder "J:\WindowsImages"

The following exception is thrown:

Exception 0xc0000420 0x0 0x0 0x7ff934412dda
PC=0x7ff934412dda

syscall.Syscall(0x7ff94914d430, 0x2, 0xc082065d68, 0xc082008900, 0x0, 0xe, 0xe, 0x0)
c:/go/src/runtime/syscall_windows.go:128 +0x5c
github.com/microsoft/hcsshim._destroyLayer(0xc082065d68, 0xc082008900, 0x0, 0x0)
e:/go/src/github.com/docker/docker/vendor/src/github.com/microsoft/hcsshim/zhcsshim.go:181 +0x9f
github.com/microsoft/hcsshim.destroyLayer(0xc082065d68, 0xc0820085c3, 0xd, 0x0, 0x0)
e:/go/src/github.com/docker/docker/vendor/src/github.com/microsoft/hcsshim/zhcsshim.go:174 +0x77
github.com/microsoft/hcsshim.DestroyLayer(0x0, 0xc0820085c0, 0x3, 0xc0820085c3, 0xd, 0x0, 0x0)
e:/go/src/github.com/docker/docker/vendor/src/github.com/microsoft/hcsshim/destroylayer.go:18 +0x38a
main.main()
E:/docker/ci/docker-ci-zap/zap.go:38 +0x23a
rax 0x20
rbx 0x0
rcx 0x60
rdi 0x20
rsi 0x8d8830
rbp 0x8f569
rsp 0x8f500
r8 0x20
r9 0x58
r10 0x17
r11 0x246
r12 0x0
r13 0x0
r14 0x0
r15 0x0
rip 0x7ff934412dda
rflags 0x246
cs 0x33
fs 0x53
gs 0x2b

The specified executable is not a valid application for this OS platform

OS: Windows Server Core (1709)

To reproduce: .\docker-ci-zap.exe -folder "D:\dockerleftoversdir"

Error:

PS D:\tools> .\docker-ci-zap.exe -folder "D:\oldDocker"
Program 'docker-ci-zap.exe' failed to run: The specified executable is not a valid application for this OS platform.At
line:1 char:1
+ .\docker-ci-zap.exe -folder "D:\oldDocker"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.
At line:1 char:1
+ .\docker-ci-zap.exe -folder "D:\oldDocker"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [], ApplicationFailedException
    + FullyQualifiedErrorId : NativeCommandFailed

Expected: it should be platform independent, I mean should work on any recent Windows version.

It gives issue as Access is denied though I opened cmd as Administrator.

C:\Program Files>.\docker-ci-zap.exe -folder "C:\Program Files\Docker"
time="2018-07-04T11:38:09+05:30" level=error msg="hcsshim::DestroyLayer - Win32 API call returned error r1=0x80070005 err=Access is denied.id=Docker flavour=0"
ERROR:  hcsshim::DestroyLayer - Win32 API call returned error r1=0x80070005 err=Access is denied.id=Docker flavour=0

C:\Program Files>ls Docker
Docker

after i gave it relative path and it said Zapped Succesfully but still Docker directory was present.

C:\Program Files>ls Docker
Docker

C:\Program Files>.\docker-ci-zap.exe -folder Docker
INFO: Zapped successfully

Exception 0xc0000420 0x0 0x0 0x7ffa8a272dda

Running the .exe gives this output:

PS C:\Users\user\docker-ci-zap-master> .\docker-ci-zap.exe -folder "C:\ProgramData\docker"
Exception 0xc0000420 0x0 0x0 0x7ffa8a272dda
PC=0x7ffa8a272dda

syscall.Syscall(0x7ffa9293d430, 0x2, 0xc082067d68, 0xc0820085e0, 0x0, 0x7, 0x7, 0x0)
        c:/go/src/runtime/syscall_windows.go:128 +0x5c
github.com/microsoft/hcsshim._destroyLayer(0xc082067d68, 0xc0820085e0, 0x0, 0x0)
        e:/go/src/github.com/docker/docker/vendor/src/github.com/microsoft/hcsshim/zhcsshim.go:181 +0x9f
github.com/microsoft/hcsshim.destroyLayer(0xc082067d68, 0xc0820065cf, 0x6, 0x0, 0x0)
        e:/go/src/github.com/docker/docker/vendor/src/github.com/microsoft/hcsshim/zhcsshim.go:174 +0x77
github.com/microsoft/hcsshim.DestroyLayer(0x0, 0xc0820065c0, 0xf, 0xc0820065cf, 0x6, 0x0, 0x0)
        e:/go/src/github.com/docker/docker/vendor/src/github.com/microsoft/hcsshim/destroylayer.go:18 +0x38a
main.main()
        E:/docker/ci/docker-ci-zap/zap.go:38 +0x23a
rax     0x20
rbx     0x0
rcx     0x60
rdi     0x20
rsi     0xa0fa90
rbp     0x8f569
rsp     0x8f500
r8      0x20
r9      0x58
r10     0x17
r11     0x246
r12     0x0
r13     0x0
r14     0x0
r15     0x0
rip     0x7ffa8a272dda
rflags  0x246
cs      0x33
fs      0x53
gs      0x2b

Getting error running this utility

If -folder parameter ends with "\" then error below is being thrown

PS D:\docker> .\docker-ci-zap.exe -folder "C:\ProgramData\Docker\windowsfilter\"
time="2016-12-18T14:13:33-06:00" level=error msg="hcsshim::DestroyLayer - Win32 API call returned error r1=0x8007007b err=The filename, directory name, or volume label syntax is incorrect.id= flavou
r=0"
ERROR:  hcsshim::DestroyLayer - Win32 API call returned error r1=0x8007007b err=The filename, directory name, or volume label syntax is incorrect.id= flavour=0

Usage?

What folder path should be supplied?

Erroneous error: The process cannot access the file because it is being used by another process

I do not have dockerd running.

PS C:\ProgramData\docker> C:\Users\mrichman\Downloads\docker-ci-zap.exe -folder "C:\ProgramData\docker"

time="2017-08-14T12:41:42-04:00" level=error msg="hcsshim::DestroyLayer - Win32 API call returned error r1=0x80070020 err=The process cannot access the file because it is being used by another process.id=docker flavour=0"
ERROR:  hcsshim::DestroyLayer - Win32 API call returned error r1=0x80070020 err=The process cannot access the file because it is being used by another process.id=docker flavour=0

Two issues I'm noticing with using the docker-container driver to work around the caching issue:

          Two issues I'm noticing with using the docker-container driver to work around the caching issue:

1. It adds some export/import steps
2. docker push seems to be pushing all layers?

With the default driver, rebuilds of code-only changes take ~1 minute (when I get proper caching of the expensive layers in my image).
With the docker-container driver, these 2 factors mean rebuilds after code-only changes take ~4-5 minutes.

export/import steps

#25 exporting to oci image format
#25 exporting layers done
#25 exporting manifest sha256:01230f6377dec5a6988c924373bb62afe2837d3afa7bb0e84e98a016481c1c81 done
#25 exporting config sha256:4f48d81bc559f074600e3088949591f885d4ef3c74b8d833408864b6bd013df4 done
#25 sending tarball
#25 ...

#26 importing to docker
#26 DONE 32.1s

#25 exporting to oci image format
#25 sending tarball 43.0s done
#25 DONE 43.0s

This seems to add an extra minute to the build. I'm working with large images (~3.5gb from various scientific Python libraries), which I'm guessing exacerbates this issue.

docker push issue

Pushing my 3.5gb image takes ~3 minutes.

It seems that with the docker-container driver, docker push isn't able to see that the expensive layers are shared, and it's pushing all the layers instead of only pushing the new layers? I'm guessing this based on the output from docker push not saying "Layer already exists":

6474dc186dfd: Preparing
2d80b2e557e9: Preparing
59149f33a870: Preparing
ed04f21afbe5: Preparing
c9ec67fe6421: Preparing
e42dc4266416: Preparing
a55e5a0e7c4a: Preparing
aef13dfbb6f9: Preparing
1e602bec2da5: Preparing
b1c4e3f331ea: Preparing
3fdf9f44ae06: Preparing
78ce42cd87aa: Preparing
82e21ae59256: Preparing
02c055ef67f5: Preparing
e42dc4266416: Waiting
3fdf9f44ae06: Waiting
a55e5a0e7c4a: Waiting
78ce42cd87aa: Waiting
aef13dfbb6f9: Waiting
1e602bec2da5: Waiting
82e21ae59256: Waiting
b1c4e3f331ea: Waiting
02c055ef67f5: Waiting
ed04f21afbe5: Pushed
59149f33a870: Pushed
c9ec67fe6421: Pushed
2d80b2e557e9: Pushed
aef13dfbb6f9: Pushed
1e602bec2da5: Pushed
b1c4e3f331ea: Pushed
3fdf9f44ae06: Pushed
6474dc186dfd: Pushed
82e21ae59256: Pushed
78ce42cd87aa: Pushed
02c055ef67f5: Pushed
e42dc4266416: Pushed
a55e5a0e7c4a: Pushed

I push several tags. The first push takes 3 minutes, and the rest of the tags finish quickly as they all say "Layer already exists" for all the layers.

Originally posted by @jli in moby/buildkit#2274 (comment)