varnish 7.x command compatibility about go-varnish-client

[martin-helmich]

Hey Robert! 👋 Thanks for the report, and your analysis. I agree that both solutions are suboptimal; it looks like dropping quoting entirely should certainly break stuff in some cases (at the very least, DefineInlineVCL).

It would be interesting to know what the original varnishadm does in these cases, since that almost certainly won't be broken with Varnish 7. Maybe it's time to read the original Varnish source code again (or to do some Wireshark'ing 😕).

[martin-helmich]

Well, good news: I've been able to reproduce this issue in a unit test (see #29, or more precisely https://github.com/martin-helmich/go-varnish-client/actions/runs/5088338453/jobs/9144624188). Now just to fix it... 😅

[martin-helmich]

I did some digging (using strace, Wireshark would have been overkill); when using quotes, varnishadm exhibits the same behaviour:

write(3, "vl.state \"boot\" \"auto\"", 22) = 22
write(3, "\n", 1)                       = 1
rt_sigprocmask(SIG_BLOCK, [HUP INT QUIT TERM CONT TSTP WINCH], [], 8) = 0
rt_sigaction(SIGINT, {sa_handler=0xffff9cb02420, sa_mask=[], sa_flags=0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGTSTP, {sa_handler=0xffff9cb02420, sa_mask=[], sa_flags=0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGQUIT, {sa_handler=0xffff9cb02420, sa_mask=[], sa_flags=0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGHUP, {sa_handler=0xffff9cb02420, sa_mask=[], sa_flags=0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGTERM, {sa_handler=0xffff9cb02420, sa_mask=[], sa_flags=0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGCONT, {sa_handler=0xffff9cb02420, sa_mask=[], sa_flags=0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGWINCH, {sa_handler=0xffff9cb02420, sa_mask=[], sa_flags=0}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig -icanon -echo ...}) = 0
ioctl(0, TCGETS, {B38400 opost isig -icanon -echo ...}) = 0
rt_sigprocmask(SIG_BLOCK, [WINCH], [], 8) = 0
ioctl(0, TIOCGWINSZ, {ws_row=59, ws_col=237, ws_xpixel=0, ws_ypixel=0}) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
write(1, "varnish> ", 9varnish> )                = 9
ppoll([{fd=3, events=POLLIN}, {fd=0, events=POLLIN}], 2, NULL, NULL, 0) = 1 ([{fd=3, revents=POLLIN}])
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=5, tv_nsec=0}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=4, tv_nsec=999819792})
read(3, "100 94      \n", 13)           = 13
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=5, tv_nsec=0}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=4, tv_nsec=999997500})
read(3, "Unknown request.\nType 'help' for"..., 95) = 95
100        r           \r100\n", 17
)    = 17
write(1, "Unknown request.\nType 'help' for"..., 94Unknown request.
Type 'help' for more info.
Syntax Error: Missing separator between arguments
) = 94
write(1, "\n", 1
)                       = 1

So for vcl.state, it might actually be prudent to just drop the quoting.

vcl.inline works in varnishadm when using quotes for the VCL itself; strace looks like this:

$ strace -s 4096 varnishadm vcl.inline test2 '"vcl 4.0; backend default { .host = \"127.0.0.1\"; }"' auto
write(3, "vcl.inline", 10)              = 10
write(3, " ", 1)                        = 1
write(3, "test2", 5)                    = 5
write(3, " ", 1)                        = 1
write(3, "\"vcl 4.0; backend default { .host = \\\"127.0.0.1\\\"; }\"", 53) = 53
write(3, " ", 1)                        = 1
write(3, "auto", 4)                     = 4
write(3, "\n", 1)                       = 1
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=5, tv_nsec=0}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=4, tv_nsec=820625042})
read(3, "200 0       \n", 13)           = 13
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=5, tv_nsec=0}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=4, tv_nsec=999999250})
read(3, "\n", 1)                        = 1

However, vcl.inline also shows the same behaviour when the VCL state is supplied as a quoted string:

$ strace varnishadm vcl.inline test3 '"vcl 4.0; backend default { .host = \"127.0.0.1\"; }"' '"auto"'
...
write(3, "vcl.inline", 10)              = 10
write(3, " ", 1)                        = 1
write(3, "test3", 5)                    = 5
write(3, " ", 1)                        = 1
write(3, "\"vcl 4.0; backend default { .hos"..., 53) = 53
write(3, " ", 1)                        = 1
write(3, "\"auto\"", 6)                 = 6
write(3, "\n", 1)                       = 1
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=5, tv_nsec=0}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=4, tv_nsec=999999334})
read(3, "100 94      \n", 13)           = 13
ppoll([{fd=3, events=POLLIN}], 1, {tv_sec=5, tv_nsec=0}, NULL, 0) = 1 ([{fd=3, revents=POLLIN}], left {tv_sec=4, tv_nsec=999999292})
read(3, "Unknown request.\nType 'help' for"..., 95) = 95
fstat(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0), ...}) = 0
write(1, "Unknown request.\n", 17Unknown request.
)      = 17
write(1, "Type 'help' for more info.\n", 27Type 'help' for more info.
) = 27
write(1, "Syntax Error: Missing separator "..., 50Syntax Error: Missing separator between arguments
) = 50
write(1, "\n", 1
)                       = 1

I think I'll just adjust the library to drop the quotes around the values that do not explicitly require them.

[gutmensch]

@martin-helmich Ah thanks for the reminder, I actually tested this with varnishadm too (taking the generated values from kube-httpcache and applied them inline) and it worked as expected (like before with varnish 6.x), so yeah, I also wondered if related changes to varnishadm exist. Wireshark'ing sounds like a great idea to see what varnishadm does differently here. 🚀

[gutmensch]

Nice one :)

[gutmensch]

I just wondered isn't this exactly the case that should actually work according to the test here https://github.com/varnishcache/varnish-cache/blob/master/lib/libvarnish/vav.c#L389 but it is still triggered through this condition https://github.com/varnishcache/varnish-cache/blob/master/lib/libvarnish/vav.c#L195 ? Or is it because they don't have a test with e.g. 1 unquoted arg and 2 quoted args?

[martin-helmich]

Honestly, mostly for reasons of time, I gave up on digging through the Varnish source code; I recently tagged a recent version of this library with my changes from #29 because I needed a v7-compatible version for the mittwald/kube-httpcache project (see mittwald/kube-httpcache#158); my first tests with the current HEAD of this library were promising.

I'll go ahead and close this issue as completed, for now. If you happen to stumble across any further insights as to why Varnish behaves the way it does in this case, feel free to add them here. 😉