wodby / varnish Goto Github PK

First, thanks for the useful varnish image!

Just trying to add the bodyaccess module, but it appears the varnish modules can't be added as documented.

The environment variable is set, but the module is not available.

Also tried the manual import:

Using image wodby/varnish:4.1-4.3.2

21 Feb 10:27:18am UTC | Error: Child (32) not responding to CLI, killed it.
21 Feb 10:27:18am UTC | Error: Unexpected reply from ping: 400 CLI communication error (hdr)
21 Feb 10:27:35am UTC | Error: Child (32) not responding to CLI, killed it.
21 Feb 10:27:35am UTC | Error: Unexpected reply from ping: 400 CLI communication error
21 Feb 10:27:35am UTC | Error: Child (32) died signal=3
21 Feb 10:27:35am UTC | Error: Unexpected reply from ping: 400 CLI communication error (hdr)
21 Feb 10:27:36am UTC | Debug: Child cleanup complete
21 Feb 10:27:36am UTC | Debug: Child (145) Started
21 Feb 10:28:36am UTC | CLI result = 400
21 Feb 10:28:36am UTC | Error: Child (145) Pushing vcls failed:
21 Feb 10:28:36am UTC | CLI communication error (hdr)
21 Feb 10:28:36am UTC | Debug: Stopping Child
21 Feb 10:28:43am UTC | Info: Child (145) ended
21 Feb 10:28:43am UTC | Info: Child (145) said Child starts
21 Feb 10:28:43am UTC | Info: Child (145) said Child dies
21 Feb 10:28:43am UTC | Debug: Child cleanup complete

K8s doesn't restart Varnish container if health-check fails

WooCommerce may use 3 cookies for anonymous users:

• woocommerce_cart_hash
• woocommerce_items_in_cart
• wp_woocommerce_session_

We need a way to avoid caching for such cookies and probably for some paths:

• /cart
• /my-account
• /checkout
• /addons
• ?add-to-cart=
• /add-to-cart

Actually, I'm not sure about paths because cookies exclusion should be enough but I see a few examples of WooCommerce integration with both cookies and paths exclusion. It seems it should be tested additionally.

I suggest to implement a generic way to exclude caching for cookies and paths.

We have a problem with VARNISH_KEEP_ALL_COOKIES and wordpress preset. We set VARNISH_KEEP_ALL_COOKIES to 1 but none of there cookies except ones compliant with VARNISH_WP_PRESERVED_COOKIES are accessible in php.
We also tried a pure php page outside of wordpress code and still getting the save results, no cookies are preserved.

The current 4.x version no longer actively supported and installed from alpine 3.6 repository that reaches EOL in May. We also need to support version 6.x as well.

Where I can find any instructions about how to test the container work?
I want to configure it with Drupal 8 and I got the message

Varnish at varnish:6081 not responding.

Hello,
could be possible to add mod dynamic (https://github.com/nigoroll/libvmod-dynamic)?
It would be very useful if using replicas.
Thank you very much in advance
Francesco

We have to consider making client VCKEY-* cookies available on a backend to have an ability to contextualize cachable request.

We're getting 1st Apple M1 devices and we saw that Varnish image has no arm build.

The url rewrite on line 18 of templates/includes/static.vcl.tmpl causes 404 image responses from Drupal when an anonymous user visits a page with responsive image styles.

This is due to the itok parameter being removed by line 18. The ImageStyleDownloadController processes the request in Drupal and passes back a 404 as there is no valid itok.

Steps to reproduce:

• Stock Docker4Drupal with Varnish and the VARNISH_CACHE_STATIC_FILES environment variable set.
• Drupal configured to cache (/admin/config/development/performance).
• The following modules enabled: responsive_image, media, media_library.
• Add a media field to a content type.
• Configure a responsive image style - My configuration is Bartik Breakpoint groups with Select a single image style for each breakpoint.
• Change default media display to use a responsive image style.
• Change media field display (if needed) to use display with responsive image style.
• Add a node with an image and while logged in do not resize browser (this would cause images to generate as varnish will pass back response without altering).
• Ensure page is cached via headers.
• View varnish version of page and resize - any styles that were not accessed by authenticated user will return 404.

Commenting out line 18 of the include resolves this issue.

On my site I'm using varnish_tag 6.0-4.0.0 with varnish_purger module on a Drupal installation.
With 4.1-3.0.7 I've no problem, I've update to latest stable release and for every request I've this error in my watchdog entries:

purger_varnish_d671be62a8: item failed due GuzzleHttp\Exception\ClientException, details (JSON): {"msg":"Client error: BAN http://varnish:6081/resulted in a405 Not allowed from 172.27.0.3 response: <!DOCTYPE html> <html> <head> <title>405 Not allowed from 172.27.0.3<\/title> <\/head> <body> <h1>Error 405 (truncated...) ","uri":"http:\/\/varnish:6081\/","method":"BAN","guzzle_opt":{"http_errors":true,"connect_timeout":1,"timeout":1},"headers":{"user-agent":"varnish_purger module for Drupal 8.","cache-tags":"config:system.menu.main","x-vc-my-purge-key":"secret"}}

This is the varnish section of my docker-compose:

varnish:
    image: wodby/varnish:$VARNISH_TAG
    container_name: "${PROJECT_NAME}_varnish"
    depends_on:
      - nginx
    environment:
      VARNISH_SECRET: secret
      VARNISH_PURGE_KEY: secret
      VARNISH_BACKEND_HOST: nginx
      VARNISH_BACKEND_PORT: 80
      VARNISH_CONFIG_PRESET: drupal
      VARNISH_PURGE_EXTERNAL_REQUEST_HEADER: X-Real-IP
      VARNISH_DRUPAL_PRESERVED_COOKIES: "SESS[a-z0-9]+|SSESS[a-z0-9]+|NO_CACHE+|Drupal.visitor.legal_hash+|Drupal.visitor.legal_id"

On the module configuration I've added an header "X-VC-My-Purge-Key" with value "secret".

Is it the right way?

Thanks a lot,
Sergio

The changes in #24 have disabled caching of all static files for authenticated users.

While it is good that Drupal private files (/system/files/*) should not be cached for authenticated users, there are lots of other static files that should still be safe candidates to cache. e.g. public files, css, js, images included in theme or module folders etc.

It would be awesome to be able to cache some POST requests (ex: product filters etc) and I get why they are bypassed by default. But would you be open to the idea of adding a header that would allow caching of POST?

Best regards

Hi,

On our dev environments, we've got the following drupal setting set to true on our development.services.yml :
http.response.debug_cacheability_headers: true

With some huge content types, create/edit forms reponses sends an extremly large X-Drupal-Cache-Tags header, which breaks varnish with the following error :

BogoHeader Header too long: X-Drupal-Cache-Tags

We've found a dirty workaround by setting VARNISHD_THREAD_POOL_MAX to 1000 -p http_resp_hdr_len=65536 -p http_resp_size=98304 since VARNISHD_THREAD_POOL_MAX is the last used variable.

Would it be possible to add VARNISHD_RESP_HDR_LEN and VARNISHD_HTTP_RESP_SIZE to the available variables please ?

Thank you !

I am facing some real issues while ruining wodby/varnish:4-4.6.4 container - Varnish does not compile.
Here is the error from logs:
wodby/varnish:4-4.6.4
Error: Running VCC-compiler failed, signal 5 VCL compilation failed
The environment for me:
docker run -it --rm -e VARNISH_BACKEND_HOST=localhost -e VARNISH_BACKEND_PORT=8080 -e VARNISH_CONFIG_PRESET=drupal wodby/varnish:4-4.6.4

We should consider adding such functionality to varnish image. It will allow us to have different versions of content for the same URL. Different versions for a country, for a currency, for a device and whatever a user wants. It's generic, flexible and easy to use.

Also we may ditch such hardcode https://github.com/wodby/varnish/blob/master/4/templates/includes/mobile.vcl.tmpl.

I'm having trouble with Drupal private files returning 403s/404s to authenticated users (including admins).

I've added the example regex for system/files in the README in VARNISH_DRUPAL_EXCLUDE_URLS but no luck.

I noticed that cee55bd addressed this but the order was later reversed in e5c91eb. Is there additional config that solves this?

Thanks for these well-documented Docker images!

Using the Varnish Purge module with the following headers set:

Cache-Tags: [invalidation:expression]
X-VC-Purge-Key:

never seems to invalidate.

I believe this is due to the wodby image vcl looking for the Cache-Tags header when drupal is returning X-Drupal-Cache-Tags.
The attached patch seems to fix the issue.

diff --git a/templates/includes/purge.vcl.tmpl b/templates/includes/purge.vcl.tmpl
index e30e2b1..aef0de8 100755
--- a/templates/includes/purge.vcl.tmpl
+++ b/templates/includes/purge.vcl.tmpl
@@ -48,8 +48,8 @@ sub vcl_recv {
         {{ end }}
         {{ end }}
 
-        if (req.method == "BAN" && req.http.Cache-Tags) {
-            ban("obj.http.Cache-Tags ~ " + req.http.Cache-Tags);
+        if (req.method == "BAN" && req.http.X-Drupal-Cache-Tags) {
+            ban("obj.http.X-Drupal-Cache-Tags ~ " + req.http.X-Drupal-Cache-Tags);
         }
 
         if (req.http.X-VC-Purge-Method) {

Hi @csandanov

Was just reading up some modifications. Haven't really use them (shame on me!), but before i forget: about this line:

https://github.com/wodby/varnish/blob/master/4/templates/defaults/vcl_pipe.vcl.tmpl#L22

That Connection="Close" should not be commented out, should it?

Thanks!!

I have my own deployment using the wodby images. And I have uncommented the Varnish container in my docker-compose.yml and followed the directions found here: https://wodby.com/docs/stacks/wordpress/containers/#varnish

My settings look like this:

Looking at the logs for the varnish container, it never logs activity and do not see the X-VC-Cache being set to the client. I am currently setting the IP field to the name of the container, is this correct?

Currently compilation fails due to libexecinfo pkg removed from alpine 3.17:

#9 58.43 checking whether we have support for visibility attributes... yes
#9 58.48 checking for setproctitle... no
#9 58.52 checking for library containing backtrace... no
#9 58.60 configure: error: Could not find backtrace() support

ref DataDog/dd-trace-php#1824, dmlc/xgboost#8595, awslabs/aws-lambda-cpp#124

If VARNISH_DRUPAL_PRESERVED_COOKIES is set, required Session and No_Cache cookies are also gets stripped contrary to documentation. Also, it's not clear how to handle multiple cookies with VARNISH_DRUPAL_PRESERVED_COOKIES on composer.yml

dockercompose:
VARNISH_DRUPAL_PRESERVED_COOKIES: "example"
actual varnish container "preset.vcl":
`/ # cat /etc/varnish/preset.vcl
sub vcl_recv {

# Pass through any administrative or AJAX-related paths.
if (req.url ~ "^(/update\.php|/([a-z]{2}/)?admin|/([a-z]{2}/)?admin/.*|/([a-z]{2}/)?system/files/.*|/([a-z]{2}/)?flag/.*|.*/ajax/.*|.*/ahah/.*)$") {
    return (pass);
}

if (req.url ~ "(^/([a-z]{2}/)?batch)") {
    return (pipe);
}

# Remove all cookies that Drupal doesn't need to know about. We explicitly
# list the ones that Drupal does need, the SESS and NO_CACHE. If, after
# running this code we find that either of these two cookies remains, we
# will pass as the page cannot be cached.
if (req.http.Cookie) {


    # 1. Append a semi-colon to the front of the cookie string.
    # 2. Remove all spaces that appear after semi-colons.
    # 3. Match the cookies we want to keep, adding the space we removed
    #    previously back. (\1) is first matching group in the regsuball.
    # 4. Remove all other cookies, identifying them by the fact that they have
    #    no space after the preceding semi-colon.
    # 5. Remove all spaces and semi-colons from the beginning and end of the
    #    cookie string.
    set req.http.Cookie = ";" + req.http.Cookie;
    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
    set req.http.Cookie = regsuball(req.http.Cookie, ";(example)=", "; \1=");
    set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

    if (req.http.Cookie == "") {
        # If there are no remaining cookies, remove the cookie header. If there
        # aren't any cookie headers, Varnish's default behavior will be to cache
        # the page.
        unset req.http.Cookie;
    }
    else {
        # If there is any cookies left (a session or NO_CACHE cookie), do not
        # cache the page. Pass it on to Apache directly.
        return (pass);
    }
}

}`

It seems we should add support of GeoIP switching in a generic fashion. Here is an example of WooCommerce currency switching for WordPress.

UPD1: It could be implemented in a generic way as described in #9, so we do not need to hardcode any special.

I noticed possible issue in varnish config:

1. https://github.com/wodby/varnish/blob/master/templates/defaults/vcl_recv.vcl.tmpl#L34

"^[; ]+|[; ]+$"

a regexp consists of 2 identical parts [; ]

2. It seems cookies does not affect hash here, so this block looks useless. I tested with FooCookieName=BarValue cookie and it does not affect on cache hash (VARNISH_KEEP_ALL_COOKIES was not enabled).

Hi, can you add vmod_digest to this image?

I tried compiling it after but from my understanding that is not possible, because you delete all the relevant header files. See the README.

Otherwise the installation seems to be very similar to vmod_geoip.