the-allanc / vcl-cache-validation Goto Github PK

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This will need to be done fairly early in the process to ensure everything works as expected.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This has really disheartened me a bit. Varnish won't pass on validation headers on to the backend server (well, at least If-Modified-Since). It took a while to figure out, but it explicitly strips out If-Modified-Since headers in cache_esi_deliver.c.

(Update: And If-None-Match headers too. Crap. Can't figure out where the code to do this is though.)
(Further update: Actually, all known validating headers seem to be stripped out.)
(More debugging later: I think it's all defined in http_headers.h. That's a tidy way to do it.)

So - Varnish strips out these headers, so I can't pass them along to the backend server. I think I'll have to find what the best way is to transmit these headers in the request (maybe just adding X- prefixes to the lot of them).

On a sidenote, there is the experimental-ims branch of Varnish which provides full support for this sort of thing, which is probably a nicer alternative than using any scripts that I write here.

However, that branch appears to be at least a year out-of-date, as well as not being entirely clear whether the API it uses will change or not. Varnish 4 might be out relatively soon, but it's still not clear that it will incorporate support for it or not. So I'll persist on this route for now.

One of my ideas to implement cache-validation. To be expanded in more detail later.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

Another one of my ideas to implement cache validation. To expanded later.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

Some of the approaches to deal with Varnish centre around the fact updating a cached document is difficult.

However, this page seems to indicate that req_always_miss will update the cache. The question is, does this cause the content to be purged before vcl_fetch has a chance to examine the beresp object?

If the content is purged, then we're still in the same boat - we can't test to see if a document has been modified or not without sacrificing the original cached content. But if we can examine the response without fear of losing the original cached document, then that would simplify the script somewhat and allow us to drop a number of various hacks which I've come up with.

I would have thought that during my research that I would have come across this already, so I do find it a bit surprising. Anyway, let's check.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

Another one of my approaches to perform cache validation. To be expanded later.

The current prototype web server which I have experimented with uses CherryPy. Before I being work on the test suite, I'd like to move to using Go - just to give myself a chance to experiment with it. May work or fail, but let's give it a try!

Not a clear idea as of yet, but look into this as an approach to cache validation.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

Possibly the most efficient, yet least complete way of handling caching - to be expanded on a later point.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

This directive should be documented, in terms of behaviour, and how (or whether) it is supported by the scripts in this repository.

If clients have their own cached representation of a document, and attempt to use validation, the scripts in this package should allow that to pass through and not interfere with that process.