The final stage in speeding up my blog was to add some serious caching to the front of it. This may even have been overkill, because it was already pretty swift under nginx/php-fpm, but cutting out the database connections would speed it up even more.
I had a quick go with the W3 Total Cache WordPress plug-in, but it seems rather biased to running Apache (which I’m not) and I experienced some strange errors that I failed to immediately fix. Rather than wrestle with it, or try other WordPress caches, I decided to get to grips with Varnish. This is something I’d been meaning to do for ages, and of course it isn’t limited to WordPress – Varnish is a fabulous caching solution for whatever site you’re building.
The challenge with deploying Varnish to any site is to ensure you cache as much as you can while ensuring dynamic content is fresh when required. With WordPress as an example, the cases are fairly obvious. While logged in to the admin screens you don’t want to cache anything, but the outside world reading your posts should get cached pages whenever possible. In their simplest forms, these cases almost work out of the box with Varnish, but there are a few gotchas. There is also scope to cache more aggressively if you choose to, and most importantly you’ll want to ensure that when content is altered, the changes are visible to everyone immediately.
Cookie gotcha no.1
By default, if you send a cookie to Varnish it assumes quite sensibly that you are in some way expecting dynamic content from the back end. e.g. you may be an admin user, logged in to the site. Of course Varnish can’t actually know what the cookie is for, so when your browser sends a completely redundant Google Analytics UTM cookie, you’ll miss the cache. On this site, that means Varnish would only serve you a maximum of one cached page per visit.
Anonymous session cookies
If a visitor comments on a WordPress site, they are pseudo-logged-in by way of a simple cookie exchange. This means the site can do niceties like pre-populate comment forms for commenting again. These niceties are exactly that, so if you overlook this functionality and block those cookies you can serve more cached pages. In my case, I am happy to make this trade.
The following snippet from my configuration shows how I’ve maximised anonymous user caching while ensuring admin users always get dynamic content
sub vcl_recv {
# ...
# admin users always miss the cache
if( req.url ~ "^/wp-(login|admin)" || req.http.Cookie ~ "wordpress_logged_in_" ){
return (pass);
}
# ignore any other cookies
unset req.http.Cookie;
return (lookup);
}
Purging when content changes
HTTP defines a wealth of cache control headers that allow applications and clients to specify what is to be cached and for how long, etc.. but there is no standard way to tell a cache that certain content is to be purged. This is the main challenge of deploying Varnish to a site like WordPress. A post may be updated, or a user may comment. You’ll want to purge any affected pages from the cache immediately, regardless of their original expiry. Putting that business logic into your VCL configuration would be hugely impractical – it’s really your application’s jurisdiction.
I looked at various purging techniques, and by far the best approach in my opinion is for your application to talk directly to the Varnish administration CLI which listens on the network on a different port to the cache itself. As long as your backend can access your front end by a secure, internal address, you can execute commands on all your Varnish front ends any time you want from your application.
To achieve this, I first had a look at the wp-varnish plug-in which uses the HTTP PURGE method. I don’t like this approach at all; it’s very limited, too closely coupled to the VCL config, and also this plug-in doesn’t purge all pages relating to changes, such as tag and archive indexes.
So, I rolled my own
I started by writing a VarnishAdminSocket class, which talks to Varnish from PHP. This was pretty easy to do, because the varnishadm program uses a very basic text-based protocol. This isn’t designed for WordPress, this is designed for use in any PHP application.
My php-varnish toolkit is on Github. It includes a WordPress plug-in which I am using on this site and you can see the VCL configuration I am using too.
Hi Tim, I see your varnishadminsocket class on your github,
Is it need to install ? how to use it ?
thx,
If you remove the ^/ it’ll work in sub-directories.
if( req.url ~ “wp-(login|admin)” || req.http.Cookie ~ “wordpress_logged_in_” ){
return (pass);
}
It also seems to fix some login redirect issues where my port number was getting added to the URL.
Looking at the “^” in the regular expression, I’d guess it won’t work for sub-directories. You’ll have to modify the expression a little.
This is working great for any install I have at the root of a site. But I’m having issues logging into a WordPress install that is in a sub-directory. WordPress keeps telling me I don’t have cookies enabled; but I do.
Should the code above allow cookies for sub-directories as well?
Geir: The plugin is using the appropriate action hooks to purge caches when comments are posted, edited or deleted.
Gregg: A single hostname pattern is configurable in the Varnish Admin page. You can alter this to catch all your hosts, or leave blank to purge only by URL path.
Do your configuration files, and the wordpress plugin work with multisite?
Hi
How are you dealing with comments and updates, im refering to.
If a user posts a comment, it wount get published untill a you publih a new story and cache get updated?
Enjoy the native php binding http://pecl.php.net/package/varnish … with this you can do even more, read varnish log for ex
I’ve updated the Git repo with some fixes including support for Varnish 3, and also a bug with the manual purge action in the Worpress admin screen.
Even though you didn’t know, from now on there is a varnish php extension. Read this
http://belsky.info/archives/30-Varnish-now-for-PHP-and-from-PHP.html
Works well to ban urls, set varnish params and more.
Cheers
Tim I am trying to use your Varnish purge plugin but can’t get it to work, I get the following errors:
Purge results
127.0.0.1:6082
Error: Authentication required; see VarnishAdminSocket::set_auth
In the configuration box that says “Specify authentication secrets (url encoded)” I am just pasting the secret from /etc/varnish/secret is this correct? I’m not sure what you mean by (url encoded)
I would appreciate any help on this, thank you.
Hi
Thanks for the great VCL which I’ve borrowed. I can’t get the plugin to work though, so I’ve added a purge request to the VCL. When I install the plugin in WP3.2 I just get a menu option Settings/Varnish Admin but nothing else.
I have a problem in that on my homepage the wordpress admin bar doesn’t work load and pages don’t flush. I tried to combat this by adding this to your VCL:
sub vcl_recv {# don't cache homepage if logged in
if( req.url ~ "^/" || req.http.Cookie ~ "wordpress_logged_in_" ){
return (pass);
}
but it didn’t work? Can you help?
Thanks in advance for any help
EB
Some of the Drupal module code regarding the challenge answer for 2.1.x may be of use to you: http://drupal.org/project/varnish (grab the latest 6.x-1.x-dev version).
This VCL example may be useful for you: http://mclear.co.uk/2011/02/27/updated-varnish-wordpress-vcl/
Is there a way around the challanging answer that is required in Varnish 2.1.x other than disabling the secret?
Great to hear that Arjen – I shall get in touch
Hi Tim,
Thanks for this is a great plugin. We are running it on thenextweb.com now. If you like, I have some enhancements for you. Like authenticated varnishadm, finer grained purging and some tips for the plugin.
Please contact me, would love to contribute to this project,
Best
Arjen Schat, co-founder thenextweb.com
I’m sure W3 is fine, I just preferred to learn something new rather than tackle the problems I was having.
Not sure I want the extra work of contributing a repackaged version to WordPress.org right now, but I may do in future.
I see your point about Github, I guess it only generates tarballs when it needs to. A shortcut link would be useful.
Hi Tim,
Thanks for doing this! I have been looking for a decent reference VCL for WordPress for a while :-)
Using your plug-in too, although I think my w3 cache plugin is purging fine.
Will you be submitting your plugin to WordPress.org? Can you convince GIT to make available a simple ZIP/TAR download of the latest version?
Axel.