Setup Varnish 4 with Wordpress W3 Total Cache on CentOS 6

Optimizing WordPress Site

Optimizing wordpress site can become tricky, if yo do not maintain it properly. Before starting optimization you need to identify your tools and log current page speed statistics of targeted site. This is very important to analyze and present improvements later on.

Following tools can be used to monitor Performance:
1. webpagetest.org
2. gtmetrix.com

We will use webpagetest.org and save results for few targeted pages to be optimized. This is a right time when you save some results for future reference.

Install and setup W3 Total Cache

Install W3TC from it's plugin website. Its a powerful caching plugin for wordpress.
A detailed guide to setup W3 Total Cache is available here. You can follow it or any other tutorial available online.

Use APCu for Dedicated or VPS Server. Following resources can help you installing APCu.
http://www.joomlaworks.net/blog/item/153-install-apc-apcu-on-a-whm-cpanel-server
https://stackoverflow.com/a/31558558/1897969

Install Varnish

Check if epel-release is installed:
rpm -qa | grep epel 
yum install epel-release
rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-4.1.el7.rpm
(or: rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-4.1.el6.rpm for RHEL6)
yum install varnish

Setup Varnish

For setting up varnish we need to update it's basic configuration and how our proxy will act for each request coming. For this you need to edit following files.
vim /etc/varnish/default.vcl
You can use this gist to update your vcl file.
# A heavily customized VCL to support WordPress
# Some items of note:
# Supports https
# Supports admin cookies for wp-admin
# Caches everything
# Support for custom error html page
vcl 4.0;
import directors;
import std;
# Assumed 'wordpress' host, this can be docker servicename
backend default {
.host = "127.0.0.1";
.port = "8080";
.connect_timeout = 1s;
.first_byte_timeout = 30s;
.between_bytes_timeout = 5s;
.max_connections = 100;
}
acl purge {
"localhost";
"127.0.0.1";
}
sub vcl_recv {
if (req.url ~ "phpmyadmin") {
return (pass);
}
if (req.url ~ "/wp-(login|admin)") {
return (pass);
}
# pass search URL
if ( req.url ~ "^/\?s" ) {
return (pass);
}
# pass search URL
if ( req.url ~ "/search/" ) {
return (pass);
}
# Only a single backend
set req.backend_hint= default;
# Setting http headers for backend
set req.http.X-Forwarded-For = client.ip;
set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
# Unset headers that might cause us to cache duplicate infos
unset req.http.Accept-Language;
# Normalize Vary Header to keep different cache versions for desktop and mobile;
if (req.http.User-Agent ~ "(Mobile|Android|iPhone|iPad)") {
set req.http.User-Agent = "mobile";
} else {
set req.http.User-Agent = "desktop";
}
if (req.http.Authorization || req.method == "POST") {
return (pass);
}
# Allow purge from allowed IPs only
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
# return(synth(405,"Not allowed."));
}
return (purge);
}
# pass wp-admin urls
if (req.url ~ "(wp-login|wp-admin|login|logout)" || req.url ~ "preview=true" || req.url ~ "xmlrpc.php") {
return (pass);
}
# drop cookies and params from static assets
if (req.url ~ "\.(gif|jpg|jpeg|swf|ttf|css|js|flv|mp3|mp4|ico|png|woff|woff2)(\?.*|)$") {
unset req.http.cookie;
set req.url = regsub(req.url, "\?.*$", "");
}
# drop tracking params
if (req.url ~ "\?(utm_(campaign|medium|source|term)|adParams|client|cx|eid|fbid|feed|ref(id|src)?|v(er|iew))=") {
set req.url = regsub(req.url, "\?.*$", "");
}
# Remove the "has_js" cookie
set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
# Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");
# Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-1=[^;]+(; )?", "");
# Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^ *$") {
unset req.http.cookie;
}
# pass wp-admin cookies
if (req.http.cookie) {
if (req.http.cookie ~ "(wordpress|wordpress_)") {
return(pass);
} else {
unset req.http.cookie;
}
}
}
sub vcl_backend_response {
# retry a few times if backend is down
if (beresp.status == 503 && bereq.retries < 3 ) {
return(retry);
}
# Set TTL Explicitly if ignored from server
if (beresp.ttl < 120s) {
set beresp.ttl = 120s;
unset beresp.http.Cache-Control;
}
if (beresp.status == 302){
set beresp.http.X-Cacheable = "NO: 302";
set beresp.uncacheable = true;
return (deliver);
}
# Remove setcookies for static resources
if (bereq.url ~ "\.(gif|jpg|jpeg|swf|ttf|css|js|flv|mp3|mp4|ico|png|woff|woff2)(\?.*|)$") {
unset beresp.http.set-cookie;
unset beresp.http.cookie;
}
# unset set-cookies from backendresponse if url is not for login users
if (!(bereq.url ~ "(wp-login|wp-admin|login|logout)")) {
set beresp.http.X-UnsetCookies = "TRUE";
unset beresp.http.set-cookie;
}
if (bereq.http.Cookie ~ "(wordpress|wordpress_logged_in)") {
# if we get a session cookie...caching is a no-go
set beresp.http.X-Cacheable = "NO:Got Session";
set beresp.uncacheable = true;
set beresp.ttl = 0s;
return (deliver);
} elsif (beresp.http.set-cookie) {
# You don't wish to cache content for logged in users
set beresp.http.X-Cacheable = "NO:Set-Cookie";
set beresp.uncacheable = true;
set beresp.ttl = 0s;
return (deliver);
} elsif (beresp.http.Cache-Control ~ "private") {
# You are respecting the Cache-Control=private header from the backend
set beresp.http.X-Cacheable = "NO:Cache-Control=private";
set beresp.uncacheable = true;
set beresp.ttl = 0s;
return (deliver);
} else {
# Varnish determined the object was cacheable
set beresp.http.X-Cacheable = "YES";
# Remove Expires from backend, it's not long enough
unset beresp.http.expires;
# Set the clients TTL on this object
set beresp.http.cache-control = "max-age=900";
# Set how long Varnish will keep it
set beresp.ttl = 365d;
# marker for vcl_deliver to reset Age:
set beresp.http.magicmarker = "1";
}
# setting ttl to 1h for backendresponse if url is not for login users
if (!(bereq.url ~ "(wp-login|wp-admin|login|logout)")) {
set beresp.ttl = 1h;
}
# long ttl for assets
if (bereq.url ~ "\.(gif|jpg|jpeg|swf|ttf|css|js|flv|mp3|mp4|ico|png|woff|woff2)(\?.*|)$") {
set beresp.http.cache-control = "max-age=604800";
set beresp.ttl = 365d;
}
set beresp.grace = 1h;
}
sub vcl_hash {
# If the client supports compression, keep that in a different cache
if (req.http.Accept-Encoding) {
hash_data(req.http.Accept-Encoding);
}
}
sub vcl_backend_error {
# display custom error page if backend down
if (beresp.status == 503 && bereq.retries == 3) {
synthetic(std.fileread("/path/to/503.html"));
return(deliver);
}
}
sub vcl_synth {
# redirect for http
if (resp.status == 750) {
set resp.status = 301;
set resp.http.Location = req.http.x-redir;
return(deliver);
}
# display custom error page if backend down
if (resp.status == 503) {
synthetic(std.fileread("/path/to/503.html"));
return(deliver);
}
}
sub vcl_deliver {
# oh backend is down
if (resp.status == 503) {
return(restart);
}
if (resp.http.magicmarker) {
# Remove the magic marker
unset resp.http.magicmarker;
# By definition we have a fresh object
set resp.http.age = "0";
}
if (obj.hits > 0) {
set resp.http.X-Cache = "cached";
} else {
set resp.http.X-Cache = "uncached";
}
set resp.http.Access-Control-Allow-Origin = "*";
}
sub vcl_hit {
if (req.method == "PURGE") {
return(synth(200,"OK"));
}
}
sub vcl_miss {
if (req.method == "PURGE") {
return(synth(404,"Not cached"));
}
}
sub vcl_purge {
return (synth(200, "Purged"));
}
view raw wp-default.vcl hosted with ❤ by GitHub 

vim /etc/sysconfig/varnish (Ubuntu path /etc/default/varnish)
You can use this gist to update your varnish configuration.
# Configuration file for Varnish Cache
#
# /etc/init.d/varnish expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#
# Maximum number of open files (for ulimit -n)
NFILES=131072
# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000
# Maximum number of threads (for ulimit -u)
NPROCS="unlimited"
# Maximum size of corefile (for ulimit -c). Default in Fedora is 0
# DAEMON_COREFILE_LIMIT="unlimited"
# Init script support to reload/switch vcl without restart.
# To make this work, you need to set the following variables
# explicit: VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_ADDRESS,
# VARNISH_ADMIN_LISTEN_PORT, VARNISH_SECRET_FILE.
RELOAD_VCL=1
# Main configuration file.
VARNISH_VCL_CONF=/etc/varnish/default.vcl
#
# Default address and port to bind to
# Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
VARNISH_LISTEN_PORT=80
#
# Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082
#
# Shared secret file for admin interface
VARNISH_SECRET_FILE=/etc/varnish/secret
#
# The minimum number of worker threads to start
VARNISH_MIN_THREADS=50
#
# The Maximum number of worker threads to start
VARNISH_MAX_THREADS=1000
#
# Cache file size: in bytes, optionally using k / M / G / T suffix.
VARNISH_STORAGE_SIZE=512M
#
# Backend storage specification
VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
#
# Default TTL used when the backend does not specify one
VARNISH_TTL=120
#
# This setting is added to avoid fetch failed error.
VARNISH_RESPONSE_HEADER_LENGTH=32000
#
# DAEMON_OPTS is used by the init script.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
-f ${VARNISH_VCL_CONF} \
-T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
-p thread_pool_min=${VARNISH_MIN_THREADS} \
-p thread_pool_max=${VARNISH_MAX_THREADS} \
-p http_resp_hdr_len=${VARNISH_RESPONSE_HEADER_LENGTH} \
-S ${VARNISH_SECRET_FILE} \
-s ${VARNISH_STORAGE}"
view raw varnish hosted with ❤ by GitHub

Setup WebServer

Move Apache from port 80 to some other port, lets use port 8080.
  • Change apache listen port from 80
    • Listen 8080
  • Change the VirtualHost line from *:80 to *:8080
    • <virtualhost *:8080>
Serve all public requests on port 80 from varnish
  • Make sure to use port 80 in varnish configuration
  • Restart apache and varnish servers to take effect

Analysis

Now you need to take new readings using the same tools mentioned above and see the difference. For debugging the cached information you can see the Headers, set by our varnish configuration (default.vcl).

Varnish Purge & Rebuild Cache

After implementing caching with any tool you must need to keep a system in place so the cache can expire and get new content on time. For this there can be many stretegies and there are also some plugins available and you can also configure W3Total Cache to purge varnish cache on time.
But here we will soon add a mechanism to purge and rebuild all varnish cache after a designated time (e.g. 24 hours).

Conclusion

This is how you can setup Varnish 4.1 on CentOS 6, we will further add about Varnish 5 configuration later on. You can add comments here if you face any difficulty in setting it up.

Comments

Post a Comment

Popular posts from this blog

PHP Frameworks

PHP Frameworks are the tools which helped developers focus on their task (i.e. business logic) more and less worried about structural behavior of the application. But for most of the developers it is difficult to choose one of the Frameworks available for the application, so as I have been in dilema when it comes to selection of a Framework. I have come across to Zend Framework (Older version) when I first got a chance to work on any PHP Framework. As they described it as [“use-at-will” design], we have used some of the components rather than whole framework, it is claimed to be the most popular used framework. My growing opinion was a bit different though, it really is a big Framework with lots of configuration required and very heavy for the application. Then I have gone through the YII framework but could not hold onto it, the key feature I endorse upon was ORM but I won't able to use it in professional application. I had to switch to the framework about which people arou...
Read more

Upgrade Magento from 1.7 to 1.9

Image
Upgrading magento involves upgrading codebase, data migration, extensions & themes migration and your custom code . You can follow these steps to upgrade your magento installation from 1.7.x to 1.9.x. For this we have an approach to change directory names at last step to avoid any conflicts and make rollback strategy simple and quick. Assumptions Directory name for current 1.7 store used in this tutorial is store17, replace all occurrences of it with correct directory name. Steps to Upgrade First we need to disable compilation and cache for our current store: Disable Magento Compilation from System > Tools > Compilation > Disable Disable Magento Cache from System > Cache Management > Select All > Action: Disable > Submit Turn on Maintenance mode for live store, take database backup and Extract 1.9 files for installation Install 1.9 Install 1.9 in separate directory but keep the database configuration same. It means that when it is a...
Read more