Articles

Umesh

Umesh's Articles

Umesh

Umesh

July 28 2013

Anahita + Varnish

I've been trying to setup mysite (2CPU 2GB Ubuntu 15.10 on Rackspace) with #Varnish. Post configuration, I must say, results are amazing. Now my entire setup includes Cloudflare CDN + Varnish 4.0 + Nginx 1.9.3 + APC 4.0.7 + PHP 5.6.11 (PHP-FPM)+MySQL. 

Please note that below content assumes that you already know about varnish and have installed Varnish on your system.

I published this article originally in 2013. As Varnish has changed the VCL syntax in version 4.0, earlier version was no longer useful hence I decided to update this page. 

Below is the VCL configuration works seamlessly with Cloudfare and PHP/Nginx combination. I've not tried this however my guess is this VCL should work with other combination as well (e.g. Apache & PHP or without CDN)

NGINX

Prefebaly you should run your server with localhost or 127.0.0.1 server. you can set this in nginx configuration file using server and listen tag. You also need to run additional server on different port (e.g. 81) to server static files from cookieless domain (image, css, js etc)

server { listen 127.0.0.1:80; }

server { listen 127.0.0.1:81; }

Varnish command like configuration

Locate varnish file under /etc/default folder. Change your alternative 2 values as 

 DAEMON_OPTS="-a :80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -s malloc,64m"

-a option enables Varnish to listen on incoming HTTP requests on port 80. 

-s option allocates memory for varnish cache. You can explore file option instead of malloc if you have limited available memory, however this is going to impact overall response time.

Update default.php 

Varnish is useful if you can cache dynamic pages. To get this done , locate default.php in /template/base/html folder and add below code just before HTML tag.

$user = JFactory::getUser(); if (!$user->guest) { JResponse::setHeader('X-Logged-In', 'True', true); } else { JResponse::setHeader('X-Logged-In', 'False', true);

Update default.vcl 

Locate default.vcl in /etc/varnish folder and add following code

vcl 4.0;

backend staticcontent {

    .host = "127.0.0.1";

    .port = "81";

}

backend default {

    .host = "127.0.0.1";

    .port = "80";

}

backend webfarm {

    .host = "127.0.0.1";

    .port = "80";

}

sub vcl_recv {

if (req.url ~ "\.(css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|ico|js)$") {

        unset req.http.cookie;

        set req.backend_hint = staticcontent;

        return (hash);

}

# If user sends an http POST, pipe to backend

if (req.method == "POST" || req.url ~ "edit") {

set req.backend_hint = default;

return(pipe);

}

# http authenticated sessions are piped

if (req.http.Authenticate || req.http.Authorization) {

set req.backend_hint = default;

return(pipe);

}

# if the user is coming FROM the login page, pipe to backend

if (req.http.referer ~ "(?i)(com_user|login)") {

set req.backend_hint = default;

return(pipe);

}

if (req.http.Cookie) {

# Remove has_js and CloudFlare/Google Analytics __* cookies.

set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");

# Remove a ";" prefix, if present.

set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");

}

if (req.http.Cookie == "") {

        unset req.http.Cookie;

}

set req.http.X-Forwarded-For = client.ip;

}

sub vcl_backend_response {

set beresp.grace = 5m;

if (bereq.url ~ "/articles/") {

        set beresp.ttl = 7d;

}

# discard backend setcookie unless it equals the following

if (!bereq.url ~ "(?i)(login|com_user|user|logout)") {

unset beresp.http.Set-Cookie;

}

if (bereq.http.referer ~ "(?i)(com_user|login|logout)") {

set bereq.backend = default;

set beresp.uncacheable = true;

return(deliver);

}

if (beresp.http.X-Logged-In ~ "False"){

set bereq.backend = webfarm;

if(bereq.url ~ "/articles/") {

        unset beresp.http.set-cookie;

        set beresp.ttl   = 86400s;

}

return(deliver);

}

if (beresp.http.X-Logged-In ~ "True"){

set bereq.backend = default;

set beresp.uncacheable = true;

return(deliver);

}

if (bereq.http.Authenticate || bereq.http.Authorization) {

set bereq.backend = default;

set beresp.uncacheable = true;

return(deliver);

}

}

sub vcl_deliver {

 if (obj.hits > 0) {

 set resp.http.X-Varnish-Cache = "HIT";

 set resp.http.X-Varnish-Hits  = obj.hits;

  }

 else {

 set resp.http.X-Varnish-Cache = "MISS";

 }

 return (deliver);

 }

Restart Services 

Follow below commands to enable your configuration changes 

sudo service nginx restart 

sudo service varnish restart

Verification 

There are multiple ways you can verify your cache setup. Try Pingdom tool and verify X-Varnish-Cache variables in HTTP header

You can also use varnishstat or varnishhist tools to verify cache usage. In varnishhist, '#' indicates cache miss and '|' indicated cache hit.

6 people liked this
Roni Mmi
Roni Mmi
July 29 2013 Permalink
Anahita + Litespeed + Varnish + Cloudflare Pro works well...
Andy Nash
Andy Nash
September 25 2013 Permalink
Great work Umesh, hope to try this out soon...

Have you tested with dynamic content to be sure this config doesn't cache anything it isn't supposed to? For example when caching an avatar image, ifn someone updates their image will it take some time for the cached avatar image to go stale or will this be picked up immediately?

I don't suppose you have a public site we could see? Or if not, perhaps you could use GT Metrix (with authentication) to measure the performance difference with/without Varnish?

Am I right in thinking this only caches the filetypes listed (jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf), and not PHP?
Umesh
Umesh
September 25 2013 Permalink
@Andy ..

This configuration catches PHP output (dynamic content) .......... as long as user is not logged in...
For all logged in user, it would simply bypass cache. This configuration really works if you have good amount of content for guest-users and if you have good amount of traffic coming from guest-users.

Regarding image/js/css files, it will cache files irrespective of whether user logged in or not.
I've seen cache getting rarely hit for these files as image/js/css are typically cached at browser level or at CDN level. I would not recommend to use Varnish only for caching these file types as nginx performance is comparable with Varnish for these file types.

Regarding avatar change, your observation is correct. It takes sometime for the cached image to go stale. One way to address this issue is to bypass cache for req.url ~ "avatar"
Unknown Person liked this
Andy Nash
Andy Nash
September 25 2013 Permalink
Thanks Umesh, sounds as though it won't offer too much beyond AWS Cloudfront for me for now, with the exception of the guest content. Will have a think about whether that will be worth it for my site...

Good tip regarding req.url!
Umesh
Umesh
September 30 2013 Permalink
I came across this Joomla plugin (freeware) ... May be worth exploring for Anahita...

http://www.jotcomponents.net/web-programming/jotcache

Additional Information

  • Updated March 30 2016 by Umesh
  • 5 Comments

Powered by Anahita