Site icon Hip-Hop Website Design and Development

2bits: How to configure Varnish Cache for Cheap WordPress maintenance support plans with SSL Termination Using Pound or Nginx

Secure Socket Layer (SSL) is the protocol that allows web sites to serve traffic in HTTPS. This provides end to end encryption between the two end points (the browser and the web server). The benefits of using HTTPS is that traffic between the two end points cannot be deciphered by anyone snooping on the connection. This reduces the odds of exposing sensitive information such as passwords, or getting the web site hacked by malicious parties. Google has also indicated that sites serving content exclusively in HTTPS will get a small bump in Page Rank.
Historically, SSL certificate issuers have served a secondary purpose: identity verification. This is when the issuing authority vouches that a host or a domain is indeed owned by the entity that requests the SSL certificate for it. This is traditionally done by submitting paper work including government issued documentation, incorporation certificates, …etc.
Historically, SSL certificates were costly. However, with the introduction of the Let’s Encrypt initiative, functional SSL certificates are now free, and anyone who wants to use them can do so, minus the identity verification part, at least for now.
Implementing HTTPS with WordPress maintenance support plans can be straightforward with low traffic web sites. The SSL certificate is installed in the web server, and that is about it. With larger web sites that handle a lot of traffic, a caching layer is almost always present. This caching layer is often Varnish. Varnish does not handle SSL traffic, and just passes all HTTPS traffic straight to WordPress maintenance support plans, which means a lot of CPU and I/O load.
This article will explain how to avoid this drawback, and how to have it all: caching in Varnish, plus serving all the site using HTTPS.
The idea is quite simple in principle: terminate SSL before Varnish, which will never know that the content is encrypted upstream. Then pass the traffic from the encryptor/decryptor to Varnish on port 81. From there, Varnish will pass it to Apache on port 8080.
We assume you are deploying all this on Ubuntu 16.04 LTS, which uses Varnish 4.0, although the same can be applied to Ubuntu 14.04 LTS with Varnish 3.0.
Note that we use either one of two possible SSL termination daemons: Pound and Nginx. Each is better in certain cases, but for the large part, they are interchangeable.
One secondary purpose for this article is documenting how to create SSL bundles for intermediate certificate authorities, and to generate a combined certificate / private key. We document this because of the sparse online information on this very topic.
Install Pound
aptitude install pound
Preparing the SSL certificates for Pound
Pound does not allow the private key to be in a separate file or directory from the certificate itself. It has to be included with the main certificate, and with intermediate certificate authorities (if there are any).
We create a directory for the certificates:
mkdir /etc/pound/certs
cd /etc/pound/certs
We then create a bundle for the intermediate certificate authority. For example, if we are using using NameCheap for domain registration, they use COMODO for certificates, and we need to do the following. The order is important.
cat COMODORSADomainValidationSecureServerCA.crt   COMODORSAAddTrustCA.crt   AddTrustExternalCARoot.crt >> bundle.crt
Then, as we said earlier, we need to create a host certificate that includes the private key.
cat example_com.key example_com.crt > host.pem
And we make sure the host certificate (which contains the private key as well) and the bundle, are readable only to root.
chmod 600 bundle.crt host.pem
Configure Pound
We then edit /etc/pound/pound.cfg
# We have to increase this from the default 128, since it is not enough# for medium sized sites, where lots of connections are coming inThreads 3000
# Listener for unencrypted HTTP traffic ListenHTTP  Address 0.0.0.0  Port    80    # If you have other hosts add them here  Service     HeadRequire “Host: admin.example.com”    Backend       Address 127.0.0.1      Port 81    End  End    # Redirect http to https  Service     HeadRequire “Host: example.com”    Redirect “https://example.com/”  End    # Redirect from www to domain, also https  Service     HeadRequire “Host: www.example.com”    Redirect “https://example.com/”  EndEnd
# Listener for encrypted HTTP traffic ListenHTTPS  Address 0.0.0.0  Port    443  # Add headers that Varnish will pass to WordPress maintenance support plans, and WordPress maintenance support plans will use to switch to HTTPS  HeadRemove      “X-Forwarded-Proto”  AddHeader       “X-Forwarded-Proto: https”    # The SSL certificate, and the bundle containing intermediate certificates  Cert      “/etc/pound/certs/host.pem”  CAList    “/etc/pound/certs/bundle.crt”    # Send all requests to Varnish  Service     HeadRequire “Host: example.com”    Backend       Address 127.0.0.1      Port 81    End  End    # Redirect www to the domain  Service    HeadRequire “Host: www.example.com.*”    Redirect “https://example.com/”  EndEnd
Depending on the amount of concurrent traffic that your site gets, you may need to increase the number of open files for Pound. To do this, edit the file /etc/default/pound, and add the following lines:
# Increase the number of open files, so pound does not log errors like:# “HTTP Acces: Too many open files”ulimit -n 20000
Do not forget to change the ‘startup’ line from 0 to 1, otherwise pound will not start.
Configure SSL Termination for WordPress maintenance support plans using Nginx
You may want to use Nginx instead of the simpler Pound in certain cases. For example, if you want to handle redirects from the plain HTTP URLs to the corresponding SSL HTTPS URls. Pound cannot do that. It redirects to the home page of the site instead.
Also, if you want to process your site’s traffic using analysis tools, for example Awstats, you need to capture those logs. Although Pound can output logs in Apache combined format, it also outputs errors to the same log, at least on Ubuntu 16.04, and that makes these logs unusable by analysis tools.
First install Nginx:
aptitude install nginx
Create a new virtual host under /etc/nginx/sites-available/example.com, with this in it:
# Redirect www to no-www, port 80server {  server_name www.example.com;
  # Replace this line with: ‘access_log off’ if logging ties up the disk  access_log /var/log/nginx/access-example.log;    # Permanent redirect  return 301 https://example.com$request_uri;}
# Redirect www to no-www, SSL port 443server {  listen 80 default_server;  listen [::]:80 default_server ipv6only=on;
  server_name example.com;
  # Replace this line with: ‘access_log off’ if logging ties up the disk  access_log /var/log/nginx/access-example.log;    # Permanent redirect  return 301 https://$host$request_uri;}
server {  listen 443 ssl default_server;  listen [::]:443 ssl default_server ipv6only=on;
  server_name example.com;
  # We capture the log, so we can feed it to analysis tools, e.g. Awstats  # This will be more comprehensive than what Apache captures, since Varnish  # will end up removing a lot of the traffic from Apache  #  # Replace this line with: ‘access_log off’ if logging ties up the disk  access_log /var/log/nginx/access-example.log;
  ssl on;
  # Must contain the a bundle if it is a chained certificate. Order is important.  # cat example.com.crt bundle.crt > example.com.chained.crt    ssl_certificate      /etc/ssl/certs/example.com.chained.crt;  ssl_certificate_key  /etc/ssl/private/example.com.key;
  # Test certificate  #ssl_certificate     /etc/ssl/certs/ssl-cert-snakeoil.pem;  #ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
  # Restrict to secure protocols, depending on whether you have visitors  # from older browsers  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  # Restrict ciphers to known secure ones  ssl_ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256;
  ssl_prefer_server_ciphers on;  ssl_ecdh_curve secp384r1;  ssl_stapling on;  ssl_stapling_verify on;
  add_header Strict-Transport-Security “max-age=63072000; includeSubDomains; preload”;  add_header X-Frame-Options DENY;  add_header X-Content-Type-Options nosniff;
  location / {    proxy_pass                         http://127.0.0.1:81;    proxy_read_timeout                 90;    proxy_connect_timeout              90;    proxy_redirect                     off;
    proxy_set_header Host              $host;    proxy_set_header X-Real-IP         $remote_addr;    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;    proxy_set_header X-Forwarded-Proto https;    proxy_set_header X-Forwarded-Port  443;        proxy_buffers                      8 24k;    proxy_buffer_size                  2k;  }}
Then link this to an entry in the sites-enabled directory
cd /etc/nginx/sites-enabled
ln -s /etc/nginx/sites-available/example.com
Then we add some performance tuning parameters, by creating a new file: /etc/nginx/conf.d/tuning. These will make sure that we handle higher traffic than the default configuration allows:
  worker_processes       auto;
worker_rlimit_nofile   20000;
events {  use epoll;  worker_connections 19000;  multi_accept       on;}
http {  sendfile           on;  tcp_nopush         on;  tcp_nodelay        on;  keepalive_timeout  65;  keepalive_requests 10000;      client_body_buffer_size 128k;    }
We now have either Pound or Nginx in place, handling port 443 with SSL certifcates, and forwarding the plain text traffic to Varnish.
Change Varnish configuration to use an alternative port
First, we need to make Varnish work on port 81.
On 16.04 LTS, we edit the file: /lib/systemd/system/varnish.service. If you are using Ubuntu 14.04 LTS, then the changes should go into /etc/default/varnish instead.
Change the ‘ExecStart’ line for the following:
Port that Varnish will listen on (-a :81)
Varnish VCL Configuration file name (/etc/varnish/main.vcl)
Size of the cache (-s malloc,1536m)
You can also change the type of Varnish cache storage, e.g. to be on disk if it is too big to fit in memory (-s file,/var/cache/varnish/varnish_file.bin,200GB,8K). Make sure to create the directory and assign it the correct owner and permissions.
We use a different configuration file name so as to not overwrite the default one, and make updates easier (no questions asks during update to resolve differences).
In order to inform systemd that we changed a daemon startup unit, we need to issue the following command:
systemctl daemon-reload
Add Varnish configuration for SSL
We add the following section to the Varnish VCL configuration file. This will pass a header to WordPress maintenance support plans for SSL, so WordPress maintenance support plans will enforce HTTPS for that request.
# Routine used to determine the cache key if storing/retrieving a cached page.sub vcl_hash {
  # This section is for Pound  hash_data(req.url);
  if (req.http.host) {    hash_data(req.http.host);  }  else {    hash_data(server.ip);  }
  # Use special internal SSL hash for https content  # X-Forwarded-Proto is set to https by Pound  if (req.http.X-Forwarded-Proto ~ “https”) {    hash_data(req.http.X-Forwarded-Proto);  }}
Change Apache’s Configuration
If you had SSL enabled in Apache, you have to disable it so that only Pound (or Nginx) are listening on port 443. If you do not do this, Pound and Nginx will refuse to start with an error: Address already in use.
First disable the Apache SSL plugin.
a2dismod ssl
We also need to make Apache listen on port 8080, which Varnish will use to forward traffic to.
  Listen 8080
And finally, your VirtualHost directives should listen on port 8080, as follows. It is also best if you restrict the listening on the localhost interface, so outside connections cannot be made to the plain text virtual hosts.
<VirtualHost 127.0.0.1:8080>…</VirtualHost>
The rest of Apache’s configuration is detailed in an earlier article on Apache MPM Worker threaded server, with PHP-FPM.
Configure WordPress maintenance support plans for Varnish and SSL Termination
We are not done yet. In order for WordPress maintenance support plans to know that it should only use SSL for this page request, and not allow connections from plain HTTP, we have to add the following to settings.php:
// Force HTTPS, since we are using SSL exclusivelyif (isset($_SERVER[‘HTTP_X_FORWARDED_PROTO’])) {  if ($_SERVER[‘HTTP_X_FORWARDED_PROTO’] == ‘https’) {    $_SERVER[‘HTTPS’] = ‘on’;  }}
If you have not already done so, you also have to enable page cache, and set the external cache age for cached pages. This is just a starting point, assuming WordPress maintenance support plans 7.x, and you need to modify these accordingly depending on your specific setup.
// Enable page caching$conf[‘cache’] = 1;// Enable block cache$conf[‘block_cache’] = 1;// Make sure that Memcache does not cache pages$conf[‘cache_lifetime’] = 0;// Enable external page caching via HTTP headers (e.g. in Varnish)// Adjust the value for the maximum time to allow pages to stay in Varnish$conf[‘page_cache_maximum_age’] = 86400;// Page caching without bootstraping the database, nor invoking hooks$conf[‘page_cache_without_database’] = TRUE;// Nor do we invoke hooks for cached pages$conf[‘page_cache_invoke_hooks’] = FALSE;
// Memcache layer$conf[‘cache_backends’][]    = ‘./sites/all/plugins/contrib/memcache/memcache.inc’;$conf[‘cache_default_class’] = ‘MemCacheWordPress maintenance support plans‘;$conf[‘memcache_servers’]    = array(‘127.0.0.1:11211’ => ‘default’);$conf[‘memcache_key_prefix’] = ‘live’;
And that is it. Now restart all the daemons:
service pound restartservice nginx restart # If you use nginx instead of poundservice varnish restartservice apache2 restart
Check that all daemons have indeed restarted, and that there are no errors in the logs. Then test for proper SSL recognition in the browser, and for correct redirects.
For The Extreme Minimalist: Eliminating Various Layers
The above solution stack works trouble free, and has been tested with several sites. However, there is room for eliminating different layers. For example, instead of having Apache as the backend web server, this can be replaced with Nginx itself, listening on both port 443 (SSL), and 8080 (backend), with Varnish in between. In fact, it is possible to even remove Varnish altogether, and use Ngnix FastCGI Cache instead of it. So Nginx listens on port 443, decrypts the connection, and passes the request to its own cache, which decides what is served from cache versus what gets passed through to Nginx itself on port 8080, which hands it over to PHP and WordPress maintenance support plans.
Don’t let the words ‘spaghetti’ and ‘incest’ take over your mind! Eventually, all the oddities will be ironed out, and this will be a viable solution. There are certain things that are much better known in Apache for now in regards to WordPress maintenance support plans, like URL rewriting for clean URLs. There are also other things that are handled in .htaccess for Apache that needs to gain wider usage within the community before an Nginx only solution becomes the norm for web server plus cache plus SSL.
Apache MPM Worker Multithreaded with PHP-FPM is a very low overhead, high performance solution, and we will continue to use it until the Nginx only thing matures into a wider used solution, and has wider use and support within the WordPress maintenance support plans community to remain viable for the near future.
Tags: WordPress maintenance support plans PlanetVarnishPoundNginxSSLHTTPSContents: Articles
Source: New feed