Handle 40x as much traffic -- Switch to Drupal 8 [Update: ok, not so much]

[Update: It turns out the Drupal 7 site we tested had page caching disabled. With page caching enabled, the scalability of anonymous pages seems very comparable -- we're seeing between 30 and 83 requests per second on both Drupal 7 and Drupal 8 on our base server configuration, with caching enabled, varying by website.

However, page caching can be enabled on many more Drupal 8 sites than Drupal 7 -- many Drupal 7 sites cannot enable page caching due to content that varies by user -- whereas nearly every Drupal 8 site can use caching, thanks to Drupal 8's intelligent cache tagging and invalidation.]

The story

We had a client reach out about an error page they got on their site:

What? Normally we get alerted to this kind of thing. Sometimes it's a result of a "Denial of Service" attack, but we have had some stability trouble recently with a Linux kernel killing processes because it prematurely thinks it's out of memory -- when this happens we need to go into the server and restart the affected container.

Not this time -- the site loaded up and looked fine. What happened?

First, I checked our alerting system. No sign of any alerts sent, no indication of a recent change. The system does wait for 5 minutes before sending an alert, but nothing got sent.

Next, I checked our performance monitoring system, and it showed a big load spike during the time in question:

... and there was a similar spike in traffic. What happened, and why didn't we get alerted?

Well, we didn't get alerted because the entire event was over and done and back to normal in less than 3 minutes -- not long enough to trigger anything.

Next, I hopped on the server to see what was going on there. Looking through the Nginx error logs, I found a bunch of messages:

2017/02/09 16:47:56 [alert] 43#43: 768 worker_connections are not enough

... This was right in the affected time. A bit of log analysis later, and I had determined that all of this traffic was generated from a single IP address over the course of 2 minutes, and had resulted in over 2900 error responses. And these all came from "artillery 2.3.3 (https://artillery.io)".

What's this?

The tool

Artillery turns out to be a very cool website load testing tool. We've used jMeter and ab in the past for this kind of thing, though we haven't yet worked this into our regular routine. So I decided to check out Artillery. It's written in node.js, which is one of the technologies we use routinely here, and it seems to be very easy to get started and use. So, of course, I had to give it a quick spin.

Drupal 7 results - Nginx, PHP5.5, MariaDB - 2 requests per second

So I pointed it at a handy Drupal 7 site out of curiosity -- you can guess which one -- built up to handle a similar amount of load as our client. This is already a bit faster stack than most regular shared hosting, but starting with the example in the documentation for "artillery quick", 10 requests per second were enough to crush the server. We had to go down to 2 requests per second to have the server able to handle the traffic without starting to back up requests. At 3 per second, the latency started creeping up over 10 seconds, and any more we started getting gateway timeouts.

Drupal 8 results - Nginx, PHP7.0, MariaDB - 83 requests per second

Curious, I pointed artillery to a recent Drupal 8 site on pretty much the same infrastructure, the only other difference being PHP7. And kept throwing more and more requests at it -- the server kept turning them around in less than 3 tenths of a second, until we went all the way to 100! At 100 requests per second, we finally got some errors -- on about 0.5% of the requests (30 out of ~6000!)

To get a sustainable amount with the current relatively stock tuning we have on the server, 83 requests per second was the magic number -- at that rate, the maximum latency was still less than 0.3 seconds (which already beat the minimum latency on the D7 site at ~0.5 seconds!)

Drupal 8 is a speed demon.

I've heard people complain about Drupal 8 being bloated, slow, and unweildy. And... that turns out to be utter hogwash. Drupal 7 with some caching configuration can easily beat similarly complex WordPress sites, but it's not even in the same race as Drupal 8, handling 1/40th as much traffic on essentially the same hardware.

And, we haven't even optimized anything on the Drupal 8 configuration yet. We can add Redis for higher speed caching, a Content Delivery Network to speed up assets and allow multiple front ends, and much more.

If you're looking to have the fastest, most flexible, most powerful CMS out there, look no further than Drupal 8. And contact us if you need any help!