WordPress Backup Restoration Practice

Intro

This is Part II of a multi-part series on creating and managing a WordPress-NextJs app. In the first part, we set up a WordPress (WP) instance on AWS EC2 with OpenLiteSpeed, MySql on RDS, and S3 for our media storage. The WP site will serve as a headless CMS for a hypothetical client to manage his/her content; it will act as the data source for nextJs builds to generate a pre-rendered site deployed on AWS Cloudfront.

In this part, having set up our WP backend, we will practice restoring the site in the event that the EC2 instance gets bricked.

Rationale

In this section, I am going to argue for why we only need to be able to recreate a corrupted EC2 instance given our setup so far.

One of the advantages of (essentially) storing all of our application’s state in AWS RDS and S3 is that these managed services provide backup functionality out of the box.

S3 has insane multi-AZ “durability” out of the box of with, to quote a quick search google, “at least 99.999999999% annual durability, or 11 nines… [; t]hat means that even with one billion objects, you would likely go a hundred years without losing a single one!” You could make your S3 content even more durable by replicating across regions but, this is plenty durable for anything I’ll be building any time soon.

RDS durability requires a little more understanding. On a default setup, snapshots are taken of the DB’s disk storage every day, and retained for a week. RDS uses these backups to provide you with the means to restore the state of your DB to any second of your choosing between your oldest and most recent backups. This is your “retention period”. At extra cost, you can set the retention period to be as far back as 35 days, and you can also create manual snapshots to be stored indefinitely.

These RDS backups, while good and all, do not guard against certain worst-case scenarios. If, say, your DB instance crashes during a write operation and corrupts the physical file, then you will need to restore a backup before the end of the retention period; this could mean that you will lose your data if you do not also regularly check the WP site to make sure everything is in order.

Perhaps even worse is the case where some mishap occurs with your data that does not show up in an immediate or obvious way by visiting your site. For example, suppose you or a client installs a crappy plugin that deletes a random bunch; which you are not going to know about unless you do a thorough audit of your site’s content (and who’s got time for that!)

For this reason, you also really want to create regular logical backups of your DB and save them to e.g. S3 Glacier.

EC2 Restoration

We’ll practice reconstructing a working EC2 instance to host our WP site. This section assumes that you have set up everything as prescribed in Part I of this series.

First, in order to create a more authentic restoration simulation, you might want to stop your existing/working EC2 instance. Before you do that though, consider whether or not you have noted the details for connecting to your WP DB; if not, first note them down somewhere “safe”. Also be aware that if you did not associate an elastic-ip address with your EC2 instance, then it will get reset when you restart it later.

Next, create a new EC2 Ubuntu 20.04 instance, open port 80 and 7080, ssh into it, and run the following:

sudo apt update
sudo apt upgrade -y
curl -k https://raw.githubusercontent.com/litespeedtech/ols1clk/master/ols1clk.sh -o ols1clk.sh
sudo bash ols1clk.sh -w

The ols1clk.sh script will print to screen the details of the server and WP it is about to install. Save those details temporarily and proceed to install everything.

In the AWS RDS console, go to your active mysql instance, go to its security group, and add an inbound rule allowing connections from your the security group attached to your new EC2 instance. Copy the RDS endpoint.

Back in the EC2 instance, check that you can connect to the RDS instance by running:

mysql --host RDS_ENDPOINT -u WPDB_USER -p

… and entering the password for this user. Also make sure you can use the WP database within this instance.

If the connection works, then open the file /usr/local/lsws/wordpress/wp-config.php in your editor and replace the mysql connection details with those corresponding to your RDS instance and WP DB_NAME. You also need to add the following two lines in order to override the fact that the DB is set with a site base url corresponding to your previous EC2 instance:

define('WP_HOME', '[NEW_IP_ADDRESS]');
define('WP_SITEURL', '[NEW_IP_ADDRESS]');

… where NEW_IP_ADDRESS is taken from your new EC2 console.

Now you can go to NEW_IP_ADDRESS in a browser and expect to find a somewhat functioning WP site. If you try navigating to post though you will get a 404 error. To fix this, you need to go into OLS Admin account on port 7080, login using the new credentials generated by ols1clk.sh, go to the “Server Configuration” section, and under the “General” tab, in the “Rewrite Control” table, set the “Auto Load from .htaccess” field to “Yes”. Now you can expect to be able to navigate around posts.

(Side note: it’s very surprising to me that ols1clk.sh, in installing WP, does not set this field to Yes.)

The images will not work of course, because the DB records their location at the address of the old EC2 instance, which is not running. So in an actual restoration scenario, we would need to point the previous hostname from the old instance to the new instance, and then set up S3fs (which I am not going to test right now).

Having gone through this backup restoration practice, you can switch back to the old EC2 instance and, since we had to play around with our login credentials on a non-SSL site, it is a good idea to update your WP login password.