Migrating Your Discourse Forum to a New Server with Zero Downtime

Moving your Discourse forum to a new server, like Amazon Lightsail, is a common task for scaling or improving your community’s infrastructure. The goal is always to ensure a seamless transition with minimal to no downtime for your users. This guide will walk you through the process of migrating your Discourse instance, focusing on achieving a smooth switch-over while addressing common challenges like SSL certificate management and DNS propagation.

Understanding the Discourse Migration Process

The standard approach to migrating Discourse involves a few key steps. Let’s break down the general workflow before diving into the specifics of minimizing downtime:

New Server Setup: First, you need to provision and configure your new server. This includes installing Discourse on your chosen platform (like Ubuntu on Amazon Lightsail).
Backup Creation: Next, you’ll create a backup of your existing Discourse forum. This backup contains all your data, including posts, users, and settings.
Backup Restoration: The backup is then restored onto your newly set up Discourse server. This populates the new server with your forum’s content.
DNS Switch: Finally, you update your domain’s DNS records to point to the IP address of your new server. This directs user traffic to the migrated forum.

The challenge arises when you want to minimize downtime during this DNS switch, especially when you’re using the same domain name for both the old and new servers. Let’s address the potential issues and solutions to achieve a zero-downtime migration.

Addressing SSL Certificate and Access Challenges

One of the primary concerns during migration is managing the SSL certificate, particularly when using Let’s Encrypt. Since Let’s Encrypt verifies domain ownership via DNS, generating a certificate for the new server before the DNS switch can be tricky. Additionally, accessing the new server for setup and backup restoration before the domain points to it requires some workarounds.

Solving the SSL Certificate Challenge

When you configure Discourse on a new server using the same domain name as your existing live forum, the discourse-setup process will attempt to obtain an SSL certificate. However, if your DNS still points to the old server, Let’s Encrypt verification will fail.

Here are a couple of strategies to overcome this:

Temporary Domain/Subdomain: During the initial setup of your new server, you can temporarily use a subdomain or a completely different domain name. For example, if your main domain is example.com, you could initially configure the new server with new-server.example.com or a temporary domain. This allows discourse-setup to successfully obtain an SSL certificate for this temporary domain. After the migration and DNS switch, you can later adjust the domain in your app.yml and rebuild Discourse to use your primary domain.
IP Address Access (with Caution): While not ideal for production, you can temporarily access your new server directly via its IP address to complete the initial setup and backup restoration. However, be aware that accessing via IP address might cause browser SSL warnings and domain mismatches within Discourse itself. This method should be used cautiously and only for the initial setup phase before the DNS switch.
Manual Certificate Copy (Advanced): If you are comfortable with server administration, you could potentially copy the SSL certificate and private key from your old server to the new server. This is a more advanced approach and requires careful handling of sensitive SSL files.

Accessing the New Server Before DNS Switch

To access your new Discourse server for backup restoration and configuration before changing your DNS, you have a few options:

Hosts File Modification: The simplest method for local testing is to modify your computer’s hosts file. This file allows you to override DNS resolution locally. By adding an entry that points your domain name to the IP address of your new server, you can access the new server as if the DNS was already switched, but only from your local machine. Remember to remove this entry after the DNS switch is complete.
Temporary Subdomain (as mentioned above): Using a temporary subdomain during setup also allows you to access the new server via that subdomain without affecting your live domain.

Step-by-Step Discourse Migration Guide with Minimal Downtime

Let’s outline a detailed step-by-step guide to migrate your Discourse forum with minimal to zero downtime, incorporating the solutions discussed above:

Prepare Your New Server:
- Provision your Amazon Lightsail instance (or your chosen server).
- Install Docker and Discourse prerequisites as per the official Discourse installation guide (How To Install Discourse on Ubuntu 18.04 | DigitalOcean or https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md).
- During discourse-setup, use a temporary subdomain (e.g., staging.yourdomain.com) or a completely different temporary domain. Ensure you obtain a valid SSL certificate for this temporary domain.
- Alternatively, if you are using the hosts file method, you can use your main domain name during discourse-setup and access the new server via your hosts file for initial configuration.
Backup Your Existing Discourse Forum:
- Access your current Discourse forum as an administrator.
- Navigate to /admin/backups.
- Create a new backup. Download this backup file to your local machine. Since your current backup is on S3, you’ll need to download it manually as a file backup for local import as you correctly understand. Refer to Restore a backup from command line for command-line backup restoration.
Restore Backup to Your New Server:
- SSH into your new server.
- Copy the downloaded backup file to the /var/discourse/shared/standalone/backups/default/ directory on your new server. You can use scp for secure file transfer.
- Run the restore command within your Discourse container:
```
./launcher enter app
discourse restore <backup_filename.tar.gz>
exit
```
  Replace <backup_filename.tar.gz> with the actual name of your backup file.
Configure Domain and SSL on New Server (If using temporary domain initially):
- If you used a temporary subdomain during setup, edit your app.yml file on the new server:
```
./launcher enter app
nano config/app.yml
exit
```
- Change the hostname to your primary domain name (yourdomain.com).
- If necessary, update the LETSENCRYPT_HOST and LETSENCRYPT_EMAIL variables to reflect your primary domain.
- Rebuild Discourse to apply these changes and obtain an SSL certificate for your primary domain:
```
./launcher rebuild app
```
Pre-DNS Switch Testing:
- Using your hosts file modification or temporary subdomain, thoroughly test your new Discourse server. Ensure everything is working as expected, including all functionalities and data.
DNS Switch (The Zero Downtime Moment):
- Log in to your DNS registrar (where you manage your domain’s DNS records).
- Locate the DNS records for your domain (usually A records pointing to your old server’s IP address).
- Update these A records to point to the IP address of your new server.
- Crucially, reduce the TTL (Time To Live) value for your DNS records significantly a few hours before you perform the DNS switch. A lower TTL (e.g., 300 seconds or 5 minutes) will ensure that DNS propagation happens much faster when you make the change, minimizing the downtime. After the switch is complete and verified, you can increase the TTL back to a higher value.
Post-DNS Switch Verification:
- After making the DNS changes, check if your domain now resolves to your new server. You can use online DNS lookup tools to verify propagation.
- Test your Discourse forum thoroughly from different locations to ensure it’s accessible and functioning correctly for all users.

By following these steps, especially reducing the DNS TTL beforehand and using a temporary domain or hosts file for setup, you can achieve a near-zero downtime migration of your Discourse forum. Remember to always back up your data and test thoroughly before making any major changes to your live forum.