Discourse S3 migration plan

Archives Community Ops/Discourse
#1

As you may or may not know, Discourse is dropping S3 support in version 1.3. From the versions later than ours, it shows an ugly warning on each page (for all visitors).

Originally, the plan was to delay upgrading until we got access to the Amazon EFS preview, but not only is that a long way off, the preview is also only in a different region, meaning we can’t put it in our VPC.

The current plan:

  1. Turn off auto-scaling for now (we’ve never had to anyway)
  2. Migrate assets hosted in S3 to local storage and update discourse
  3. Enable daily Discourse backups to S3
  4. Migrate assets to EFS once it becomes available

I also looked into using GlusterFS and while it’s redundant and easy to set up, I’m not sure if we want to set up monitoring/backups for a temporary solution.

Thoughts/things I might have missed? I’m going to schedule maintenance for sometime next week to get this done and knock out a few other things (CDN, app manifest).

3 Likes
#2

Is it likely we’ll need to auto-scale before we get access to EFS?

#3

What are their reasons for dropping the support, and what are others doing
about it? I suppose most people don’t have to auto-scale either?

#4

It was buggy from the beginning, and Discourse is meant to be self-sustainable. Using S3 makes it depend on external services.

#5

Jeff Atwood wrote the reasoning behind it in response to @tanner: https://meta.discourse.org/t/warning-amazon-s3-is-deprecated-in-1-3-for-image-attachment-storage/26594/4?u=logan

#6

Looks like they’ve rethought the decision and they will keep supporting S3 for 1.3. This should give us enough time for EFS to become available so we can switch to that, as S3 support is only going to get more and more buggy.

3 Likes