Pleroma and S3
Update 9/13/2020: Removed link to micromookie.com. Find me on Twitter or mastodon.social.
Update 12/28/2018: Removed link to trblmkr.net which was retired a few months ago. My new personal instance is micromookie.com, you can find me there being social.
This document shows how to setup Pleroma to use Wasabi as a storage location for media uploads. Using an S3-compatible object store can offload I/O and storage needs to a different system which costs less per GB than block storage. This works well for media uploads because once uploaded, they never change. Wasabi is a S3-compatible object store that costs $5/mo for 1TB of storage, unlimited egress and unlimited API calls.
The Wasabi and nginx sections are based on this excellent document on how to configure Wasabi with Mastodon.
Wasabi Setup
I won’t cover much of the Wasabi setup other than these high-level things. For the bucket that is created, it looks like Pleroma creates objects that are publicly available so there’s no policy needed for the bucket itself. If you do want to make the bucket readable, this is the policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::BUCKET_NAME/*"
}
]
}
A user needs to be created for Pleroma to upload files. Before creating the user, create a policy. The policy should look like:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::BUCKET_NAME"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::BUCKET_NAME/*"
}
]
}
When creating the user, make sure that it has Programatic access. Attach the above policy to the user. Save the Access and Secret Access Keys for the next step.
Pleroma Setup
In the config for Pleroma (usually config/prod.secret.exs), there’s some stuff to add and some stuff to change.
In the Pleroma.Upload section, change the uploader from Pleroma.Uploaders.Local to Pleroma.Uploaders.S3. The strip_exif doesn’t need to be changed, I changed it for my instance to strip out extra data in there (camera model, location, etc). Update (thanks @lanodan@queer.hacktivis.me): strip_exif requires ImageMagick to be installed.
config :pleroma, Pleroma.Upload,
uploader: Pleroma.Uploaders.S3,
strip_exif: true
Add these three sections to the config to configure the S3 uploader.
The public_endpoint is the hostname for where the files will be served from. My bucket is in Wasabi’s us-west-1 region, so it is set to s3.us-west-1.wasabisys.com, for the us-east-1 region use s3.wasabisys.com. This public_endpoint setting can be set to point to your domain also and then the files can be served using an nginx reverse-proxy. This is the way that I have mine setup and I document that part later.
config :pleroma, Pleroma.Uploaders.S3,
bucket: "BUCKET_NAME",
public_endpoint: "https://s3.us-west-1.wasabisys.com"
config :ex_aws, :s3,
access_key_id: "YOUR_ACCESS_KEY",
secret_access_key: "YOUR_SECRET_ACCESS_KEY",
region: "us-west-1",
scheme: "https://"
config :ex_aws, :s3,
host: "s3.us-west-1.wasabisys.com"
nginx Setup
If all you want is to serve the files via Wasabi using their domain name, then you can skip this section. This section shows how to setup an nginx reverse proxy and proxy cache to serve the files under your own domain.
In the Pleroma config, in the section Pleroma.Uploaders.S3, change the public_endpoint setting to be your domain name.
Add this to the top of the nginx configuration file for your domain, outside of the server block:
proxy_cache_path /tmp/nginx-cache levels=1:2 keys_zone=nginx-cache:10m max_size=2g inactive=1440m use_temp_path=off;
This sets up a cache for reverse-proxied files and they will be stored in /tmp/nginx-cache. 10MB will be created as shared memory (“10m” in the conf) for storing keys and info about the data. 2GB of disk storage is the max that nginx will use for caching the files locally. The cached files will be flushed if they are inactive for a day.
Next add this section in the server block in your domain’s nginx configuration file to create the reverse-proxy.
location /BUCKET_NAME {
proxy_cache nginx-cache;
proxy_cache_revalidate on;
proxy_buffering on;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_cache_background_update on;
proxy_cache_lock on;
proxy_cache_valid 1d;
proxy_cache_valid 404 1h;
proxy_ignore_headers Cache-Control;
add_header X-Cached $upstream_cache_status;
proxy_ignore_client_abort on;
proxy_pass https://s3.us-west-1.wasabisys.com/BUCKET_NAME;
}
The proxy_cache_use_stale is great for if/when Wasabi is experiencing issues and is down or returning 5XX errors.
It also helps your end-user’s privacy because Wasabi will not see who they are serving files to other than your server.
If you’re like me and have been running with local uploaded files and have them in the uploads directory. Copy the pleroma/uploads directory to the Wasabi bucket (try using awscli’s sync, it’ll make it easier). You’ll want to make the bucket world readable if you do this part.
Then add this section to your nginx config for the domain.
location /media {
proxy_cache nginx-cache;
proxy_cache_revalidate on;
proxy_buffering on;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_cache_background_update on;
proxy_cache_lock on;
proxy_cache_valid 1d;
proxy_cache_valid 404 1h;
proxy_ignore_headers Cache-Control;
add_header X-Cached $upstream_cache_status;
proxy_ignore_client_abort on;
proxy_pass https://s3.us-west-1.wasabisys.com/trblmkr-assets;
}
This way, everything can be stored in Wasabi.
That’s it. With all these setup, Pleroma + S3 will work and work nicely. Let me know if you have any questions at trblmkr.net.