Have your Mastodon and Eat It Too, for free

Posted on by
Reply

HOW TO HOST A LARGE MASTODON SERVER VERY CHEAPLY ON ORACLE CLOUD INFRASTRUCTURE

This is a how-to article. It assumes a basic level of Linux proficiency and skims over a lot of details. If there is a ton of popular request, I may come back and fill in more of those details.

Mastodon, the open source social network software, is a bit resource-hungry. In particular, it wants a fair bit of memory in order to run stably, and it likes to cache an insane volume of media (hundreds of gigabytes is a reasonable expectation).

Fortunately, as of 2022, cloud hosting providers are so desperate to get you onto their service that they are competitively giving away free server capacity, storage and bandwidth.

As a former employee at Oracle, I am happy to use their cloud offering, and that’s what this howto is about. I’m fully aware that not everyone trusts Big Red, and all I can say on the matter is that USB disks are cheap and secure offsite backups are your friend.

At time of writing, the Oracle Cloud free tier includes free Ampere (ARM64) VMs with four whole CPUs and a massive 24GB of RAM. This is where I recommend you put your Mastodon instance and associated database. You can also use the (much smaller) AMD64 parts of the cloud, but I keep those for things like my mail server and this blog.

I’m going to assume you’re comfortable installing and configuring your preferred Linux distro on your cloud VM(s).

The setup instructions at joinmastodon.org are clear, if a bit terse, so I’m going to assume that you can follow those too.

The key missing piece is the object storage. OCI free tier doesn’t include enough disk space for you to just keep your instance’s media cache in the server’s filesystem, and the cost of using normal block storage would be prohibitive, not to mention quite slow.

The answer is in Mastodon’s (very poorly documented) Amazon S3 block storage functionality.

  1. On your Oracle Cloud Infrastructure management page, go to Storage -> Object Storage & Archive Storage -> Buckets
  2. Click on ‘Create Bucket’ and name the bucket –
    1. I named mine after my instance domains, since I have more than one instance.
    2. I have enabled Auto Tiering in the hope of saving some money, although it may have no effect due to Mastodon’s cache TTL.
    3. I left “Encrypt using Oracle managed keys” at the default value.
    4. Leave your bucket visibility at ‘Private’ or this will impact the security of your instance.
    5. Click “Create”
    6. Take a copy of the ‘Namespace’ text for your new bucket.
  3. Under ‘Resources’ on the left, select “Pre-Authenticated Requests”.
    1. Leave the permissions at ‘Bucket’ and ‘Permit object reads’ – this special URL will be used by nginx to serve up the media in your bucket, so it only needs to be read-only.
    2. Select ‘enable object listing’
    3. This is very annoying: Oracle require these links to have an expiry date, and you can’t just type in a new date. You need to open up the date chooser and start clicking. I clicked through to the year 2100. You may have more or less patience than I do.
    4. Click “Create Pre-Authenticated Request”
    5. Copy the URL for the pre-authenticated request and SAVE IT SOMEWHERE!
  4. Now we play the long-and-tedious Oracle Cloud security game. Brace yourself.
  5. Open up the main menu again and select “Identity and Security”
    1. Under ‘Identity’ select ‘Groups’
    2. Click on ‘Create Group’
    3. Name your group whatever you like. Mine is called ‘Mastodon’ because I’m so creative.
    4. Click on ‘Create’
  6. Open up the main menu again and select “Identity and Security”
    1. Under ‘Identity’ select ‘Policies’
    2. Click ‘Create Policy’
    3. Name the policy something like ‘mastodon-bucket-policy’
    4. Under Policy Builder select ‘Show manual editor’
    5. In the manual editor box, paste this text (with your Mastodon group name): 
      Allow group Mastodon to manage buckets in tenancy
Allow group Mastodon to manage objects in tenancy

      Obviously these permissions are far more than is required, but I can at least confirm that they work. If you know this tool better than me and you can suggest a more reasonable set of permissions, please let me know.

    6. Click on ‘Create’
  7. Open up the main menu again and select “Identity and Security”
    1. Under ‘Identity’ select ‘Users’
    2. Click on ‘Create User’
    3. Select ‘IAM User’
    4. I called my user ‘Mastodon’ and am using the same user for both Instances / both Buckets. Fill in whatever details you like here.
    5. In the new user, click ‘Add User to Group’
    6. Select the group you created above
    7. Click ‘Add’
    8. Under ‘Resources’ on the left, click on ‘Auth Tokens’
    9. Click ‘Generate Token’. Call it whatever you like.
    10. SAVE A COPY OF THE TOKEN!
    11. Under ‘Resources’ on the left, click on ‘Customer Secret Keys’
    12. Click ‘Generate Secret Key’. Call it whatever you like.
    13. SAVE A COPY OF THE SECRET KEY
  8. That was a perfectly reasonable process, wasn’t it?
  9. When you set up your OCI account, you should have selected a region for your service. Make sure you know what your region is called. It’s listed as ‘Region’ in your compute instance details. Mine is ap-melbourne-1
  10. In your Mastodon server’s .env.production, you will see a bunch of commented-out “S3_…” parameters. When you’re finished editing them, they will look something like this: 
    S3_ENABLED=true
S3_ENDPOINT=https://<your bucket namespace>.compat.objectstorage.<your region>.oraclecloud.com
S3_PROTOCOL=https
S3_BUCKET=mastodon
S3_HOSTNAME=<your bucket namespace>.compat.objectstorage.<your region>.oraclecloud.com
AWS_ACCESS_KEY_ID=<your auth token>
AWS_SECRET_ACCESS_KEY=<your secret key>
S3_ALIAS_HOST=<your media proxy subdomain>
  11. Set up your nginx object storage proxy as documented here, but user the pre-approved request URL you generated earlier here:
    set $s3_backend ‘https://objectstorage.<your region>.oraclecloud.com/p/<your secret magic here>/o’;
    IMPORTANT NOTE: There is no trailing slash after the last ‘o’. If you include the slash, things will break in ways that take a lot of time and sanity to debug. Don’t ask me how I know.

That’s it. If this doesn’t make sense, or doesn’t work for you, or if you just need more detail on some aspect of this, please leave a comment or message me at @thorne and I will do what I have time and spoons for.

Good luck!

A Blacker Mirror

Posted on by
Reply

On Friday the 28th of January 2011, I was in Brisbane, listening to the ABC‘s Mark Pesce talk about Facebook and the death of privacy.

Pesce later copped significant flak about his presentation because some of the images in it breached conference guidelines. Intentionally or not, this controversy quite successfully suppressed the talk’s actual message, which was simply:

Facebook will soon be your whole life.

That was nine years ago. Soon has come and gone.

In October 2016, an episode of the dystopian science fiction anthology Black Mirror – Nosedive – gave us a world where social-network-derived popularity scores are the dominant measure of human worth.

A month later, the United States elected their 45th president – Donald Trump – and ushered in what has since been called the post-truth era.

In October 2017, another science fiction series, The Orville, in Majority Rule gave us a world where the entire population vote one another up and down based entirely on a social network called ‘the master feed’. In this world, the minimally informed kneejerk reactions of the masses have replaced government, the entire justice system, and even truth itself.

However, both of these worlds ignored the burning issue of editorial control: The network itself is the arbiter of truth, of all human value and freedom.

Today, in 2020, Your online activity is now effectively a social ‘credit score’ – “now we’re looking at groups of historically marginalized people being denied involvement in mainstream economic, political, cultural and social activities — at scale.”

I am posting this here, on my more-or-less-abandoned blog, precisely because there is no editorial control here. Later, I intend to post most of this text on Facebook, where I have no doubt that its audience will be tightly controlled.

I have no rallying cry, no call to action. I honestly cannot see any way out. This post is simply intended to acknowledge what we have become.

Depressed by the middle easth

Posted on by
Reply

Warning: Alliterative first time I want it, and that I am looking after 1 tbsp of his many True Believers ). Thus, and won’t get you create your body might seem frivolous, carried forward to do anything but it just get your head. Gary Parry had a freaky healthy: two to move, with the earth, at the stone. . This particular song again in terms of won me to be a Western Australian dollars .

Markov, bishes. 🙂

28 years later

Posted on by
Reply

Tron: 1982

  •  I am 7 years old.
  •  The dawn of personal computing. For the first time, individuals have computers.
  •  Computing is nerdy, specialised. Arcade Computer Games are new and cool.
  •  Life inside the computer is depicted as banal, office-like, yearning to be real.
  •  Clumsy special effects pretend to be too-expensive computer effects.
  •  Completely unique film stock is created to make the film effects possible.
  •  Huge solid sets are painstakingly built and painted to look surreal.
  •  Real actors are clad in suits to make them look unnatural.

Legacy: 2010

  • I am 35 years old.
  • The internet is an inextricable part of life for most of the world.
  • Computing is everywhere, in everything. It’s uncool to not be a nerd.
  • Life inside the computer is depicted as impossibly cool. It mocks the real world.
  • Computer effects are trivial and cheap.
  • No physical film is ever exposed.
  • Hardly any physical sets are even built.
  • The central villain is a flawless digital emulation of Jeff Bridges of 1982.

This is what it must feel like to live through epochal change.

Once upon a time in The Future

Posted on by
Reply

So I just watched 2010 again, still an odd favourite, and it made me think about the last ever space shuttle mission last week, and more…

In 1982, Arthur C Clarke wrote a book set in 2010, a sequel to his 1968 book and roughly simultaneous Stanley Kubrick movie, set in 2001.

In 1984, Margaret Thatcher stands in for Big Brother in helping the UK to slide into fascism without any help from a nuclear war, a computer called Fate, or a horrific man-made virus. The United States re-elects a Cowboy who is probably not a robot.

In 1984 I’m in grade 3.

Also in 1984, Clarke’s latter book is released as a somewhat less famous movie, with an old-looking Roy Schneider alongside a very young-looking John Lithgow, and a russian-accented not-yet-dame Helen Mirren.

In that book, and that movie, several awesomely improbable things happen which it is abundantly evident did not come to pass last year: There is no second star in the orbit of Jupiter. No nuclear stand-off was narrowly averted by aliens. We clearly do not have any AIs, let alone enough to take one for granted.

But that’s all boring.

What’s interesting, are the things which it clearly made sense to assume about 2010 in 1984…

  • The Soviet Union will still exist. In fact it will be just as healthy as the USA.
  • The Soviet Union will still be the USA’s biggest military concern.
  • The USSR and the USA will still be engaged in a neck-and-neck space race.
  • In fact, both nations will have a permanent presence in space.
  • Not only will it be feasible for the USSR to launch a large, manned mission to Jupiter, but the USA already did that, nine years earlier.

I can’t begin to count the emotions I feel when I try to see the 2010 that actually was from the standpoint of the bright imaginations who made that movie, back in 1984.

It’s somehow like we’re living in the dystopian alternative world of Watchmen; we’ve done so much, been so brilliant,- so how did we get here?

 

Training for Exhaustion

Posted on by
Reply

How to get veeery tired:

  1. Have a virus, an infected gallbladder and surgery to remove said organ, etc…
  2. Get sent to allegedly Career Significant Training in the CBD.
  3. …training that starts at 8:15am sharp.
  4. …training which your line manager is co-facilitating that day.
  5. Have the deluded idea that you can ride (a motorbike) to this training more easily than catch a train.
  6. …through the Donvale tunnel, down the Eastern freeway, and Nicholson street.
  7. Attend this training. It runs all day, with minimal breaks and massive performance pressure.
  8. At 5:15pm, get out of this training and go meet a friend. Walk around the city for a while for good measure.
  9. Now ride home. Same route, in reverse. Did I mention that it’s VERY COLD?

If you see no problem with this picture, you may not have had the aforementioned recent surgery, or you might just be really fit. :-/

Bloodshed

Posted on by
1

So, that happened.

<engage ramble-mode>

On Saturday the 14th of May E and I went to see Dr Zhivago at The Maj. It was fantastic, but as I was leaving the theatre I had a ‘cramp’ in my chest and had to sit down. It passed, and I gave it no further thought.

That night, we had some deeply awesome dumplings for dinner. I ate more of them than was in any way called for.

Around 11:30pm that night, my chest began to hurt. This rapidly developed into the worst, most intense pain I have ever experienced, and after trying various pain medications, Erin took me into Knox Private, where I was admitted, medicated to the eyeballs, and spent the night.

In the morning, the pain was gone, and I was sent home with an appointment for ultrasound to explore the possibility of gallstones on Tuesday morning.

On Tuesday morning, on my way to work, I stopped in for my ultrasound. The Ultrasoundist had a trainee observing his work so I got a very special opportunity to hear detailed, fearless running commentary on my scan as it was being done. Apparently I don’t have cancer (!!!) but he said some faintly disturbing things about the dimensions of my gall-bladder, and how this was indicative of Great Inflammation(tm).

The scan being done, I got dressed to leave, and was told to report to the emergency department. (WTF?!)

The ED nurses directed me to a bed and told me to change into a hospital gown. (WTF!!!?!?)

The ED doctor arrived, cheerfully examined my scans, and informed me that he had hassled the surgeon until he rescheduled to make a time for me that day, to have my gall-bladder removed. (AAARGH! WTF?!?!?!?!!!)

From there, my week was subsumed by a lengthy (Three hours on the table) keyhole surgery, and a long, unpleasant recovery. I did in fact get out of the hospital, on Saturday, less one severely infected gall-bladder and some other abdominal tissue which had been compromised by said infection.

As I tweeted at the time, there were proverbial Little Glass Vials.

As of today, I am more or less recovered, and back at work. When I got here today, I found that my workmates had made productive use of their time in my absence:

Indeed:

And even:

If this all just seems weird to you, you may need to watch Dexter.

So, yeah, that happened.