User:Woozle/toot.cat/2018/01/14

from HTYP, the free directory anyone can edit if they can prove to me that they're not a spambot
< User:Woozle
Revision as of 11:57, 15 January 2018 by Woozle (talk | contribs) (Woozle moved page User:Woozle/toot.cat to User:Woozle/toot.cat/2018/01/14: this was one set of things)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

2018-01-14

My current plan is:

  1. create new system user "tootcat"
  2. create a new Postgres db for toot.cat's Mastodon
  3. migrate the data there
  4. install Mastodon under user "tootcat"
  5. configure nginx
  6. run Mastodon's schema upgrade rake task
  7. test the result as https://new.toot.cat (get it working)
  8. write a script to automate the data migration
  9. in close succession:
    1. run the migration script
    2. reconfigure nginx to point to the new instance

revisions

  • Decided it would simplify things if I went ahead and created the "tootcat" system user first.
  • Instructions wanted to configure nginx after installing Mastodon, and this did need to be looked into at some point, so added it as step 5.

Notes

Upgrading Mastodon:

Work Log

step 1

Created user and group "tootcat" with /bin/bash as shell, no login allowed. I checked user "polymerwitch" (user for current Mastodon instance), and it has "sudo" membership -- but I'm not adding that, because it shouldn't be needed.

step 2

I'm calling the new db tootcat-masto-r1 (toot.cat Mastodon db revision 1).

root@tootcat:~# createdb --owner=tootcat tootcat-masto-r1
createdb: could not connect to database template1: FATAL: role "root" does not exist
Apparently pg assumes whatever the current system user is as its internal user as well.
root@tootcat:~# su - postgres -c 'createdb --owner=tootcat tootcat-masto-r1'
createdb: database creation failed: ERROR: role "tootcat" does not exist
root@tootcat:~# createuser --createdb tootcat
createuser: could not connect to database postgres: FATAL: role "root" does not exist - can't say I'm surprised
root@tootcat:~# su - postgres -c 'createuser --createdb tootcat'
Success.
root@tootcat:~# su - postgres -c 'createdb --owner=tootcat tootcat-masto-r1'
Success.

step 3: migrate the data

Done earlier -- inside /root/backups:

su - postgres -c 'pg_dump mastodon' > tootcat.sql

Importing the data dump into the new db:

root@tootcat:~# cd /root/scratch
root@tootcat:~/scratch# ls
tootcat.sql
root@tootcat:~/scratch# psql tootcat-masto-r1 < tootcat.sql
psql: FATAL: role "root" does not exist
I actually expected this, but just wanted to make sure.
root@tootcat:~/scratch# su - postgres -c 'psql tootcat-masto-r1 < tootcat.sql
-su: tootcat.sql: No such file or directory
Apparently this operates relative to the postgres user's home folder, or wherever you get put by default when logging in. Okay.
root@tootcat:~/scratch# su - postgres -c 'psql tootcat-masto-r1 < /root/scratch/tootcat.sql'
-su: /root/scratch/tootcat.sql: Permission denied
This is also unsurprising. Other users shouldn't have access to /root files.

It was more or less at this point that I discovered that the tootcat.sql file was zero bytes long, so apparently the export process wasn't happy either.

There's no /home/postgres, so we can't go there.

/root/scratch is already mode 777, so apparently that's insufficient.

Oh, but the actual tootcat.sql file is in /root/backups. Just to be sure, then:

root@tootcat:~/scratch# cd ../backups/
root@tootcat:~/backups# su - postgres -c 'psql tootcat-masto-r1 < /root/backups/tootcat.sql'
-su: /root/backups/tootcat.sql: Permission denied
As expected.

And then it finally occurred to me:

root@tootcat:~/backups# su - postgres -c 'pwd'
/var/lib/postgresql
Oh! Sneaksy, having a home folder outside of the /home folder. Just because /root does it doesn't make it okay; do as I say, not as I do....
root@tootcat:~/backups# cd /var/lib/postgresql/
root@tootcat:/var/lib/postgresql# mv /root/backups/tootcat.sql ./
root@tootcat:/var/lib/postgresql# chown postgres:postgres tootcat.sql
root@tootcat:/var/lib/postgresql# su - postgres -c 'psql tootcat-masto-r1 < tootcat.sql'
...and then stuff started happening that might have been the output of a successful import.

After taking a brief break, I come back to find that the process has completed... and a brief inspection shows that the db seems to be populated with data. I'm calling this step complete, then.

step 4: installing Mastodon

The official upgrade instructions mention that you should only ever use "tagged releases on production Mastodon instances", and therefore: 

su - mastodon
cd ~/live
git pull
git checkout $(git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1)

...except corrected for how things are set up here... and also I need to git clone instead of git pull since I'm doing the upgrade on a fresh deployment rather than in place on an existing one.

root@tootcat:/var/lib/postgresql# cd /home/tootcat
root@tootcat:/home/tootcat# su - tootcat
tootcat@tootcat:~$ mkdir web
tootcat@tootcat:~$ cd web
tootcat@tootcat:~/web$ mkdir mastodon
tootcat@tootcat:~/web$ cd mastodon
tootcat@tootcat:~/web/mastodon$ git clone https://github.com/rbenv/rbenv.git ~/.rbenv
Cloning into '/home/tootcat/.rbenv'...
remote: Counting objects: 2706, done.
remote: Total 2706 (delta 0), reused 0 (delta 0), pack-reused 2706
Receiving objects: 100% (2706/2706), 503.75 KiB 0 bytes/s, done.
Resolving deltas: 100% (1695/1695), done.
Checking connectivity... done.
tootcat@tootcat:~/web/mastodon$ cd ~/.rbenv && src/configure && make -C src
make: Entering directory '/home/tootcat/.rbenv/src'
gcc -fPIC -c -o realpath.o realpath.c
gcc -shared -Wl,-soname,../libexec/rbenv-realpath.dylib -o ../libexec/rbenv-realpath.dylib realpath.o
make: Leaving directory '/home/tootcat/.rbenv/src'
tootcat@tootcat:~/.rbenv$
tootcat@tootcat:~/.rbenv$ echo 'eval "$(rbenv init -)"' >> ~/.bashrc
tootcat@tootcat:~/.rbenv$ exec bash
tootcat@tootcat:~/.rbenv$ type rbenv
rbenv is a function
rbenv () 
{ 
    local command;
    command="${1:-}";
    if [ "$#" -gt 0 ]; then
        shift;
    fi;
    case "$command" in 
        rehash | shell)
            eval "$(rbenv "sh-$command" "$@")"
        ;;
        *)
            command rbenv "$command" "$@"
        ;;
    esac
}
tootcat@tootcat:~/.rbenv$ git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
Cloning into '/home/tootcat/.rbenv/plugins/ruby-build'...
remote: Counting objects: 8588, done.
remote: Compressing objects: 100% (25/25), done.
remote: Total 8588 (delta 9), reused 24 (delta 3), pack-reused 8556
Receiving objects: 100% (8588/8588), 1.79 MiB 0 bytes/s, done.
Resolving deltas: 100% (5481/5481), done.
Checking connectivity... done.

Skipping the bit about installing Ruby, since that is already installed on this server.

chdir to /home/tootcat/web, then:

tootcat@tootcat:~/web$ git clone https://github.com/tootsuite/mastodon.git mastodon
Cloning into 'mastodon'...
remote: Counting objects: 56389, done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 56389 (delta 0), reused 3 (delta 0), pack-reused 56376
Receiving objects: 100% (56389/56389), 52.51 MiB 30.41 MiB/s, done.
Resolving deltas: 100% (37140/37140), done.
Checking connectivity... done.
tootcat@tootcat:~/web$ cd mastodon/
tootcat@tootcat:~/web/mastodon$ git checkout $(git tag -l
Note: checking out 'v2.1.3'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 35b8498... Skip ActivityPub Announces of non-public objects (#6230)
tootcat@tootcat:~/web/mastodon$ gem install bundler
Fetching: bundler-1.16.1.gem (100%)
ERROR: While executing gem ... (Gem::FilePermissionError)
You don't have write permissions for the /var/lib/gems/2.3.0 directory.
tootcat@tootcat:~/web/mastodon$ bundle exec rails c
Could not find rake-12.3.0 in any of the sources
Run `bundle install` to install missing gems.
tootcat@tootcat:~/web/mastodon$ bundle install
Warning: the running version of Bundler (1.15.3) is older than the version that created the lockfile (1.16.1). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Fetching rake 12.3.0

Your user account isn't allowed to install to the system RubyGems.
  You can cancel this installation and run:

      bundle install --path vendor/bundle

  to install the gems into ./vendor/bundle/, or you can enter your password
  and install the bundled gems to RubyGems using sudo.

  Password:

Decided this was not the way to do it, and logged in as root:

root@tootcat:~# gem install bundler
Fetching: bundler-1.16.1.gem (100%)
Successfully installed bundler-1.16.1
Parsing documentation for bundler-1.16.1
Installing ri documentation for bundler-1.16.1
Done installing documentation for bundler after 9 seconds
1 gem installed
root@tootcat:~# bundle install --deployment --without development test
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine.
Could not locate Gemfile

Okay, so I guess it's back to the mastodon user...

tootcat@tootcat:~/web/mastodon$ bundle install --deployment --without development test
This seemed to work: /paste/2018/01/14/01
tootcat@tootcat:~/web/mastodon$ yarn install --pure-lockfile
This also seemed successful, but it also asked me to upgrade Yarn with these specific commands:
tootcat@tootcat:~/web/mastodon$ npm upgrade --global yarn
tootcat@tootcat:~/web/mastodon$ npm rebuild node-sass
> node-sass@4.7.2 install /home/tootcat/web/mastodon/node_modules/node-sass
> node scripts/install.js

node-sass build Binary found at /home/tootcat/web/mastodon/node_modules/node-sass/vendor/linux-x64-57/binding.node

> node-sass@4.7.2 postinstall /home/tootcat/web/mastodon/node_modules/node-sass
> node scripts/build.js

Binary found at /home/tootcat/web/mastodon/node_modules/node-sass/vendor/linux-x64-57/binding.node
Testing binary
Binary is fine
node-sass@4.7.2 /home/tootcat/web/mastodon/node_modules/node-sass

...and then we move on to configuring Postgres (already done) and nginx, which I think I should add as step 5.

step 5: configure nginx - ABANDONED for now

I made a copy of /etc/nginx/sites-available/default and called it toot-cat-new, then changed just a couple of things:

  • The server_name was changed from toot.cat to new.toot.cat in two places.

...and then I realized that this plan was going to require setting up extra instances of Puma and the media server (name?) on different ports from 3000 and 4000... which I think I know how to do, but this kinda crosses a threshold into territory uncertain enough that I decided to let a professional do it this time, and live to fight another day.

Retrieved from "https://htyp.org/mw/index.php?title=User:Woozle/toot.cat/2018/01/14&oldid=23632"