View source for User:Woozle/toot.cat/2018/04/07

=Take 1=
Loosely following [https://anystack.xyz/how-to-install-mastodon-ubuntu/ these instructions]...

I just do everything in root rather than using sudo, because that's how I roll.

Things done:
* Create droplet (4GB- 2CPU - 80GB - $20, Ubuntu 16.04, SFO2, IPv6, Gonzo key)
* <code>apt update</code>
* <code>apt upgrade</code>
* install node.js:
** <code>curl -sL https://deb.nodesource.com/setup_6.x | sudo bash -</code>
** <code>apt install imagemagick ffmpeg libpq-dev libxml2-dev libxslt1-dev nodejs</code>
** <code>npm install -g yarn</code>
* install redis-server:
** <code>apt install redis-server redis-tools</code>
* install [[PostgreSQL]] (Pg):
** <code>apt install postgresql postgresql-contrib</code>
* switch to postgres user, create Mastodon user in Pg:
** <code>su - postgres</code>
** <code>psql</code>
** <code>CREATE USER mastodon CREATEDB;</code>
** <code>\q</code>

Now, we're trying to enable ident auth in Pg so users can log in without a password.
* I don't understand why we want to do this; seems like bad security.
* I tried to do this bit exactly as written, because it looked like it might be user-sensitive, but it didn't work (see below).
* Can this be done as all one line?

* <code>sudo sed -i '/^local.*postgres.*peer$/a host all     all     127.0.0.1/32    ident' \</code>
* <code>/etc/postgresql/9.?/main/pg_hba.conf</code>

That didn't work, because after the second line it asked for a password for postgres, which I don't have, and wouldn't accept a blank. So, backing out and trying as root (and all one line):

* <code>sed -i '/^local.*postgres.*peer$/a host all     all     127.0.0.1/32    ident' /etc/postgresql/9.?/main/pg_hba.conf</code>

No error, but also no success message. I will assume successful.

* install ident daemon:
** <code>apt install pidentd</code>
** <code>systemctl enable pidentd</code>
** Response:
*** <code>pidentd.service is not a native service, redirecting to systemd-sysv-install</code>
*** <code>Executing /lib/systemd/systemd-sysv-install enable pidentd</code>
** <code>systemctl start pidentd</code>
** <code>systemctl restart postgresql</code>
==Setting Up Ruby==
* <code>apt install autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm3 libgdbm-dev rbenv</code>
** Note: the instructions omitted <code>rbenv</code>, but it seems to be needed.
* <code>adduser --disabled-password --disabled-login mastodon</code>
** Just hitting ENTER in response to all questions, "Y" in response to "Y/n".
* Change to user <code>mastodon</code> (which also puts you in <code>/home/mastodon</code>):
** <code>su - mastodon</code>
* <code>git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build</code>
* install Ruby 2.4.1:
** <code>rbenv install 2.4.1</code>

That didn't work because rbenv was not installed. (Also not sure this is the best version of Ruby, anymore, but that seems fixable later now that we have the command). So I had to back out to root, <code>apt install rbenv</code>, and then <code>su - mastodon</code> again.

* <code>rbenv install 2.4.1</code>
** This takes a couple of minutes.
* <code>rbenv global 2.4.1</code>
** This is instantaneous.
* <code>ruby -v</code>
** response: <code>ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]</code>
*** ...um, okay.
==Installing Mastodon==
* <code>cd ~</code>
** This is probably redundant.
* <code>git clone https://github.com/tootsuite/mastodon.git live</code>
* <code>cd live</code>
* <code>git checkout $(git tag | tail -n 1)</code>
** I don't know what this does.
* "Install bundler to manage the dependencies and disable the gem documentation."
** <code>echo "gem: --no-document" > ~/.gemrc</code>
** <code>gem install bundler --no-ri</code>
*** Response:
**** <code>Fetching: bundler-1.16.1.gem (100%)</code>
**** <code>ERROR:  While executing gem ... (Gem::FilePermissionError)</code>
**** <code>   You don't have write permissions for the /var/lib/gems/2.3.0 directory.</code>

Okay, '''Simon says'''...
* <code>exit</code>
* <code>gem install bundler --no-ri</code>
** Response:
*** <code>Fetching: bundler-1.16.1.gem (100%)</code>
*** <code>Successfully installed bundler-1.16.1</code>
*** <code>1 gem installed</code>
* <code>su - mastodon</code>
* <code>cd live</code>
* <code>bundle install --deployment --without development test</code>
** A bunch of stuff happened, and then this error message:
<pre>
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /home/mastodon/live/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.8.2/ext/nokogiri
/usr/bin/ruby2.3 -r ./siteconf20180407-2653-idvqpi.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h

extconf failed, exit code 1

Gem files will remain installed in /home/mastodon/live/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.8.2 for inspection.
Results logged to /home/mastodon/live/vendor/bundle/ruby/2.3.0/extensions/x86_64-linux/2.3.0/nokogiri-1.8.2/gem_make.out

An error occurred while installing nokogiri (1.8.2), and Bundler cannot continue.
Make sure that `gem install nokogiri -v '1.8.2'` succeeds before bundling.

In Gemfile:
  rails-settings-cached was resolved to 0.6.6, which depends on
    rails was resolved to 5.1.4, which depends on
      actioncable was resolved to 5.1.4, which depends on
        actionpack was resolved to 5.1.4, which depends on
          actionview was resolved to 5.1.4, which depends on
            rails-dom-testing was resolved to 2.0.3, which depends on
              nokogiri
</pre>
Okay, trying that command...
* <code>gem install nokogiri -v '1.8.2'</code>
** Response:
*** <code>Fetching: mini_portile2-2.3.0.gem (100%)</code>
*** <code>ERROR:  While executing gem ... (Gem::FilePermissionError)</code>
*** <code>    You don't have write permissions for the /var/lib/gems/2.3.0 directory.</code>

So again, Simon says...
* <code>exit</code>
* <code>gem install nokogiri -v '1.8.2'</code>
** Response:
<pre>Fetching: mini_portile2-2.3.0.gem (100%)
Successfully installed mini_portile2-2.3.0
Fetching: nokogiri-1.8.2.gem (100%)
Building native extensions.  This could take a while...
ERROR:  Error installing nokogiri:
	ERROR: Failed to build gem native extension.

    current directory: /var/lib/gems/2.3.0/gems/nokogiri-1.8.2/ext/nokogiri
/usr/bin/ruby2.3 -r ./siteconf20180407-2736-1uhzyd6.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h

extconf failed, exit code 1

Gem files will remain installed in /var/lib/gems/2.3.0/gems/nokogiri-1.8.2 for inspection.
Results logged to /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/nokogiri-1.8.2/gem_make.out
</pre>

Okay. Let's start over, and use the [https://github.com/staticsafe/mastodon-documentation/blob/master/Running-Mastodon/Production-Guide.md official instructions] this time, leaving off any bits that have already clearly been done. (I don't know why I didn't use them to begin with... I guess these alternate instructions just looked so distro-specific and friendly.)
=Take 2=
As root...
* <code>cd /root</code>
* <code>mkdir setup</code>
* <code>cd setup</code>
* <code>wget https://deb.nodesource.com/setup_6.x</code>
* ...and then "review the script" using <code>less</code> -- as if I'd know how to evaluate it. Glanced at it, what does it mean. Now running it...
* </code>bash setup_6.x</code>
==Yarn Repository==
* <code>apt -y install curl</code>
** was already installed, but might not have been if I hadn't tried the other instructions first
* <code>curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -</code>
* <code>echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list</code>
* <code>apt update</code>
* Install [[Yarn]] plus a bunch of other stuff:
** <code>apt -y install imagemagick ffmpeg libpq-dev libxml2-dev libxslt1-dev file git g++ libprotobuf-dev protobuf-compiler pkg-config nodejs gcc autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm3 libgdbm-dev nginx redis-server redis-tools postgresql postgresql-contrib nginx letsencrypt yarn</code>
==User-Specific Stuff==
* <code>su - mastodon</code>
* <code>git clone https://github.com/rbenv/rbenv.git ~/.rbenv</code>
** This does not seem to have been done earlier, or at least not with the same command -- but there is a .rbenv folder. So can't do it again, unless we want to erase it. Maybe later, if something seems to be wrong with rbenv.
* <code>cd ~/.rbenv && src/configure && make -C src</code>
** Response: <code>-su: src/configure: No such file or directory</code>
** Okay, something seems to be wrong with rbenv.
* <code>cd ~</code>
* <code>rm -rdf .rbenv</code>
* <code>git clone https://github.com/rbenv/rbenv.git ~/.rbenv</code>
* <code>cd ~/.rbenv && src/configure && make -C src</code>
* <code>echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc</code>
* <code>echo 'eval "$(rbenv init -)"' >> ~/.bashrc</code>
* Restart the shell:
** <code>exec bash</code>
* Check if rbenv is correctly installed:
** <code>type rbenv</code>
*** Response:<pre>
rbenv is a function
rbenv () 
{ 
    local command;
    command="${1:-}";
    if [ "$#" -gt 0 ]; then
        shift;
    fi;
    case "$command" in 
        rehash | shell)
            eval "$(rbenv "sh-$command" "$@")"
        ;;
        *)
            command rbenv "$command" "$@"
        ;;
    esac
}</pre>
* Install ruby-build as rbenv plugin:
** <code>git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build</code>
* <code>rbenv install 2.4.1</code>
** This is probably redundant now, but why not.
** It still takes a couple of minutes, though.
* <code>rbenv global 2.4.1</code>
==Installing Mastodon==
Much of this has already been done -- but this instruction set does explain it a bit. Skipping redundant bits...
* Install bundler (again, but without the options):
** <code>gem install bundler</code>
* Use bundler to install the rest of the Ruby dependencies (this might be redundant, but maybe not):
** <code>bundle install --deployment --without development test</code>
*** Response: <code>Could not locate Gemfile</code>

I have to stop for tonight.

===part 2===

Looks like [[Gemfile]] is an app-config file which needs to already exist when you run <code>bundle install [...]</code> -- and there's one in <code>/home/mastodon/live</code>, so let's run it from there.

Log in as root, then:
* <code>su - mastodon</code>
* <code>cd live</code>
* <code>bundle install --deployment --without development test</code>
** Response: <code>rbenv: version `2.5.0' is not installed (set by /home/mastodon/live/.ruby-version)</code>

Looks like the instructions may not have been updated; Charlotte tells me that Masto does, in fact, now require Ruby 2.5.0. So...

* <code>cd ~</code>
* <code>rbenv install 2.5.0</code>
* <code>rbenv global 2.5.0</code>
* <code>cd live</code>
* <code>bundle install --deployment --without development test</code>
** Response:<pre>rbenv: bundle: command not found

The `bundle' command exists in these Ruby versions:
  2.4.1
</pre>

I'm going on the assumption that this doesn't mean "Ruby did away with Bundler after 2.4.1" but rather "You haven't installed it for 2.5.0, you've only installed it for 2.4.1".

We're already in ~/live, which seems to be the right folder, so no need to cd there.

* <code>gem install bundler</code>
*: <code>Successfully installed bundler-1.16.1</code>
*: <code>1 gem installed</code>
* <code>bundle install --deployment --without development test</code>
** This seemed to be running successfully for quite awhile, installing various things, then failed with the following message:
<pre>
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /home/mastodon/live/vendor/bundle/ruby/2.5.0/gems/idn-ruby-0.1.0/ext
/home/mastodon/.rbenv/versions/2.5.0/bin/ruby -r ./siteconf20180409-7030-grmz7k.rb extconf.rb
checking for -lidn... no
ERROR: could not find idn library!

  Please install the GNU IDN library or alternatively specify at least one
  of the following options if the library can only be found in a non-standard
  location:
    --with-idn-dir=/path/to/non/standard/location
        or
    --with-idn-lib=/path/to/non/standard/location/lib
    --with-idn-include=/path/to/non/standard/location/include

*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/home/mastodon/.rbenv/versions/2.5.0/bin/$(RUBY_BASE_NAME)
	--with-idn-dir
	--without-idn-dir
	--with-idn-include
	--without-idn-include=${idn-dir}/include
	--with-idn-lib
	--without-idn-lib=${idn-dir}/lib
	--with-idnlib
	--without-idnlib

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /home/mastodon/live/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-ruby-0.1.0/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /home/mastodon/live/vendor/bundle/ruby/2.5.0/gems/idn-ruby-0.1.0 for inspection.
Results logged to /home/mastodon/live/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-ruby-0.1.0/gem_make.out

An error occurred while installing idn-ruby (0.1.0), and Bundler cannot continue.
Make sure that `gem install idn-ruby -v '0.1.0'` succeeds before bundling.

In Gemfile:
  idn-ruby
</pre>
Get back out to root, then:
* <code>apt install idn</code>
* <code>su - mastodon</code>
* <code>cd live</code>
* <code>bundle install --deployment --without development test</code>
** Response:<pre>
Installing idn-ruby 0.1.0 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /home/mastodon/live/vendor/bundle/ruby/2.5.0/gems/idn-ruby-0.1.0/ext
/home/mastodon/.rbenv/versions/2.5.0/bin/ruby -r ./siteconf20180409-19186-1hcp4ek.rb extconf.rb
checking for -lidn... no
ERROR: could not find idn library!

  Please install the GNU IDN library or alternatively specify at least one
  of the following options if the library can only be found in a non-standard
  location:
    --with-idn-dir=/path/to/non/standard/location
        or
    --with-idn-lib=/path/to/non/standard/location/lib
    --with-idn-include=/path/to/non/standard/location/include

*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/home/mastodon/.rbenv/versions/2.5.0/bin/$(RUBY_BASE_NAME)
	--with-idn-dir
	--without-idn-dir
	--with-idn-include
	--without-idn-include=${idn-dir}/include
	--with-idn-lib
	--without-idn-lib=${idn-dir}/lib
	--with-idnlib
	--without-idnlib

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /home/mastodon/live/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-ruby-0.1.0/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /home/mastodon/live/vendor/bundle/ruby/2.5.0/gems/idn-ruby-0.1.0 for inspection.
Results logged to /home/mastodon/live/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-ruby-0.1.0/gem_make.out

An error occurred while installing idn-ruby (0.1.0), and Bundler cannot continue.
Make sure that `gem install idn-ruby -v '0.1.0'` succeeds before bundling.

In Gemfile:
  idn-ruby
</pre>

Ohhkayy... well... let's try pointing it directly at the idn library file...

* <code>bundle install --deployment --without development test --with-idn-lib=/usr/lib/x86_64-linux-gnu/libidn.so.11</code>
** Response: <code>Unknown switches '--with-idn-lib=/usr/lib/x86_64-linux-gnu/libidn.so.11'</code>
*** Okay, so the error message is lying to me, or I'm misreading it.

<code>bundle --help</code> gives man-style help which lists all the bundle commands; <code>bundle install --help</code> lists options specific to <code>bundle install</code>, and none of the <code>--with-idn-*</code> options appear to be among either of them.

* How about: <code>gem install idn-ruby</code>... nope, same error message...
* How about <code>gem install idn-ruby --with-idn-lib=/usr/lib/x86_64-linux-gnu/libidn.so.11</code>...
** nope: <code>invalid option: --with-idn-lib=/usr/lib/x86_64-linux-gnu/libidn.so.11</code>
===part 3===
Some [https://cybre.space/@benhamill/99831608699607241 research] by Ben Hamill suggested this: <code>gem install --remote idn</code>

The resulting error message pointed me at a log file, which seemed to be saying that the problem was coming from [[gcc]]. Since this means there was compiling involved, I thought maybe I should install the header file for idn as well (as root): <code>apt install libghc-gnuidn-dev</code>.

Followed by...
* <code>su - mastodon</code>
* <code>gem install --remote idn</code>

Output:
<pre>Building native extensions. This could take a while...
ERROR:  Error installing idn:
	ERROR: Failed to build gem native extension.

    current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext
/home/mastodon/.rbenv/versions/2.5.0/bin/ruby -r ./siteconf20180409-23716-rrnxuy.rb extconf.rb
checking for -lidn... yes
checking for idna.h... yes
checking for punycode.h... yes
checking for stringprep.h... yes
creating Makefile

current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext
make "DESTDIR=" clean

current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext
make "DESTDIR="
compiling idn.c
compiling idna.c
idna.c: In function ‘toASCII’:
idna.c:88:37: error: ‘struct RString’ has no member named ‘ptr’
   rc = idna_to_ascii_8z(RSTRING(str)->ptr, &buf, flags);
                                     ^
idna.c: In function ‘toUnicode’:
idna.c:128:41: error: ‘struct RString’ has no member named ‘ptr’
   rc = idna_to_unicode_8z8z(RSTRING(str)->ptr, &buf, flags);
                                         ^
idna.c: At top level:
cc1: warning: unrecognized command line option ‘-Wno-self-assign’
cc1: warning: unrecognized command line option ‘-Wno-constant-logical-operand’
cc1: warning: unrecognized command line option ‘-Wno-parentheses-equality’
cc1: warning: unrecognized command line option ‘-Wno-tautological-compare’
Makefile:241: recipe for target 'idna.o' failed
make: *** [idna.o] Error 1

make failed, exit code 2

Gem files will remain installed in /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2 for inspection.
Results logged to /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-0.0.2/gem_make.out
</pre>

Oops, maybe I was supposed to be in <code>~/live</code>?

Logging in again as root...
* <code>su - mastodon</code>
* <code>cd live</code>
* <code>gem install --remote idn</code>

Output:<pre>
Building native extensions. This could take a while...
ERROR:  Error installing idn:
	ERROR: Failed to build gem native extension.

    current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext
/home/mastodon/.rbenv/versions/2.5.0/bin/ruby -r ./siteconf20180409-24783-1trgchl.rb extconf.rb
checking for -lidn... yes
checking for idna.h... yes
checking for punycode.h... yes
checking for stringprep.h... yes
creating Makefile

current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext
make "DESTDIR=" clean

current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext
make "DESTDIR="
compiling idn.c
compiling idna.c
idna.c: In function ‘toASCII’:
idna.c:88:37: error: ‘struct RString’ has no member named ‘ptr’
   rc = idna_to_ascii_8z(RSTRING(str)->ptr, &buf, flags);
                                     ^
idna.c: In function ‘toUnicode’:
idna.c:128:41: error: ‘struct RString’ has no member named ‘ptr’
   rc = idna_to_unicode_8z8z(RSTRING(str)->ptr, &buf, flags);
                                         ^
idna.c: At top level:
cc1: warning: unrecognized command line option ‘-Wno-self-assign’
cc1: warning: unrecognized command line option ‘-Wno-constant-logical-operand’
cc1: warning: unrecognized command line option ‘-Wno-parentheses-equality’
cc1: warning: unrecognized command line option ‘-Wno-tautological-compare’
Makefile:241: recipe for target 'idna.o' failed
make: *** [idna.o] Error 1

make failed, exit code 2

Gem files will remain installed in /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2 for inspection.
Results logged to /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-0.0.2/gem_make.out
</pre>

Looks pretty much the same.