Upspin part 3: Configuring an Upspin server [WIP]
Jul 2018    |    upspin   upspinserver  


Please note: this is a work in progress and some items below may change without notice

Intro

In this tutorial we’ll be setting up an upspinserver on an Ubuntu 18.04 x64 VM on DigitalOcean.

Our upspinserver will be composed of a directory and store server, aka dir/store. In theory you could host them separately, but for simplicity we’ll keep them together on the same machine.

As a side note, the upspin-ui command can automate an Upspin deployment to GCP, but we’ll be taking the manual route.

Do remember it’s irrelevant which platform you choose.

The official docs can be found here: Setting up upspinserver.


Prerequisites

In part 2 you already setup an Upspin user. For this tutorial I’ll be using: fridman0101@gmail.com

You should also have an Internet domain where you can add DNS records. I’ll be using: upspin.online

and the host name of the server (for both dir/store) will be: my.upspin.online


Tell Upspin about host name

We do not need to run upspin signup because we already created a user with upspin-ui in part 2.

Instead we need to tell the Upspin world that we have chosen a location for our dir/store server, accessible at the same host name.

Currently the keyserver has no knowledge our dir/store server:

"Dirs":null,"Stores":null

So we modify our $HOME/upspin/config file by adding dirserver: and storeserver:, substitute with your own domain.

Again, for the sake of this tutorial dir/store will live on the same machine and accessible with a single host name.

username: fridman0101@gmail.com
dirserver: my.upspin.online
storeserver: my.upspin.online
packing: ee
cmdflags:
  cacheserver:
    writethrough
upspin user | upspin user -put
# which will update the keyserver 
# upspin: user: local configuration differs from public record in key server:
# dirs in configuration: [remote,my.upspin.online:443]
# dirs in key server: []

Now the keyserver will return:

"Dirs":["remote,my.upspin.online:443"],"Stores":["remote,my.upspin.online:443"]

Configure domain

We will not be using the -cluster flag, because dir/store live on the same machine.

When -cluster is not specified, keys for a single user (upspin@domain) are generated

upspin setupdomain -domain=upspin.online
# which will return a bunch of text...

A few things to note:

  1. The “secret seed”:
    Write this down and store it in a secure, private place.
    Do not share your secret key or this string with anyone.
  2. $HOME/upspin/deploy/upspin.online is created

Next we need to prove that our user name, fridman0101@gmail.com is the same user that owns (or has access to) the domain upspin.online

This step will vary depending on your internet domain registrar, here is an example for namecheap:

name-cheap-1.png

Domain List > Advanced DNS > add a TXT record as specified in the resulting text from the upspin setupdomain -domain=upspin.online command.

name-cheap-2.png

At this point you’ll be waiting either a few minutes or a few hours for changes to propagate. You can run the following command to check the status:

host -t TXT upspin.online
# should return
# upspin.online descriptive text "upspin:a4386744462680e3c2243ce4cd409eb49402452f6a241..."

# oops
# .. has no TXT record
# either keep waiting or double-check your configuration (consult the official Upspin docs if necessary)

Setup a static IP

This step will be carried out on your cloud provider, e.g., AWS, GCP, Linode, etc. in my case it’s DigitalOcean.

You’ll want your VM to have a static IP. On DigitalOcean navigate to Networking > Floating IPs and assign your droplet a floating, i.e., static, IPv4 address.

digital-ocean-1.png

Create DNS record

Now we tell our registrar that our subdomain, my.upspin.online, should point at the static IP of our VM.

Here is a screenshot from namecheap or follow this guide

name-cheap-3.png

Build upspinserver binary with Go

We’re now going to ssh to our remote VM, in my case it’s a droplet on DigitalOcean running:

lsb_release -d
# Description:  Ubuntu 18.04 LTS
1 - Install Go on 64-bit x86 Linux
wget -qO- https://dl.google.com/go/go1.10.3.linux-amd64.tar.gz | sudo tar -C /usr/local -xz
export PATH=$PATH:/usr/local/go/bin # or add to .bash_profile
go version
# go version go1.10.3 linux/amd64
2 - Build upspinserver binary
export GOPATH="$HOME/go"
export PATH="$PATH:$GOPATH/bin"
# or add exports to .bash_profile

go get -u upspin.io/cmd/upspinserver
# this will install the binary to $HOME/go/bin

Configure upspinserver on Ubuntu 18.04

We’ll setup a non-root user, in my case it’ll be mfridman, substitute your own.

These commands must be executed as root

useradd -m mfridman -s /bin/bash
# passwd mfridman # no need for passwd, just keeping this here
su mfridman
cd $HOME
mkdir -p upspin/letsencrypt
mkdir .ssh
chmod 0700 .ssh
# copy your public key (for easy ssh access) to .ssh/authorized_keys
vi .ssh/authorized_keys
chmod 0600 .ssh/authorized_keys
# create a systemd file, assumes you're running systemd
touch /etc/systemd/system/upspinserver.service

add the following systemd service definition to the above .service file

[Unit]
Description=Upspin server

[Service]
ExecStart=/home/mfridman/upspinserver
User=mfridman
Group=mfridman
Restart=on-failure

[Install]
WantedBy=multi-user.target

Copy the upspinserver to our newly created user’s home directory and change its ownership:

# we're doing this because we were previously logged in as root
cp $HOME/go/bin/upspinserver /home/mfridman && chown mfridman:mfridman /home/mfridman/upspinserver

This chunk is copied directly from the official docs

Normally only user root can bind ports below 1024. Instead of running upspinserver as root (which is generally discouraged), we will grant the upspinserver binary this capability by using setcap (as root):

setcap cap_net_bind_service=+ep /home/mfridman/upspinserver

Note that you need to run this setcap command whenever the upspinserver binary is updated.

And finally enable the upspinserver systemd service (as root):

systemctl enable --now /etc/systemd/system/upspinserver.service
# to stop service 
systemctl stop upspinserver.service
# start
systemctl start upspinserver.service
# view logs 
journalctl -f -u upspinserver

Is it working? open a web browser and navigate to your host name, i.e., subdomain, https://my.upspin.online and you should see Unconfigured Upspin Server

Also, the server at this point is running in “setup mode”. If you check the logs there will be a line that’ll read:

unable to read configuration: open /home/mfridman/upspin/server/serverconfig.json: no such file or directory

This is good, real good!

Configure upspinserver

Now we go back to our local machine, where we ran the command upspin setupdomain -domain=upspin.online in the Configure a custom domain section and run:

Note that this command does A LOT.

It registers the user created by ‘setupdomain’ domain with the key server,
copies the configuration files from $where/$domain to the upspinserver and
restarts it, puts the Writers file, and makes the root for the calling user.

upspin setupserver -host=my.upspin.online -domain=upspin.online
# should see 
# Successfully put "upspin@upspin.online" to the key server.
# Configured upspinserver at "my.upspin.online:443".
# Created root "fridman0101@gmail.com".

Side note, if you mess up navigate to advanced tips and tricks, I’ll have a section on fixing this up.

On the remote server, you’ll notice a new folder /home/mfridman/upspin/server which contains a bunch of goodies.

If you open the browser and navigate to your Upspin URL: https://my.upspin.online/ it should return a 404

Test your installation

Place a file into the Upspin world:

echo "We made it" | upspin put fridman0101@gmail.com/made-it.txt

Retrieve that file

upspin get fridman0101@gmail.com/made-it.txt
# We made it