Arseny Zinchenko

Posted on Apr 2, 2019 • Originally published at rtfm.co.ua on Apr 2, 2019

JetBrains Upsource: installation, configuration on Linux with NGINX and SSL and PyCharm integration

#linux #jetbrains #upsource #nginx

Upsource is a self-hosted service from JetBrains for code reviews by teams of developers which can be integrated with any IDE from JetBrains.

Is a Java-based application with Apache Cassandra as a backend for storing data and indexes.

Minimal requirements – 8 GiB memory on a host – not surprise at all.

Up to 10 users is free for usage.

In this post – an installation and configuration example on an AWS EC2 instance and connection from PyCharm.

Behind Upsource will have NGINX with SSL from Let’s Encrypt.

NGINX and Let’s Encrypt client is already installed – check the TestRail: QA Test Case Management installation on Linux post for details.

Documentation is here>>>.

Installing Upsource

Create a new user:

root@bttrm-testrail:/home/admin# useradd -r upsource

Create a working directory:

root@bttrm-testrail:/var/www# mkdir upsource.example.com

Download an archive from the downloads page, unpack it:

root@bttrm-testrail:/var/www# wget https://download.jetbrains.com/upsource/upsource-2018.2.1154.zip
root@bttrm-testrail:/var/www# unzip upsource-2018.2.1154.zip

Move content to the working directory:

root@bttrm-testrail:/var/www# mv upsource-2018.2.1154/\* upsource.example.com/
root@bttrm-testrail:/var/www# cd upsource.example.com/

Launch scripts located at the bin/ catalog:

root@bttrm-testrail:/var/www/upsource.example.com# ll bin/
total 8
-rw-r--r-- 1 root root 3458 Dec 31 16:06 upsource.bat
-rwxr-xr-x 1 root root 2981 Dec 31 16:06 upsource.sh

Install Java:

root@bttrm-testrail:/var/www/upsource.example.com# apt install default-jdk
root@bttrm-testrail:/var/www/upsource.example.com# java -version
openjdk version "1.8.0\_212"

Launch Upsource.

Use run command for tests-debug and start to run the application in the background. Other commands can be found with help:

root@bttrm-testrail:/var/www/upsource.example.com# ./bin/upsource.sh help
Usage:

/bin/sh /var/www/upsource.example.com/bin/upsource.sh <command>

<command> can be one of the following:

configure

configures Upsource, call "help configure" for details

start

starts Upsource in background, call "help start" for details

...

For now – run with the run:

root@bttrm-testrail:/var/www/upsource.example.com# ./bin/upsource.sh run
Starting Upsource...
* Configuring JetBrains Upsource 2018.2
command [hostname]: hostname: Name or service not known
* Made default base-url 'http://127.0.0.1:8080/' from hostname '127.0.0.1' and listen port '8080'
* JetBrains Upsource 2018.2 runtime environment is successfully configured
* Loading logging configuration from /var/www/upsource.example.com/lib/ext/log4j.xml
* Redirecting JetBrains Upsource 2018.2 logging to /var/www/upsource.example.com/logs/internal/services/bundleProcess
* Configuring Service-Container[bundleProcess]
* Configuring Bundle Backend Service
* Configuring Configuration Wizard
command [hostname]: hostname: Name or service not known
* Starting Service-Container[bundleProcess]
* Starting Bundle Backend Service
* Starting Configuration Wizard
* JetBrains Upsource 2018.2 Configuration Wizard will be available on [http://127.0.0.1:8080?wizard_token=pikhg8iuYJ7OdssGkXbF] after start

Looks like works – go to SSL.

Let’s Encrypt

Obtain a new certificate via standalone authenticator as there is no NGINX config for a new virtual host yet:

root@bttrm-testrail:/var/www/upsource.example.com# systemctl stop nginx
root@bttrm-testrail:/var/www/upsource.example.com# /opt/letsencrypt/letsencrypt-auto certonly -d upsource.example.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
How would you like to authenticate with the ACME CA?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Nginx Web Server plugin (nginx)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-3] then [enter] (press 'c' to cancel): 2
Plugins selected: Authenticator standalone, Installer None
...
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/upsource.example.com/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/upsource.example.com/privkey.pem
...

NGINX

Create the virtual host config, documentation is here>>>:

server {
    listen 80;
    server_name  upsource.example.com;
    # Lets Encrypt Webroot
    location ~ /.well-known {
    root /var/www/html;
        allow all;
    }
    location / {
        # office1
        allow 194.***.***.24/29;
        # office2
        allow 91.***.***.78/32;
        # arseny home
        allow 188.***.***.48/32;
        # unknown
        #allow 93.***.***.125/32;
        deny  all;
        return 301 https://upsource.example.com;
    }
}
server {
    listen       443 ssl;
    server_name  upsource.example.com;
    root /var/www/upsource.example.com;
    access_log  /var/log/nginx/upsource.example.com-access.log;
    error_log /var/log/nginx/upsource.example.com-error.log warn;
    # office1
    allow 194.***.***.24/29;
    # office2
    allow 91.***.***.78/32;
    # arseny home
    allow 188.***.***.48/32;
    deny  all;
    ssl_certificate /etc/letsencrypt/live/upsource.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/upsource.example.com/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_dhparam /etc/nginx/dhparams.pem;
    ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:ECDHE-RSA-AES128-GCM-SHA256:AES256+EECDH:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
    ssl_session_timeout 1d;
    ssl_stapling on;
    ssl_stapling_verify on;
    location / {
        proxy_set_header X-Forwarded-Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_pass http://localhost:8080;
        proxy_pass_header Sec-Websocket-Extensions;
    }
}

Check and reload NGINX’s configs:

root@bttrm-testrail:/var/www/upsource.example.com# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
root@bttrm-testrail:/var/www/upsource.example.com# service nginx reload

Starting Upsource

Update /etc/security/limits.conf.

The documentation says to set limits for all users – but IMHO not the best idea, so set it only for our user:

upsource        -       memlock         unlimited
upsource        -       nofile          100000
upsource        -       nproc           32768
upsource        -       as              unlimited

Update the /etc/hosts file – it must have $HOSTNAME name added – otherwise, Apache Cassandra will not run.

Check the name:

root@bttrm-testrail:/var/www/upsource.example.com# hostnamectl
Static hostname: bttrm-testrail
...

Add it:

...
127.0.0.1 localhost bttrm-testrail
...

Change directory and files owner:

root@bttrm-testrail:/var/www/upsource.example.com# chown -R upsource:upsource /var/www/upsource.example.com/

Switch to the upsource user and run using the start command:

root@bttrm-testrail:/var/www/upsource.example.com# su -l upsource
bash
upsource@bttrm-testrail:~$ cd /var/www/upsource.example.com/
upsource@bttrm-testrail:/var/www/upsource.example.com$ ./bin/upsource.sh start
Starting Upsource...
* Configuring JetBrains Upsource 2018.2
* JetBrains Upsource 2018.2 runtime environment is successfully configured
* Loading logging configuration from /var/www/upsource.example.com/lib/ext/log4j.xml
* Redirecting JetBrains Upsource 2018.2 logging to /var/www/upsource.example.com/logs/internal/services/bundleProcess
* Configuring Service-Container[bundleProcess]
* Configuring Bundle Backend Service
* Configuring Starting Page Service
* Configuring Upsource Configurator
* Configuring Hub
* Configuring Bundle Hub Configurator
* Configuring Apache Cassandra
* Configuring Upsource Cluster Initialize Service
* Configuring Upsource Frontend
* Starting Service-Container[bundleProcess]
* Starting Bundle Backend Service
* Starting Starting Page Service
* JetBrains Upsource 2018.2 will be available on [https://upsource.example.com/bundle/starting] after start
* Starting Upsource Configurator
Upsource is running

On the first start, Upsource will generate an authorization token and will save it to the /var/www/upsource.example.com/conf/internal/services/configurationWizard/wizard_token.txt file.

`systemd` unit

To mange Upsource as a system service – create a /etc/systemd/system/upsource.service file:

[Unit]
Description=JetBrains Upsource
Requires=network.target
After=network.target
[Service]
User=upsource
Group=upsource
TimeoutStartSec=10min
WorkingDirectory=/var/www/upsource.example.com/
PIDFile=/var/www/upsource.example.com/logs/upsource.pid
ExecStart=/var/www/upsource.example.com/bin/upsource.sh start
ExecStop=/var/www/upsource.example.com/bin/upsource.sh stop
ExecReload=/var/www/upsource.example.com/bin/upsource.sh restart
[Install]
WantedBy=multi-user.target

Stop Upsource started manually:

upsource@bttrm-testrail:/var/www/upsource.example.com$ ./bin/upsource.sh stop

And run via systemd:

root@bttrm-testrail:/etc/nginx/conf.d# systemctl start upsource.service
root@bttrm-testrail:/etc/nginx/conf.d# systemctl status upsource.service
● upsource.service - Upsource Code Review service
Loaded: loaded (/etc/systemd/system/upsource.service; disabled; vendor preset: enabled)
Active: active (running) since Tue 2019-04-02 10:23:43 UTC; 2s ago
Main PID: 5356 (bash)
Tasks: 53 (limit: 4915)
CGroup: /system.slice/upsource.service
├─5356 /bin/bash upsource.sh start
...
Apr 02 10:23:43 bttrm-testrail systemd[1]: Started Upsource Code Review service.
Apr 02 10:23:44 bttrm-testrail bash[5356]: Starting Upsource...

Add to auto launch:

root@bttrm-testrail:/etc/nginx/conf.d# systemctl enable upsource.service
Created symlink /etc/systemd/system/multi-user.target.wants/upsource.service → /etc/systemd/system/upsource.service.