DEV Community: Remy Gardette

Using multiple SSH keys concurrently

Remy Gardette — Mon, 10 Jul 2023 14:05:13 +0000

When using multiple GitLab accounts (e.g. private account and work account) on the same computer, you can't use the same SSH key for both. This can become problematic when Git has to decide which key to use to communicate with the distant repository.

There are 2 main solutions to this problem. You can specify the SSH key to use:

based on the distant repository host,
or based on the project path.

The following examples assume that you have the 2 SSH keys ~/.ssh/id_rsa_perso and ~/.ssh/id_rsa_work.

Host-based configuration

The first solution requires you to change the URL of the repository when cloning it.

For example, when cloning a work project located at https://gitlab.com/my-org/my-project, the command would be:

git clone git@work-gitlab.com:my-org/my-project.git

and for a personal project located at https://gitlab.com/me/my-project:

git clone git@home-gitlab.com:me/my-project.git

Then, in the SSH configuration file (~/.ssh/config):

# Personal account:
Host home-gitlab.com
  HostName gitlab.com
  User git
  IdentityFile ~/.ssh/id_rsa_perso
  PreferredAuthentications publickey
  PasswordAuthentication no
  IdentitiesOnly yes

# Work account:
Host work-gitlab.com
  HostName gitlab.com
  User git
  IdentityFile ~/.ssh/id_rsa_work
  PreferredAuthentications publickey
  PasswordAuthentication no
  IdentitiesOnly yes

This works fine, but it requires you to always remember to modify the project URL when cloning it.

Path-based configuration

Since Git 2.13, a conditional include is available in the local Git configuration. This means that you can include a specific configuration file when your current path matches a specific filter.

For example, if your work projects are located in ~/work/sources and your personal project in ~/perso/sources, you can create a Git work-related configuration file ~/.gitconfig-work:

[user]
    email = "your.email@work.com"
[core]
    sshCommand = "ssh -i ~/.ssh/id_rsa_work"

and a personal configuration file ~/.gitconfig-perso:

[user]
    email = "your.email@home.com"
[core]
    sshCommand = "ssh -i ~/.ssh/id_rsa_perso"

Now you only need to include the correct configuration file based on your path, by modifying the global ~/.gitconfig file and including:

[includeIf "gitdir:~/work/sources/"]
    path = .gitconfig-work
[includeIf "gitdir:~/perso/sources/"]
    path = .gitconfig-perso

Conclusion

I personally find the path-based solution much easier to use, since you only need to change your configuration once, and it automatically works in the future (as long as you use the correct folders for your projects). The host-based solution requires you to always be careful to change the URL when cloning a repository. But it can be useful if you don't want to group your projects in specific local directories.

Originally published at https://remyg.fr.

Deploying Jekyll with GitLabCI

Remy Gardette — Wed, 27 Feb 2019 10:00:00 +0000

This post was first published on my blog.

My blog is based on Jekyll, a static website generator. This means that the pages need to be generated before they’re deployed. Until recently, I used to build the content locally using a Jekyll Docker image, commit and push the generated content to my GitHub repo, SSH to my web server, and pull the changes from the repo.

The process was quite annoying, which is why I decided to set up a CI/CD (Continuous Integration and Delivery) pipeline, using GitLabCI.

Pipeline

The CI/CD pipeline I’ve implemented is the following:

The pipeline is defined in a file .gitlab-ci.yml at the root of the project:

image: ruby:2.5

cache:
  key: "jekyll"
  paths:
    - vendor

variables:
  JEKYLL_ENV: production
  LC_ALL: C.UTF-8
  NOKOGIRI_USE_SYSTEM_LIBRARIES: "true"

before_script:
- bundle install

test:
  stage: test
  script:
  - bundle exec jekyll build -d ./public
  - bundle exec htmlproofer ./public --only-4xx --check-favicon --check-html --assume-extension
  artifacts:
    paths:
    - public

deploy:
  stage: deploy
  before_script:
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 -d)
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh
  - echo "$SSH_HOST_KEY" > ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts
  - which rsync || ( apt-get update -y && apt-get install rsync -y )
  script:
  - ssh -p$SSH_PORT $SSH_SRV "mkdir -p /tmp/jekyll"
  - ssh -p$SSH_PORT $SSH_SRV "rm -rf /tmp/jekyll_old"
  - ssh -p$SSH_PORT $SSH_SRV "mv /tmp/jekyll /tmp/jekyll_old"
  - rsync -rav -e "ssh -p $SSH_PORT" --exclude='.git/' --exclude='.gitlab-ci.yml' --delete-excluded ./public $SSH_SRV:/tmp/jekyll
  - ssh -p$SSH_PORT $SSH_SRV "sh /home/gitlabci/rsync-jekyll.sh"
  only:
  - master

It consists of 2 steps: test and deploy.

The test step builds the Jekyll project, then runs HTMLProofer on the generated files. This step will check and validate the HTML output.

The deploy step transfers the generated files to my home gateway, then runs a script (located on the gateway) to transfer those files to my web server (the web server is not exposed outside of my network).

SSH Configuration

The gateway is accessed via SSH. To do that, I’ve created a specific SSH key only used by GitLab CI.

To generate a new SSH key:

ssh-keygen -o -t rsa -b 4096 -C "gitlabci"

The public key needs to be added to the authorized keys before it can be used:

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

Configuration

The pipeline contains variables to improve the security:

SSH_PRIVATE_KEY: a base-64 representation of the SSH private key used to access my gateway. The value is retrieved by running this command on the gateway: cat id_rsa > base64 -w0
SSH_HOST_KEY: the RSA key of the gateway. Retrieved by running on the gateway: ssh-keyscan -t rsa server_ip
SSH_SRV: the SSH connection information, like my_user@my_server
SSH_PORT: the SSH port on the gateway

Usage

I can now follow the following process to publish changes to my website:

if needed, clone the GitHub repo on my computer
make the changes (create a new blog post,…)
test the changes locally (the Docker compose file allows me to build and serve the website on my computer)
commit and push the changes
- if I push to a branch other than master, only the test step is executed
- if I push to master, the whole pipeline is executed, so my site is built, tested and automatically deployed.

NGINX Reverse Proxy

Remy Gardette — Sun, 29 Jul 2018 10:00:00 +0000

My home server setup is composed of several Raspberry Pi, where I host different web applications (this blog, an RSS reader, some home IOT apps…). I’ve decided to setup a front gateway, that proxies the request to the right server:

The requests are proxied by an NGINX reverse proxy, running in a Docker container on the gateway. It redirects the HTTP requests based on the host (eg. remyg.ovh runs on rpi1 when rss.remyg.ovh runs on rpi2).

NGINX Configuration

The main NGINX conf file (nginx.conf) looks like this:

user nginx;
worker_processes 1;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log /var/log/nginx/access.log main;

    sendfile on;
    #tcp_nopush on;

    keepalive_timeout 65;

    #gzip on;

    include /etc/nginx/sites-enabled/*.*;
}

The only difference with the base conf file (from the default NGINX Docker image) is the last line:

include /etc/nginx/conf.d/*.conf;

is replaced by

include /etc/nginx/sites-enabled/*.*;

It ignores the default configuration (/etc/nginx/conf.d/default.conf) and uses the proxy configuration files that I defined.

Hosts Configuration

Each host has its own configuration file:

for remyg.ovh , running on rpi1 (with a local IP 192.168.0.10, and port 8080):

server {
    listen 80;
    server_name remyg.ovh;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
    location / {
        proxy_pass http://192.168.0.10:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect off;
    }
}

for rss.remyg.ovh , running on rpi2 (with a local IP 192.168.0.11, and port 8081):

server {
    listen 80;
    server_name rss.remyg.ovh;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
    location / {
        proxy_pass http://192.168.0.11:8081;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect off;
    }
}

These files indicate that a request incoming to rss.remyg.ovh:80 (server_name and listen) will be redirected to 192.168.0.11:8081 (proxy_pass).

That’s all the configuration you need to serve websites on HTTP.

Running in Container

To run the reverse proxy in a Docker container, the file tree looks like this:

nginx-reverse-proxy
  -> conf
    -> nginx.conf
  -> sites
    -> remyg.ovh
       rss.remyg.ovh

With this structure, the command launching the container will be:

docker run --name mynginx-proxy \
-v /home/pi/nginx-reverse-proxy/sites:/etc/nginx/sites-enabled:ro \
-v /home/pi/nginx-reverse-proxy/conf/nginx.conf:/etc/nginx/nginx.conf:ro \
-p 80:80 -d nginx:alpine

HTTPS

To enable HTTPS on the different sites, I’m using Let’s Encrypt, and their utility app Certbot.

I’m starting by installing the certbot package:

sudo apt install certbot

When generating a certificate, Certbot will need to validate that it can access a specific file that it generates, pointing to the URL http://your-host/.well-known/acme-challenge/{token}. To do that, start by creating and mounting a new volume on the reverse proxy container:

docker run --name mynginx-proxy \
        -v /home/pi/nginx-reverse-proxy/sites:/etc/nginx/sites-enabled:ro \
        -v /home/pi/nginx-reverse-proxy/conf/nginx.conf:/etc/nginx/nginx.conf:ro \
        -v /home/pi/letsencrypt_www:/var/www/letsencrypt \
        -p 80:80 -p 443:443 -d nginx:alpine

Then specify in the sites proxy configuration that this volume is used when pointing to /.well-known/acme-challenge/:

server {
    listen 80;
    server_name remyg.ovh;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    location /.well-known/acme-challenge/ {
        root /var/www/letsencrypt;
    }

    location / {
        proxy_pass http://192.168.0.10:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect off;
    }
}

And reload your NGINX config:

docker exec -it mynginx-proxy nginx -s reload

Now you can generate the certificate(s) :

sudo certbot certonly --authenticator webroot -w /home/pi/letsencrypt_www -d remyg.ovh -d rss.remyg.ovh

This will generate the ACME challenge files in /home/pi/letsencrypt_www, and validate the challenge. It will also generate the certificates, in /etc/letsencrypt/certs/live/remyg.ovh/ and /etc/letsencrypt/certs/live/rss.remyg.ovh/.

The last step is to use the new certificates, and only allow HTTPS requests.

Start by mounting a new volume, containing the certificates:

docker run --name mynginx-proxy \
        -v /home/pi/nginx-reverse-proxy/sites:/etc/nginx/sites-enabled:ro \
        -v /home/pi/nginx-reverse-proxy/conf/nginx.conf:/etc/nginx/nginx.conf:ro \
        -v /etc/letsencrypt:/etc/nginx/certs \
        -v /home/pi/letsencrypt_www:/var/www/letsencrypt \
        -p 80:80 -p 443:443 -d nginx:alpine

Then update your proxy configuration:

server {
    listen 80;
    server_name remyg.ovh;
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    location /.well-known/acme-challenge/ {
        root /var/www/letsencrypt;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name remyg.ovh;

    ssl_certificate certs/live/remyg.ovh/fullchain.pem;
    ssl_certificate_key certs/live/remyg.ovh/privkey.pem;

    location / {
        proxy_pass http://192.168.0.10:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect off;
    }
}

Reload the NGINX configuration, and you’re all set!

Hi, I'm Remy Gardette

Remy Gardette — Fri, 30 Jun 2017 21:29:23 +0000

I have been coding for 8 years.

You can find me on GitHub as RemyG

I live in Lille, France.

I work for IBM Client Innovation Center.

I mostly program in these languages: Java / JEE.

Nice to meet you.