DEV Community: Timofey Chuchkanov

Encrypted Remote Terraform State with Postgres and LUKS

Timofey Chuchkanov — Wed, 03 Jan 2024 12:33:07 +0000

Either if you have already decided to use a PostgreSQL remote state backend for Terraform, or stumbled on this guide out of curiosity, let me help you set everything up in a more secure manner 😁!

*upd 12/26/2024: Dev Community formatting has gone nuts
edit 2024-01-20: Added a tip on workspaces.

Why?
Prerequisites
What Versions of Software are Used?
Set Up a Postgres Server
- Prepare Storage
- Configure Postgres
Configure a Terraform Client
What to Do After a Reboot?
The Result

Why?

When you work with Terraform by yourself, don't store sensitive data in state, and have a couple of projects, it's OK to use the local state backend.

But as soon as the number of projects increases, you start working in a team, and suddenly realize there're things like passwords and API keys in your state file, uhmm... Things start to get tricky 🫠.

A number of questions arise:

How do I protect sensitive data stored in state?
How can I collaborate with my colleagues? The state must be somehow shared in real time.

That's where a remote state backend comes in handy. It can be encrypted, allows to team up on the same projects simultaneously by having a SSOT for state, and supports locking to prevent collisions.

But Why PostgreSQL?

It's quicker and easier to set up than other remote state backends. It doesn't require a ton of resources. The thing just works, yet highly flexible.

Prerequisites

What you need is a single server and a way to obtain a TLS certificate for it. For me, it's a VM on Proxmox and a self-signed certificate.

If you want to use self-signed certs for this, check out my old article on how to generate them via openssl 👇.

Trusted self-signed TLS certificates for dummies (w/ thorough explanations included)

Timofey Chuchkanov ・ Nov 25 '22

#security #https #ssl #beginners

Play around and find the exact amount of resources suitable for your specific case. Maybe less cores and RAM, more storage, etc.

Ubuntu was chosen as one of the most popular server distros. The configuration can be adjusted for any OS.

OS	CPU	RAM	Disk
Ubuntu Server	2	2 GB	8 GB + 1 GB

📒 NOTE: For this demo, 1 GB disk is dedicated for a Postgres database cluster. Be aware, in production such disk size may be inappropriately small.

What Versions of Software are Used?

Ubuntu 22.04.3 LTS
PostgreSQL 14.10
cryptsetup 2.4.3
Terraform v1.5.7 (on a separate host)

What is LUKS, btw 🤔?
Check out cryptsetup's README to learn more: https://gitlab.com/cryptsetup/cryptsetup/

Set Up a Postgres Server

Prepare Storage

Create a LUKS Device

First, we need to find out our disk's block device name. In my case, it's /dev/sdb.

$ lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0     7:0    0  63.5M  1 loop /snap/core20/2015
loop1     7:1    0 111.9M  1 loop /snap/lxd/24322
loop2     7:2    0  40.8M  1 loop /snap/snapd/20092
sda       8:0    0     8G  0 disk 
├─sda1    8:1    0   7.9G  0 part /
├─sda14   8:14   0     4M  0 part 
└─sda15   8:15   0   106M  0 part /boot/efi
sdb       8:16   0     1G  0 disk 
sr0      11:0    1     4M  0 rom

Now, create a new partition table and a partition on this block device.

I prefer to use GPT instead of MBR even for smaller disks most of the time.

# fdisk /dev/sdb

fdisk example

Welcome to fdisk (util-linux 2.37.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x20d6b116.

Command (m for help): g
Created a new GPT disklabel (GUID: 0B0CD5AB-337A-CF47-A2BF-F377A158D0FE).

Command (m for help): n
Partition number (1-128, default 1): 
First sector (2048-2097118, default 2048): 
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-2097118, default 2097118): 

Created a new partition 1 of type 'Linux filesystem' and of size 1023 MiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

When the partition is created, LUKS can be initialized using cryptsetup.

Considering that block device naming in Linux is not persistent between reboots, we won't rely on device names. Instead, we'll use labels for both a LUKS partition and a filesystem on it later.

# cryptsetup luksFormat /dev/sdb1 --type luks2 --label tfstate-pg-luks

👆 LUKS version 2 is used because it supports labels. Use any label of your liking. Don't forget to adjust all the commands and configurations.

Configure crypttab and fstab

Opening an encrypted disk partition requires providing a password or a key file. We could automate this process, but as always, there's a trade-off between security and convenience.

/etc/crypttab and /etc/fstab — these two files can greatly simplify the experience of opening and mounting an encrypted disk manually after reboots.

crypttab

Open /etc/crypttab with your editor of choice and add the line below:

tfstate-pg-luks LABEL=tfstate-pg-luks   none    luks,noauto

📒 NOTE: 👆 The noauto option prevents automatic opening and mapping of the partition.

fstab

Do the same thing with /etc/fstab.

📒 NOTE: Don't forget to change the filesystem name here if you don't use Btrfs 👇.

You can use a single mount point, and mount the device straight to /var/lib/postgres. It's all a matter of preference. I like my devices to be mounted under /media, and then bind-mounted to other places.

LABEL=tfstate-pg        /media/tfstate-pg       btrfs   defaults,noauto 0       0
/media/tfstate-pg       /var/lib/postgres       none    bind,noauto     0       0

Create a Filesystem

Choose whatever filesystem you're comfortable with. Don't forget to update fstab accordingly.

To create a filesystem on the encrypted disk partition, connect to it first.

# cryptdisks_start tfstate-pg-luks

Create the filesystem itself with the label specified in fstab:

# mkfs.btrfs --label tfstate-pg

Mount and Prepare the Filesystem

At this point, we're halfway there.

Create directories for the mount points specified in /etc/fstab, and mount the LUKS-encrypted partition.

# mkdir /media/tfstate-pg /var/lib/postgres
# mount /media/tfstate-pg/
# mount /var/lib/postgres/

To check if everything is appropriately mounted, issue findmnt.

$ findmnt

Example output

$ findmnt
***OUTPUT TRUNCATED***
├─/var/lib/postgres                           /dev/mapper/tfstate-pg-luks btrfs       rw,relatime,space_cache=v2,subvolid=5,subvol=/
└─/media/tfstate-pg                           /dev/mapper/tfstate-pg-luks btrfs       rw,relatime,space_cache=v2,subvolid=5,subvol=/

The directory where the database cluster will be located must be owned by the postgres user and group.

# chown -R postgres: /var/lib/postgres

Configure Postgres

Install

# apt install postgresql

Disable and Stop the Service

Since the database cluster will be located on the encrypted partition that must be manually connected after a reboot, the Postgres service should be disabled.

# systemctl disable --now postgresql

💡 TIP: 👆 The --now option stops the daemon.

Copy a TLS Certificate and Key

Encrypted storage is useless if the network traffic between the server and its remote clients is not encrypted.

Grab a TLS certificate and its key, and copy them to your server. You can place them wherever you want — filenames don't matter either, as long as the postgres user has access.

# mv deathroll-internal.pem /etc/ssl/certs/
# mv deathroll-internal-key.pem /etc/ssl/private/
# chown postgres: /etc/ssl/private/deathroll-internal-key.pem

File permissions

$ stat --printf '%n\nMode: %a (%A)\nOwner: %u (%U)\nGroup: %g (%G)\n' /etc/ssl/certs/deathroll-internal.pem
/etc/ssl/certs/deathroll-internal.pem
Mode: 644 (-rw-r--r--)
Owner: 1000 (t_chuchkanov)
Group: 1000 (t_chuchkanov)
$ sudo -u postgres stat --printf '%n\nMode: %a (%A)\nOwner: %u (%U)\nGroup: %g (%G)\n' /etc/ssl/private/deathroll-internal-key.pem
/etc/ssl/private/deathroll-internal-key.pem
Mode: 600 (-rw-------)
Owner: 113 (postgres)
Group: 122 (postgres)

Configure

Settings

Instead of editing the config file manually, it's easier and quicker to use pg_conftool.

# pg_conftool set ssl_cert_file /etc/ssl/certs/deathroll-internal.pem
# pg_conftool set ssl_key_file /etc/ssl/private/deathroll-internal-key.pem
# pg_conftool set listen_addresses localhost,tfstate-pg.deathroll.internal

To check if all the values are correct:

$ for P in listen_addresses ssl_cert_file ssl_key_file; do pg_conftool show $P; done

Access Control

Configure access control for your Postgres installation. Append the line below to the /etc/postgresql/14/main/pg_hba.conf file:

📒 NOTE: 👆 Pay attention to the Postgres version. It may differ in your case.

hostssl all             all             172.17.0.0/24            scram-sha-256

📒 NOTE: 👆 Change the allowed network to whatever you need. It may even be 0.0.0.0/0.

Start

# systemctl start postgresql.service

If your configuration is correct, the Postgres systemd unit must be in the running state, the Postgres cluster online, and the daemon listening on the port 5432. Congrats 🥳.

$ systemctl list-units postgres*.service

Example

$ systemctl list-units postgres*.service
  UNIT                       LOAD   ACTIVE SUB     DESCRIPTION               
  postgresql.service         loaded active exited  PostgreSQL RDBMS
  postgresql@14-main.service loaded active running PostgreSQL Cluster 14-main

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
2 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

$ pg_lsclusters

Example

$ pg_lsclusters
Ver Cluster Port Status Owner    Data directory              Log file
14  main    5432 online postgres /var/lib/postgresql/14/main /var/log/postgresql/postgresql-14-main.log

# ss -tlpn | grep postgres

Example

# ss -tlpn | grep postgres
LISTEN 0      244      172.17.0.40:5432      0.0.0.0:*    users:(("postgres",pid=19728,fd=7))       
LISTEN 0      244        127.0.0.1:5432      0.0.0.0:*    users:(("postgres",pid=19728,fd=6))       
LISTEN 0      244            [::1]:5432         [::]:*    users:(("postgres",pid=19728,fd=5))

Almost there! The last few steps of Postgres configuration are left.

Configure a Database

Now we need to:

Change the postgres user password
Create a Postgres user for Terraform clients
Create a database for Terraform state
Grant some privileges on this database to the Terraform user

Let's roll!

$ sudo -u postgres psql

📒 NOTE: 👆 Depending on your $PWD, psql may throw a warning about its inability to access this directory. Ignore it.

postgres=# ALTER USER postgres ENCRYPTED PASSWORD '******';
postgres=# CREATE USER terraform ENCRYPTED PASSWORD '******';
postgres=# CREATE DATABASE terraform_backend;
postgres=# GRANT CREATE,CONNECT,TEMPORARY ON DATABASE terraform_backend TO terraform;

Example

Configure a Firewall

Don't forget to configure a firewall on your system.

# ufw allow 22,5432/tcp
# ufw enable

To check the status:

# ufw status

That's it! It's time to test the connection from another host and configure a Terraform client 😎.

Check Connection from Another Host

Install psql and issue the command below:

$ psql postgres://terraform@tfstate-pg.deathroll.internal/terraform_backend

💡 TIP: The connection string scheme is: postgres://<user>@<host>/<database_name>

💡 TIP: On Ubuntu, the package is called "postgresql-client".

You can quit out of the psql client prompt right after it establishes the connection, because there's nothing to do in it apart from testing connectivity.

Example

$ psql postgres://terraform@tfstate-pg.deathroll.internal/terraform_backend
Password for user terraform: 
psql (15.5 (Debian 15.5-0+deb12u1), server 14.10 (Ubuntu 14.10-0ubuntu0.22.04.1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.

terraform_backend=> \dn
  List of schemas
  Name  |  Owner   
--------+----------
 public | postgres
(1 row)

terraform_backend=>

Configure a Terraform Client

💡TIP: When you need to work with multiple terraform projects, use workspaces, since all of the freshly initialized projects have only the default workspace. Hence, there will be remote state conflicts, if a single workspace is used for several independent states.

In a Terraform project's main file, create a backend block inside of terraform. Provide a string value for conn_str.

It is really that simple.

terraform {
  backend "pg" {
    conn_str = "postgres://terraform@tfstate-pg.deathroll.internal/terraform_backend"
  }
}

Before issuing any Terraform commands, export the Postgres password as an environment variable:

$ read -s PGPASSWORD
$ export PGPASSWORD

For a fresh project, terraform init is sufficient. For an existing project, add the -reconfigure flag to this command.

$ terraform init

Initializing the backend...

Successfully configured the backend "pg"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding telmate/proxmox versions matching "2.9.11"...
- Installing telmate/proxmox v2.9.11...
- Installed telmate/proxmox v2.9.11 (unauthenticated)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

╷
│ Warning: Incomplete lock file information for providers
│ 
│ Due to your customized provider installation methods, Terraform was forced to calculate lock file checksums locally for the following providers:
│   - telmate/proxmox
│ 
│ The current .terraform.lock.hcl file only includes checksums for linux_amd64, so Terraform running on another platform will fail to install these providers.
│ 
│ To calculate additional checksums for another platform, run:
│   terraform providers lock -platform=linux_amd64
│ (where linux_amd64 is the platform to generate)
╵

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

If you want to learn how to use Terraform providers in isolated environments, I have a yet another article for you 😉.

Using Terraform Providers in Isolation

Timofey Chuchkanov ・ Aug 6 '23

#devops #tutorial #opensource #terraform

What to Do After a Reboot?

Open and map the LUKS-encrypted partition
Mount its filesystem
Start the Postgres service

As simple as three to four commands!

The Result

As the result, we have a PostgreSQL server with a database cluster located on a LUKS-encrypted partition.

This Postgres installation is used for storing remote Terraform state, and the network traffic between it and its remote clients is encrypted.

Sounds great, innit 🤌?

Scroll down to the comments section to get some enhancement tips.

Using Terraform Providers in Isolation

Timofey Chuchkanov — Sun, 06 Aug 2023 07:54:32 +0000

*upd 12/26/2024: Dev Community formatting has gone nuts

Is This Guide for Me?

If you need to use Terraform providers in an isolated environment, or in case you don't have access to the Terraform Registry, look no further.

The Telmate/proxmox provider is used as an example in this guide. Configurations are performed on a Linux distribution, but can be applied anywhere with slight modifications.

Quick tip: To install Terraform CLI on Linux or Mac without hassle, check out Homebrew.

Find the Source Code
Get the Binaries
Place the Binaries Somewhere Locally
Configure the Terraform CLI
Test It Out!

Find the Source Code

Option 1. Go to GitHub and Search for Providers There

By convention, all providers' names start with the terraform-provider- prefix. Try typing this prefix in the search bar in combination with the name of the target provider — no hyphens needed. For example, terraform provider proxmox.

There's a high chance the first link will be a good choice. Also, I'd look for a greater number of stars. Most of the time, popular projects have lots of them.

Option 2. Search the Registry Beforehand or Ask Someone to Do This for You

Yeah, I know. That's kind of obvious.

Just type the name of the target provider in the search bar and pick one. Each provider's main page has a link to the source code.

Get the Binaries

Many providers' devs are kind enough to provide archives with binaries compiled for various platforms. They are located on the Releases page.

But Not Always

That is the case with HashiCorp, of course.

If there're no binaries, then you have to compile the provider yourself. Search for docs in the repo.

Place the Binaries Somewhere Locally

Terraform expects a directory structure of one of the two layouts:

Packed (.zip archive)
Unpacked (binaries)

About the Layouts
From the official documentation:

Packed layout: HOSTNAME/NAMESPACE/TYPE/terraform-provider-TYPE_VERSION_TARGET.zip is the distribution zip file obtained from the provider's origin registry.

Unpacked layout: HOSTNAME/NAMESPACE/TYPE/VERSION/TARGET is a directory containing the result of extracting the provider's distribution zip file.

Packed Layout Visualized

YOUR.REGISTRY.HOSTNAME/
└── ORGANIZATION-NAME
    └── PROVIDER
        └── terraform-provider-PROVIDER_X.X.X_PLATFORM.zip

We'll cover the first one.

There're no strict rules on where you should place a local mirror on the filesystem. Just make sure the files are accessible by other users if you're not the only one using the system.

I'm a Linux user, so according to FHS, /usr/libexec is a suitable option. There will be a directory for Terraform. Providers will have a separate directory under it — terraform/providers.

/usr/libexec summary
From the Linux Foundation Referenced Specifications website:

/usr/libexec includes internal binaries that are not intended to be executed directly by users or shell scripts. Applications may use a single subdirectory under /usr/libexec.

Create a proper directory layout
```
# mkdir -p /usr/libexec/terraform/providers/registry.terraform.io/Telmate/proxmox
```
Note: The path before registry.terraform.io is up to you, but after it the layout must be either of two types mentioned earlier.

Copy the archive(s) so the result looks similar to this:

deathroll@hellish:~$ tree /usr/libexec/terraform/
/usr/libexec/terraform/
└── providers
    └── registry.terraform.io
        └── Telmate
            └── proxmox
                ├── terraform-provider-proxmox_2.9.11_linux_amd64.zip
                └── terraform-provider-proxmox_2.9.14_linux_amd64.zip

5 directories, 2 files
deathroll@hellish:~$

Configure the Terraform CLI

According to the documentation, the config file for the cli utility on Linux is placed in the home directory and is called .terraformrc.

Create the config file.

Populate it with the following content:

Make sure to replace the path with your own.

provider_installation {
  filesystem_mirror {
    path    = "/usr/libexec/terraform/providers"
    include = ["*/*"] # */* is a shorthand for registry.terraform.io/*/*
  }

  direct {
    exclude = ["*/*"]
  }
}

Note: The path doesn't include the registry host name.

Test It Out!

Now you can go to your Terraform project directory and start working on it. Everything should work as expected.

A Linux Geek Tried PowerShell — It Feels Powerful Indeed

Timofey Chuchkanov — Sun, 14 May 2023 10:47:19 +0000

This article represents my personal opinion and is meant mostly for Linux (and other Unix-like) users, but feel free to join anyways ;)

*upd 12/26/2024: Dev Community formatting has gone nuts
Update 2023-05-17: Changed the parameters examples formatting.

Preamble
The Good Stuff
- Working With Types
- Object-oriented Approach
- Standard Cmdlet Names
- Script and Function Parameters
- Full Integration With the Underlying Platform (.NET)
Conclusion

Preamble

I'm on the finish line to get a college degree as a Network and System Administrator. Even though I use Unix-like operating systems for more than five years and want to specialize in them — especially those based on the Linux kernel — the enterprise still runs on Windows partially. That's why even we, the Unix(-like) enthusiasts, still may have to learn it to some extent.

Less than a week ago, I was preparing for one of the last exams of the last semester. As a part of an assignment, an administrator is to create multiple AD OUs and populate them with a defined number of users. Each user is a member of a security group with a name similar to the OU's. Since the number of users for each OU is somewhat huge, up to 30, the most productive way to create them is via a script. A PowerShell script. It takes less than five minutes to write such a script but can save a lot of time and prevent headaches in the future. In a real-world scenario, this script could be placed in a version control system repository.

While I was writing the script, I learned a bit about PowerShell and liked how some things are approached. The Unix-like systems' communities would benefit from borrowing some techniques or even using the PowerShell itself, I think. Don't get me wrong, I know who's behind this project, and not trying to convince anyone to abandon their beloved Unix shells ;)

The Good Stuff

Working With Types

Oh boy, it is so satisfying when a shell enables you to work with types like most of the modern programming languages do. Type casting? Check. Type-specific auto-completion? Here you go. Explicit type declarations? Check.

While using this shell, one can nearly stop worrying about weird behavior when working with data of mixed types. For example, collections like arrays and lists may contain objects of one specific type or multiple types.

Unambiguous Types
A String is a String, a Double is a Double, and so on...

PS /home/deathroll> [Math]::PI.GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Double                                   System.ValueType

PS /home/deathroll> $(665).GetType() 

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Int32                                    System.ValueType

PS /home/deathroll>

Type Casting

PS /home/deathroll> $([Int] 14.15)          
14
PS /home/deathroll> $([String] 14).GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     String                                   System.Object

PS /home/deathroll>

Type-Specific Auto-Completion
It also prints the last command containing the input string. It can be inserted by pressing the right arrow key (→).

Yes, what you see is indeed PowerShell on a Linux distribution.

Object-oriented Approach

Processing complex data is not a problem anymore when you can work with types and utilize one of the most widely used programming paradigms. One can use tons of predefined types from the .NET platform or create their own.

Everything is heavily documented:

.NET API

PowerShell Documentation

Here's a quick example. We have created a custom class User that has a Name, an Age, and a list of Interests. Now we can create objects of this type.

PS /home/deathroll> class User {                                     
>>     [String] $Name                                  
>>     [Int] $Age                                                  
>>     [System.Collections.Generic.List[String]] $Interests
>> 
>>     User($Name, $Age, $Interests) { $this.Name = $Name; $this.Age = $Age; $this.Interests = $Interests }
>> }
PS /home/deathroll> [User[]] $ExampleUsers = @(                      
>>     [User]::new("Jonh Doe", 47, @("Who knows O_o")),
>>     [User]::new("Jake Smith", 22, @("Programming", "Unix-like"))
>> )
PS /home/deathroll> $($ExampleUsers[1]).Interests.Add("Writing Docs")
PS /home/deathroll> $ExampleUsers                                    

Name       Age Interests
----       --- ---------
Jonh Doe    47 {Who knows O_o}
Jake Smith  22 {Programming, Unix-like, Writing Docs}

PS /home/deathroll> $ExampleUsers[0].GetType()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     False    User                                     System.Object

PS /home/deathroll>

Do you feel how empowering is this?

Standard Cmdlet Names

Cmdlets is the PowerShell's way of referring to what we know as "commands."

There's a document for PowerShell that defines how cmdlets should be named. According to it, each cmdlet consists of a verb and a noun. This makes the names easy to read, understand, and memorize. All of the predefined commands adhere to such a convention.

Is it obvious what this command does?

PS /home/deathroll> ConvertTo-Json -InputObject $ExampleUsers

Output

[
  {
    "Name": "Jonh Doe",
    "Age": 47,
    "Interests": [
      "Who knows O_o"
    ]
  },
  {
    "Name": "Jake Smith",
    "Age": 22,
    "Interests": [
      "Programming",
      "Unix-like",
      "Writing Docs"
    ]
  }
]

This is just a recommendation, and people don't have to follow it, but it's better to do so to be consistent.

Approved verbs

Script and Function Parameters

Please note that the script below may not be secure or heavily polished. Also, it's not a good idea to generate passwords in such a way, but that was part of an assignment to simplify things a bit.

Remember I mentioned a script in the Preamble of this article?

Here it is:

Have you already noticed the Param() statement? That's how you define parameters for a script or a function. No more $1, $2 and case...esac, my friend.

Just look at this beauty:

PS /home/deathroll> function Greet-User {
>>     param([Switch] $Informal, [String] $UserName)
>> 
>>     Write-Host $($(If ($Informal) { "Hey" } Else { "Hello" } ) + ", $UserName")
>> } 
PS /home/deathroll> Greet-User $env:USER      
Hello, deathroll
PS /home/deathroll> Greet-User $env:USER -Informal
Hey, deathroll
PS /home/deathroll>

There are multiple types of parameters that can be declared:

dynamic (available only in certain circumstances)
static
switch (work like on and off flags)

They can have attributes like Mandatory (required) and so on.

And a cherry on top of this is that PowerShell reads parameters defined in a script and can perform their names auto-completion when you type them on the command line.

This is an example for PowerShell >=7.0. It contains the ternary operator.

PS /home/deathroll> Get-Content ./Greet.ps1 # Read the contents of the file, like the `cat` command.           
Param([Switch] $Informal, [String] $UserName)

Write-Host $($($Informal ? "Hey" : "Hello") + ", $UserName")
PS /home/deathroll> ./Greet.ps1 - # Here the TAB key was smashed :D
Informal  UserName  
PS /home/deathroll> ./Greet.ps1 -Informal $env:USER
Hey, deathroll
PS /home/deathroll>

Full Integration With the Underlying Platform (.NET)

While everything aforementioned are killer features, this is a killer-killer feature.

Normally we complement shell scripts with Python, Ruby, JavaScript, or whatever scripting languages if the logic needs to be more complex. But in theory, we could just use PowerShell for all of this and tackle specific tasks by calling the .NET API where needed. Maybe even extend the shell itself using other .NET languages like C#.

Conclusion

PowerShell borrows some features from Unix shells: reverse and forward history search, pipes, etc., and adds tons of useful stuff. Why don't we borrow features from it or accept the technology itself where it can make our lives easier? Think about it.

Meanwhile, I'll discuss with our team if we should use PowerShell to automate things on Linux.

Mastering Bash 0 — Internal Field Separators

Timofey Chuchkanov — Sun, 23 Apr 2023 09:01:09 +0000

Preamble
A Bit of Theory
- Shell Expansions
- Builtins
- Fields
A Word on Internal Field Separators
Time for Tuning | Use IFS to Your Advantage

*upd 12/26/2024: Dev Community formatting has gone nuts.
*upd 08/06/2025: there's no point in rewriting the manual

Preamble

Are you a developer, sysadmin, or DevOps engineer? Then you probably spend lots of time on your terminal. I bet you use the GNU Bash shell or something highly similar — take Zsh, for example.

It's essential to know your tools well to work more efficiently. As a Linux geek, I use Bash to automate my daily routine even when it's not for my job, but tired of constantly searching the Web to learn how to achieve certain things in Bash. So, recently I've read through the whole manual. It's relatively small, a little more than 190 pages in total.

~~This article is going to start a series on mastering GNU Bash.~~

A Bit of Theory

Throwing a bunch of commands at the reader without somewhat detailed explanations is as helpful as teaching a five-year-old how to swim by throwing them in the sea. That's why tutorials should start at least with some general knowledge IMHO.

Shell Expansions

Bash is very flexible when it comes to writing commands. You can always do something a lot simpler and shorter than you would without different types of expansions Bash performs on the commands it receives from a user.

In essence, shell expansions enable users to conveniently manipulate data on the command line, such as regular text or variables, using special syntactic constructs. Think of them as shortcuts.

Currently, there are seven types of shell expansions supported in Bash:

brace expansion
tilde expansion
parameter and variable expansion
command substitution
arithmetic expansion
word splitting
filename expansion

We won't cover all of them since that would take too much time. Besides, there's no point in duplicating the manual.

Here's a short example of a parameter expansion:

[deathroll@fedora ~]$ EXAMPLE_TEXT='sOmE tExT Owo'; echo ${EXAMPLE_TEXT,,}

Parameter expansion — alongside command substitution and arithmetic expansion — is denoted by the "$" symbol.

The command above produces the text below because the shell performs a parameter expansion for modifying the case of alphabetic characters to lowercase on the given variable.

some text owo

You could also provide a pattern for operating only on specific characters. One character at a time is matched, though, so no whole words and character sequences can be used.
[deathroll@fedora ~]$ EXAMPLE_TEXT='sOmE tExT Owo'; echo ${EXAMPLE_TEXT,,O}
somE tExT owo
[deathroll@fedora ~]$ echo ${EXAMPLE_TEXT,,[OT]}
somE tExt owo
[deathroll@fedora ~]$

Builtins

You may already know this, but not all commands you type on the command line are actual programs found somewhere on a filesystem. Bash has built-in commands called "builtins." And you use them all the time. Probably the most commonly used are echo and cd.

My personal favorite is the declare builtin. It enables you to declare a variable of a specific type (e.g., array or integer).

Fields

Since the topic is about internal field separators, it's also a good idea to touch a bit on what are fields.

Here's how the manual defines field:

A unit of text that is the result of one of the shell expansions. After expansion, when executing a command, the resulting fields are used as the command name and arguments.

Pretty self-explanatory.

A Word on Internal Field Separators

Internal field separators tell Bash how to split a field into words — these are just sequences of characters treated as single units.

The separators are listed in the IFS variable that contains three symbols by default: space, tab, newline.

— But what does it mean for me as a user?
— Things will get clear with a couple of simple examples.

Suppose you want to get a list of directories and perform some actions on their names. If you search for directories with the find command, the output will contain multiple lines (i.e., separated by newline characters).

[deathroll@fedora ~]$ find . -mindepth 1 -maxdepth 1 -type d -not -name '.*'
./Pictures
./bin
./Videos
./Public
./Music
./Templates
./git
./nvim
./Downloads
./projects
./NAS
./Desktop
./Documents
./snap
./learn
[deathroll@fedora ~]$

When you substitute the command above so that its output becomes the value of some variable or a part of another command, the shell splits this field into words, dividing where it finds the newline character — or any other character listed in the IFS variable.

[deathroll@fedora ~]$ declare -a EXAMPLE_ARR=(`find . -mindepth 1 -maxdepth 1 -type d -not -name '.*'`)
[deathroll@fedora ~]$ echo ${EXAMPLE_ARR[@]}
./Pictures ./bin ./Videos ./Public ./Music ./Templates ./git ./nvim ./Downloads ./projects ./NAS ./Desktop ./Documents ./snap ./learn
[deathroll@fedora ~]$ echo ${EXAMPLE_ARR[0]}
./Pictures
[deathroll@fedora ~]$

OK, that seems to be pretty reasonable. Now I want you to see the commands below and think about the data stored in a variable. What do the array elements look like? More specifically, the first element. Does it look like "2023-02-26 22-10-40.mp4?"

[deathroll@fedora Videos]$ ls *' '*.mp4
'2023-02-26 22-10-40.mp4'  '2023-03-14 00-22-19.mp4'  '2023-03-31 21-12-39.mp4'  '2023-04-08 13-45-36.mp4'
'2023-02-26 22-16-21.mp4'  '2023-03-24 11-54-46.mp4'  '2023-04-04 10-05-41.mp4'  '2023-04-11 11-53-24.mp4'
'2023-03-12 18-59-36.mp4'  '2023-03-24 11-54-57.mp4'  '2023-04-04 11-37-05.mp4'  '2023-04-11 12-00-41.mp4'
'2023-03-14 00-10-02.mp4'  '2023-03-24 11-55-14.mp4'  '2023-04-04 12-06-14.mp4'  '2023-04-11 12-00-51.mp4'
'2023-03-14 00-11-13.mp4'  '2023-03-30 16-16-21.mp4'  '2023-04-04 12-19-03.mp4'  '2023-04-11 13-55-54.mp4'
'2023-03-14 00-17-25.mp4'  '2023-03-31 21-07-22.mp4'  '2023-04-04 12-19-23.mp4'  '2023-04-20 19-41-44.mp4'
[deathroll@fedora Videos]$ EXAMPLE_ARR=(`ls *' '*.mp4`)
[deathroll@fedora Videos]$

The moment of truth... 😰
As you may have already noticed, the filenames contain the space character. Now, recall what is written about IFS at the start of this section of the article.

Let's list the array elements, each on a separate line, to make the output more readable, and pick just the first ten lines to make it short...

[deathroll@fedora Videos]$ head -10 <(for F in ${EXAMPLE_ARR[@]}; do printf '%b\n' $F; done)
2023-02-26
22-10-40.mp4
2023-02-26
22-16-21.mp4
2023-03-12
18-59-36.mp4
2023-03-14
00-10-02.mp4
2023-03-14
00-11-13.mp4
[deathroll@fedora Videos]$ # Oh no! My filenames are broken! 😱
[deathroll@fedora Videos]$

Bash splits filenames into words where it finds any of the characters listed in the IFS variable. Sometimes that's crucial, and you want to change the shell's behavior to split only at the characters you specify.

Congrats if your guess was right! 🎉

Time for Tuning | Use IFS to Your Advantage

Since IFS is a variable, it can be altered by a user. All you need to do is to assign a new value to it. But make sure to use appropriate quoting when you intend to use escape sequences — double quotes ("") or ANSI-C Quoting ($'').

Changing the IFS value for the rest of the shell process execution is not recommended since it can cause you more trouble than bring benefits.

What I prefer is either change the IFS value, execute the commands where custom field separators are needed, and unset IFS, or execute the commands — including IFS variable assignment — in a subshell to avoid mutating the current shell environment.

Don't worry. The shell can't be broken by unsetting the IFS variable. When IFS is unset, Bash will use the default value identical to the variable's initial value.

Example 1 — Unsetting

[deathroll@fedora Videos]$ IFS=$'\n'
[deathroll@fedora Videos]$ head -10 <(for F in `ls *' '*.mp4`; do printf '%b\n' $F; done)
2023-02-26 22-10-40.mp4
2023-02-26 22-16-21.mp4
2023-03-12 18-59-36.mp4
2023-03-14 00-10-02.mp4
2023-03-14 00-11-13.mp4
2023-03-14 00-17-25.mp4
2023-03-14 00-22-19.mp4
2023-03-24 11-54-46.mp4
2023-03-24 11-54-57.mp4
2023-03-24 11-55-14.mp4
[deathroll@fedora Videos]$ unset IFS
[deathroll@fedora Videos]$

Example 2 — Subshell

[deathroll@fedora Videos]$ (IFS="\n"; head -10 <(for F in `ls *' '*.mp4`; do printf '%b\n' $F; done))
2023-02-26 22-10-40.mp4
2023-02-26 22-16-21.mp4
2023-03-12 18-59-36.mp4
2023-03-14 00-10-02.mp4
2023-03-14 00-11-13.mp4
2023-03-14 00-17-25.mp4
2023-03-14 00-22-19.mp4
2023-03-24 11-54-46.mp4
2023-03-24 11-54-57.mp4
2023-03-24 11-55-14.mp4
[deathroll@fedora Videos]$ # Since the command was executed in a subshell, the current environment is left untouched.
[deathroll@fedora Videos]$ head -10 <(for F in `ls *' '*.mp4`; do printf '%b\n' $F; done)
2023-02-26
22-10-40.mp4
2023-02-26
22-16-21.mp4
2023-03-12
18-59-36.mp4
2023-03-14
00-10-02.mp4
2023-03-14
00-11-13.mp4
[deathroll@fedora Videos]$

Now that you hopefully know a bit more about Bash, it's time to experiment yourself! Good luck and have fun, see ya in the next article 🤟!

Set up a private ClamAV database mirror with a Tor-based local SOCKS5 proxy

Timofey Chuchkanov — Thu, 09 Feb 2023 11:11:31 +0000

WARNING! NONE OF THE PERFORMED CONFIGURATIONS CONSIDER SPECIFIC PROJECTS' SECURITY REQUIREMENTS. USE THIS ARTICLE AS A SECONDARY

SOURCE OF KNOWLEDGE ONLY AND HARDEN YOUR SYSTEMS AS APPROPRIATE.

*upd 12/26/2024: Dev Community formatting has gone nuts.

Preface
Why set up a private mirror?
Setting up a server
- What tools are used
- Creating an LXC container on Proxmox
- A local Tor proxy (SOCKS5) with obfs4 bridges
- ClamAV with automatic database updates
- Web server with HTTPS
- Enabling firewall
Configuring clients
The result

Preface

If you are an active computer user, you probably often get a lot of new files from the outside world. The environment is harsh, and it's quite easy to infect a device with malware. To reduce the chances of being infected, it's a good practice to scan everything with special software. For small files that don't contain any confidential info, services like VirusTotal can be utilized. For other data, a standalone or a Live CD-based antivirus application should be preferred.

One of the most popular Free and Open Source applications of this kind is ClamAV by Cisco Talos. The tool is the de facto standard antivirus for UNIX-like systems (both on servers and clients).

To be able to find threats, antivirus software needs a special database of virus signatures. These signatures are continuously created by large teams of security specialists.

Why set up a private mirror?

Well, there might be lots of reasons. The most significant one coming straight to mind is to reduce network traffic and to increase the database update speed for clients.

The decision to host a private virus signature database mirror for ClamAV at my Homelab emerged because I use the tool a lot, and it's not available in my region.

I encourage you to try out this small project even for your Homelab, because it is really easy!

Setting up a server

What tools are used

Since the aforementioned Homelab is based on the Proxmox VE hypervisor, everything is set up there, in a container.

In this scenario, freshclam is used for fetching database files to a mirror server. It's not possible to serve CDIFF files using this method. Consider cvdupdate if you need to reduce network bandwidth consumption even more during clients' database updates.

What	Tool
Hypervisor	Proxmox VE 7.3-4
Guest	Fedora Linux 37 (Container Image)
Antivirus (mirror)	ClamAV 0.103.7
Anonymous communication	Tor 0.4.7.13

Creating an LXC container on Proxmox

Select your storage pool for VM disks and container templates, go to CT Templates and click on the Templates button.
Find the latest Fedora version in the list and download it.
Click on the Create CT button at the top of the Proxmox dashboard and follow the wizard. It's ok to designate 1 CPU core to the container, but it needs sufficient RAM to update databases. Set the limit to something around 4096 MiB.
Don't forget to set an option to start the container at boot.
Start up the container and connect to its console.

A local Tor proxy (SOCKS5) with obfs4 bridges

In countries where Tor is censored, bridges can be used.

Some operating systems' repositories don't provide pre-built packages for obfs4, but the tool can be easily compiled from source code. If you are interested in a quick guide on how to compile obfs4, let me know!

Install Tor

Thankfully, Fedora has everything we need in its repositories <3.

# dnf install tor obfs4

Configure Tor

Locate the obfs4proxy binary

$ which obfs4proxy

Example

[deathroll@ClamAV-DB ~]$ which obfs4proxy
/usr/bin/obfs4proxy
[deathroll@ClamAV-DB ~]$

Open the /etc/tor/torrc config file with an editor of choice. Mine is vim.
```
# vim /etc/tor/torrc
```
Next, add the following two lines:
```
ClientTransportPlugin obfs4 exec /usr/bin/obfs4proxy
UseBridges 1
```
👆 Replace the path after exec with what you got in step 1.
Now, get some bridges!
Go to https://bridges.torproject.org/options/, select obfs4, and click on Get Bridges.

In order to get a list of bridges, complete a captcha. When the list is displayed, just copy it as-is.

Paste what you've just copied to the torrc file and prefix each line with Bridge. Here's how it will look in the file (the output was modified due to privacy concerns):

[deathroll@ClamAV-DB ~]$ grep -i ^bridge /etc/tor/torrc 
Bridge obfs4 <some.ip.address.here:port> <ALONGSTRINGOFSOMEJIBBERISH> cert=<ONEMORELONGSTRINGOFSOMEJIBBERISH> iat-mode=<number>
Bridge obfs4 <some.ip.address.here:port> <ALONGSTRINGOFSOMEJIBBERISH> cert=<ONEMORELONGSTRINGOFSOMEJIBBERISH> iat-mode=<number>
[deathroll@ClamAV-DB ~]$

After saving and closing the config file, perform a test connection by executing tor as toranon.
```
$ sudo -u toranon tor
```
In case of success, kill the process with Ctrl + C, and enable and start the service.
```
# systemctl enable --now tor
```
Check the status with the following command:
```
# systemctl status tor
```

ClamAV with automatic database updates

Install ClamAV

# dnf install clamav clamd clamav-update

Configure freshclam

Open the /etc/freshclam.conf file and add the following:

HTTPProxyServer socks5://127.0.0.1
HTTPProxyPort 9050

Enable database compression (optional).
```
CompressLocalDatabase yes
```
Also, you can enable logging (optional). For more, read freshclam.conf(5).
1. Add this to the config:
```
UpdateLogFile /var/log/freshclam.log
LogTime yes
LogSyslog yes
```
2. Create the log file.
```
# touch /var/log/freshclam.log
```
3. And change its owner to clamupdate.
```
# chown clamupdate: /var/log/freshclam.log
```

After that, locate the clamav-freshclam service file.

$ find /etc/systemd/ -name '*clam*'

Example

[root@ClamAV-DB deathroll]# find /etc/systemd/ -name '*clam*'
/etc/systemd/system/multi-user.target.wants/clamav-freshclam.service
[root@ClamAV-DB deathroll]#

Open the file you've just found and append tor.service to the lines that begin with Wants and After so they look like this:
```
Wants=network-online.target tor.service
After=network-online.target tor.service
```
Reload the systemd manager configuration.
```
# systemctl daemon-reload
```
Enable and start the clamav-freshclam service.
```
# systemctl enable --now clamav-freshclam
```
Check the status with the following command:
```
# systemctl status clamav-freshclam
```

Web server with HTTPS

In this example, we use NGINX, but you're free to experiment.

Prepare a directory for serving

Create a new directory. For me it is /srv/cvd.
```
# mkdir -p /srv/cvd
```

Find where the database is located.

# find / -name clamav -xtype d |& grep -iv permission

👆 -xtype d makes find search for directories only. |& pipes both stdout and stderr to grep. Here we are just filtering the output to remove read errors—don't worry, they are insignificant.

In my case, the dir is /var/lib/clamav.

Example

[deathroll@ClamAV-DB nginx]$ sudo find / -name clamav -xtype d |& grep -iv permission
/usr/share/licenses/clamav
/var/lib/clamav
[deathroll@ClamAV-DB nginx]$

Without grep filtering

[deathroll@ClamAV-DB nginx]$ sudo find / -name clamav -xtype d
find: ‘/proc/tty/driver’: Permission denied
/usr/share/licenses/clamav
/var/lib/clamav
find: ‘/dev/.lxc/sys/kernel’: Permission denied
find: ‘/dev/.lxc/sys/power’: Permission denied
find: ‘/dev/.lxc/sys/class’: Permission denied
find: ‘/dev/.lxc/sys/devices’: Permission denied
find: ‘/dev/.lxc/sys/dev’: Permission denied
---OUTPUT TRIMMED---

Edit /etc/fstab to set a read-only bind mount from the database dir to the newly created one.
```
/var/lib/clamav/    /srv/cvd    none    bind,ro    0   0
```
Reload the systemd manager configuration.
```
# systemctl daemon-reload
```
Try to mount /srv/cvd.
```
# mount /srv/cvd
```

Check the mountpoint.

$ findmnt /srv/cvd

Example

[deathroll@ClamAV-DB nginx]$ findmnt /srv/cvd
TARGET   SOURCE                                                 FSTYPE OPTIONS
/srv/cvd NAS/Virtualization/subvol-1000-disk-0[/var/lib/clamav] zfs    ro,noatime,xattr,posixacl
[deathroll@ClamAV-DB nginx]$

Install NGINX

# dnf install nginx

Copy certificates

In order to be able to use HTTPS, it's required to have valid certificates. Refer to this article to learn how to create a trusted self-signed certificate.

Create a directory for certificates in a preferred location.
```
# mkdir /etc/ssl/www
```
Copy a chain of trust file and a key file to the newly created directory.

Configure NGINX

In this example, it's assumed that a DNS A record exists for the container's static IP address. The record name is cvd.deathroll.internal. It's also possible to implement HTTPS without DNS, though not recommended.

The configuration is split up into multiple files for ease of administration, but this is not required.

Create a file with shared SSL/TLS configuration for server blocks. Choose a filename suitable for you.
```
# vim /etc/nginx/ssl_params.conf
```
1. Add at least two things: locations for the chain of trust and key files.
```
ssl_certificate     "/etc/ssl/www/fullchain.pem";
ssl_certificate_key "/etc/ssl/www/deathroll-internal-key.pem";
```
2. Change any other options at your discretion. For example:
```
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1.2 TLSv1.3;
```

Change the default server block in the /etc/nginx/nginx.conf file to the following:

server {
    listen      80;
    server_name _; # A catch-all server name for handling requests to any host.

    location / {
        return 301 https://$host$request_uri; # Redirect to HTTPS.
    }
}

Create a file for a <domain> HTTPS server block in /etc/nginx/conf.d.

vim /etc/nginx/conf.d/cvd.deathroll.internal.conf

And populate it with contents similar to these:

server {
    listen      443 ssl;
    server_name cvd.deathroll.internal;

    root        /srv/cvd; # This is the bind mount directory you have created earlier.

    include     /etc/nginx/ssl_params.conf; # Include shared TLS/SSL parameters.

    location / {
        autoindex on; # Enable directory listing.
    }
}

Create a file with a server block for denying requests to any host. For example, /etc/nginx/conf.d/deny-conn-by-ip.conf. Add something like this:
```
server {
    listen      443 default_server; # Make this server default for port 443 to be able to catch unwanted requests.
    server_name _;

    include     /etc/nginx/ssl_params.conf;

    location / {
        return 403;
    }
}
```
👆 From what I could infer from inspecting packets, observing NGINX behavior, and reading its documentation, when NGINX receives a request, it checks the Host HTTP header to decide what virtual host should process the request.

The catch-all (_) server for port 443 has a default_server parameter, so HTTPS requests to any Host will be processed by this virtual host. If this parameter is omitted, any other virtual server for this port may become the default server, and hence requests won't be denied.
Reload NGINX config.
```
# systemctl reload nginx
```
Check NGINX status.
```
# systemctl status nginx
```

Enabling firewall

In this example, firewalld is used.

Install firewalld

# dnf install firewalld

Configure firewalld

Enable and start the service
```
# systemctl enable --now firewalld
```

Add ports. If you use SSH, add it as well.

# firewall-cmd --permanent --add-port={80,443}/tcp

Reload firewalld
```
# firewall-cmd --complete-reload
```

Configuring clients

Disable database compression (optional) and add a private mirror in the freshclam.conf file, so these options look something like this:

deathroll@fdesk:~ % egrep -i '^(private|compress)' /usr/local/etc/freshclam.conf
CompressLocalDatabase no
PrivateMirror https://cvd.deathroll.internal
deathroll@fdesk:~ %

The result

After the setup is complete, what we have is a Fedora 37 container with a local Tor-based SOCKS5 proxy that is used for updating the ClamAV database. On top of this, we have an NGINX web server that acts as a private mirror that allows clients to list and download database files for ClamAV. This web server redirects all requests to HTTPS and denies requests to any host different from the server's FQDN.

Deny a request to an IP address

Directory listing

Update the ClamAV database on a client

Trusted self-signed TLS certificates for dummies (w/ thorough explanations included)

Timofey Chuchkanov — Fri, 25 Nov 2022 12:57:30 +0000

*upd 12/26/2024: Dev Community formatting has gone nuts.

upd 1/20/2023: Added FreeBSD steps, CA certificate SAN, and fixed shell commands formatting.

upd 1/23/2023: Added Android 13 steps.

upd 1/25/2023: Added openssl v1.1.1 fix for the step 5.2 in 🪄 How to generate a self-signed cert. Removed FreeBSD instructions for installing openssl v3.

This is a translation of my article, initially written in a different language. Please, let me know about language-related mistakes if you notice any.

Also, I'm a student and not a professional with a rich knowledge base, though that's my goal. Let me know if I misunderstood something and consequently shared misinformation here.

WARNING! NONE OF THE PERFORMED CONFIGURATIONS CONSIDER SPECIFIC PROJECTS' SECURITY REQUIREMENTS. USE THIS ARTICLE AS A SECONDARY SOURCE OF KNOWLEDGE ONLY AND HARDEN YOUR SYSTEMS AS APPROPRIATE.

🧑‍🎓 A bit of theory
- 🏛️ Certificate authority
- 🔏 Digital certificate
- ✒️ Digital signature
🧐 When you might need this
🔍 How HTTPS works (abstract)
📜 What you will need to generate a self-signed cert
🪄 How to generate a self-signed cert
🔫 How to force devices to trust our very own CA
- Windows
- Linux
- FreeBSD
- Android
- iOS
- Some additional configuration required by Firefox
🖥️ The result
📖 Additional sources of useful information

This article is my slight extension to the How to create a valid self signed SSL Certificate? video by Christian Lempa—I love this channel <3—that covers some topics more in-depth. The video turned out to be pretty helpful for me during the integration of HTTPS into my HomeLab services. Now I have trusted TLS certs yay!

It's barely feasible to do anything without understanding what exactly is happening under the hood for me. I always try to dig deeper to get a grasp of truth—how things work. That's why I wrote this article. The goals are: firstly, to learn more about HTTPS and secondly, to help people that are just like me.

🧑‍🎓 A bit of theory

🏛️ Certificate Authority

Certificate Authority (CA) is an entity/party responsible for generating, storing, and issuing digital certificates. There are many CAs: governmental, commercial, non-profit, and private.

Almost every device stores trusted Certificate Authorities' public keys and digital certificates: a smartphone, a PC, or a single-board computer for IoT. If a device supports connecting to a network by design, its operating system has these artifacts. This approach allows us to verify network resources' identities and safely connect to them.

🔏 Digital certificate

It is an electronic document used to validate the identity of a public key. It contains info about the key itself, the subject (owner) of this key, and a digital signature of an entity/party that approved the certificate.

✒️ Digital signature

An electronic signature type. It's a mathematical algorithm used for validating data authenticity and integrity. A digital signature creates a unique fingerprint for an entity.

Abstractly, digital signature creation and usage process may be performed like so:

Party A has a pair of keys—a public and a private one. This party wants to transfer a document DOC to party B. It also wants B to be able to validate the DOC and check its integrity.

To achieve this, A applies a hash function to DOC to get a digest—an alphanumeric value unique to this document. A digest may be of any fixed size depending on the hash algorithm. When the digest is calculated, the party encrypts it using its private key. That's how a digital signature SIG is created.

When B receives DOC from A, it also gets SIG and A's public key PUB. Just like A, B applies a hash function to DOC. But after that, it uses A's public key PUB to decrypt the digest. The next step is to compare the calculated digest to the decrypted one.

If digests are equal, the document wasn't altered during transmission.

🧐 When you might need this

Before we start doing anything, we always ask ourselves: but why do I need to do this?

There are many situations when a self-signed certificate might be needed/preferred. For example, it's common for an enterprise network to have a couple or a hundred internal services. I intentionally highlighted the word "internal" since using self-signed certs for public services is a security-violating habit. By doing so, you bring danger to your users, at least. As another example, such certificates may be used for your HomeLab services—just like in my case—or other cases when a non-public domain is utilized.

🔍 How HTTPS works (abstract)

A PKI or Public Key Infrastructure is the foundation of HTTPS.

Sure, we could encrypt all network traffic with asymmetric algorithms. But it would be too slow and ineffective. Both time and computing resources spent on encryption and decryption increase as the amount of data grows, as you might already know. Symmetric encryption is used to avoid this, and both parties have session keys for it.

But here's another problem: how to transfer session keys in such a way that they could not be accessed by a threat actor? 🤔

— Asymmetric encryption! 🎉

To provide HTTPS traffic encryption, the client and the server must perform a "handshake." During the TLS handshake, parties agree to use a highest TLS version supported by both parties and cipher suites supported alike. A couple more things happen at this stage. The client verifies the identity of the server by a digital certificate belonging to the latter one, and in case of success, session keys are generated. These session keys allow to encrypt traffic symmetrically.

Both participants of the connection have key pairs consisting of public and private ones. To generate session keys, participants share some data that is encrypted using public keys.

Data sent after the end of the handshake is symmetrically encrypted with the session keys.

📜 What you will need to generate a self-signed cert

To create a certificate, we need several other things first:

A pair of keys for the Certificate Authority
A self-signed CA certificate
A pair of keys for the target resource
A Certificate Signing Request (CSR) for the resource

CSR is a message for the CA, signed by an entity's private key. It contains info such as an email, an organization's full name, a country code, a city, a key type and its length, etc. An entity sends this message to a CA with the intent to have it signed by the CA's private key.

🪄 How to generate a self-signed cert

In the following example, we use a 4096-bit long RSA key and an SHA-256 hashing algorithm, but it's just a personal preference. Aside from that, the whole process is explicitly separated into several steps, though some things can be achieved quicker, involving lesser commands.

And remember that you can choose filenames different from mine. Just make sure they are consistent. :)

There's a high number of ways you can generate certs. In this scenario, an openssl utility is demonstrated (version 3 and higher).

1. Certificate Authority key pair

👇 For the sake of security, the key will be encrypted using an AES-256 algorithm. That's why you'll need to come up with a password.

$ openssl genrsa -aes256 -out CA-key.pem 4096

2. Self-signed CA certificate

It's possible to create a certificate valid for one day to several years (👇). More commonly, they're valid for several months.

A certificate that is expired in more than 13 months will be considered invalid by a huge number of modern devices. It was the case with my iOS-powered phone.

It's better to add a subjectAltName extension since the Common Name (CN) is deprecated and the Subject Alternative Name (SAN) is a more flexible replacement.

$ openssl req -new -x509 -sha256 -days 365 -key CA-key.pem -out CA.pem -addext 'subjectAltName = DNS:hl-ca.deathroll.internal'

👆 Initially, the openssl req command with a flag -new creates a new CSR, but we also specified a flag -x509 and an option -key. This caused a change in the program's behavior:

-x509 tells the program to generate an X.509 certificate instead
The certificate is signed with a private key you provide via -key, and the corresponding public key is attached to the certificate itself.

The -sha256 flag tells openssl to use the SHA256 algorithm for digest computation. This algorithm is the default, but in case it changes somewhere in the near future, it's better to be explicit.

OpenSSL 3.0.2 15 Mar 2022 supported digests

You can check that the certificate is indeed for a CA

And a command for this is pretty simple:

$ openssl x509 -in CA.pem -text

3. Resource key pair

👇 This key could also be encrypted.

$ openssl genrsa -out deathroll-internal-key.pem 4096

4. CSR for a resource/domain

To simplify my life, I generated a CSR for the whole HomeLab domain at once—*.deathroll.internal. Such a format is called wildcard. It allows one to use a single certificate no matter what subdomain it is. Sounds convenient, right?

$ openssl req -new -sha256 -key deathroll-internal-key.pem -out deathroll-internal.csr

5. Resource certificate

Create a small config file for X.509 extensions and add the subjectAltName extension to it. In our case it's a file called certconf.cnf with the following contents:
```
subjectAltName="DNS:*.deathroll.internal"
```
subjectAlternativeName supported values - x509v3_config (5)

That's how X.509 extensions look like in a certificate

Create the certificate itself
For openssl <3.0.0
Add the -CAcreateserial flag to create a certificate serial number. If omitted, an error will occur.

$ openssl x509 -req -sha256 -days 365 -in deathroll-internal.csr -CA CA.pem -CAkey CA-key.pem -out deathroll-internal.pem -extfile certconf.cnf

6. Certificate Chain (Chain of trust)

It's a list of certificates that begins with a resource certificate and ends with a root CA certificate. There can also be intermediate CAs between them.

Such chains can be found on most web servers—among other places—and are easily generated. To create a chain, copy the contents of a resource cert and then a CA cert into a single file.

The cat command can be applied here. Read the contents and redirect the stdout to a file.

$ cat deathroll-internal.pem > fullchain.pem

$ cat CA.pem >> fullchain.pem

🔫 How to force devices to trust our very own CA

Don't forget to restart any running browsers so they can trust your certificates.

Windows

Open the Microsoft Management Console
For example, Win + R -> mmc.exe.
Select Add/Remove Snap-in in the File menu.
Select Certificates among available snap-ins on the left-hand side of the window and click Add in the middle.
Next, choose for what account you will manage certificates. For me, it's a Computer account, and on the next page—Local computer.
Click OK in the previously opened window.
Expand Certificates in the console root -> Trusted Root Certification Authorities -> Certificates.
Right-click on Certificates -> All tasks -> Import.
On the second page of the opened window, choose a certificate file.

If the file is not listed, change the file type filter to show all files.

Linux

The methods vary vastly depending on a distribution. I use a Debian-based distro.

Check if the ca-certificates package is installed

$ apt list --installed | grep ca-certificates

Install it if it's not present on your system.
```
# apt install ca-certificates
```
Copy the certificate file to /usr/share/ca-certificates and reconfigure the ca-certificates package.
```
# cp CA.pem /usr/share/ca-certificates/CA.crt
```
👆 Note that the certificate in the target directory must have the .crt extension. Otherwise, it won't be recognized.
```
# dpkg-reconfigure ca-certificates
```
👆 In a TUI window choose your certificate by pressing space on your keyboard.

FreeBSD

Create a directory listed in TRUSTPATH (see ENVIRONMENT at certctl(8)). For example, /usr/local/etc/ssl/certs.
```
# mkdir -p /usr/local/etc/ssl/certs
```
Copy the certificate file to the created directory
```
# cp CA.pem /usr/local/etc/ssl/certs/homelab-ca.crt
```
👆 Note that the certificate in the target directory must have the .crt extension. Otherwise, it won't be recognized.
Rebuild SSL certificates list
```
# certctl rehash
```
Verify that the CA certificate was installed (use part of your CN instead of internal)
```
$ certctl list | grep internal
```

Verify the certificate of your service

$ openssl s_client -connect nas.deathroll.internal:443 | head

Android

7.1

Surprisingly, it's stupidly simple to install a custom CA cert. Just download the file and open it with a certificate manager. After that, follow the installation wizard.

13

Try to search in your settings app. It should be somewhere in the privacy section.

iOS

Download the certificate and place it somewhere on your filesystem so it can be located in the Files app later.
Open Files and tap on the certificate file. You will be notified that the profile was downloaded.
Open Settings and tap on the Profile Downloaded item at the top.
Tap on the profile and then Install.
While still in the Settings app, go to General -> About -> Certificate Trust Settings.
Toggle the switch near your cert.

Some additional configuration required by Firefox

If you are a happy Firefox user, one more additional step is required. You need to permit the browser to use CAs installed on your system.

Open about:config in a new tab—just type it in the search bar—and accept the risk.
Type security.enterprise_roots.enabled in the search field on the page and then change the value of this key—it will appear under the input field—from false to true by clicking the ⇌ button on the right-hand side.

🖥️ The result

Congrats! You have created and installed your own Certificate Authority and a resource certificate. Now, if you load a web page or a web app that uses your certificate, a pleasing lock icon will greet you 🔒. Your devices trust your internal services, and annoying messages about insecurity are gone like a fart in a desert. It's worth it, believe me. :)

Internal HTTPS-secured web services opened in various web browsers

📖 Additional sources of useful information

openssl man pages

A helpful one. There are manuals for both config files and openssl subcommands. For example, if you want to learn more about openssl x509, issue man openssl-x509.

DEV Community: Timofey Chuchkanov

Encrypted Remote Terraform State with Postgres and LUKS

Table of Contents

Why?

But Why PostgreSQL?

Prerequisites

Trusted self-signed TLS certificates for dummies (w/ thorough explanations included)

Timofey Chuchkanov ・ Nov 25 '22

What Versions of Software are Used?

Set Up a Postgres Server

Prepare Storage

Create a LUKS Device

Configure crypttab and fstab

crypttab

fstab

Create a Filesystem

Mount and Prepare the Filesystem

Configure Postgres

Install

Disable and Stop the Service

Copy a TLS Certificate and Key

Configure

Settings

Access Control

Start

Configure a Database

Configure a Firewall

Check Connection from Another Host

Configure a Terraform Client

Using Terraform Providers in Isolation

Timofey Chuchkanov ・ Aug 6 '23

What to Do After a Reboot?

The Result

Using Terraform Providers in Isolation

Is This Guide for Me?

Table of Contents

Find the Source Code

Option 1. Go to GitHub and Search for Providers There

Option 2. Search the Registry Beforehand or Ask Someone to Do This for You

Get the Binaries

Place the Binaries Somewhere Locally

Configure the Terraform CLI

Test It Out!

A Linux Geek Tried PowerShell — It Feels *Power*ful Indeed

Table of Contents

Preamble

The Good Stuff

Working With Types

Object-oriented Approach

Standard Cmdlet Names

Script and Function Parameters

Full Integration With the Underlying Platform (.NET)

Conclusion

Mastering Bash 0 — Internal Field Separators

Preamble

A Bit of Theory

Shell Expansions

Builtins

Fields

A Word on Internal Field Separators

Time for Tuning | Use IFS to Your Advantage

Now that you hopefully know a bit more about Bash, it's time to experiment yourself! Good luck and have fun, see ya in the next article 🤟!

Set up a private ClamAV database mirror with a Tor-based local SOCKS5 proxy

WARNING! NONE OF THE PERFORMED CONFIGURATIONS CONSIDER SPECIFIC PROJECTS' SECURITY REQUIREMENTS. USE THIS ARTICLE AS A SECONDARY

Contents

Preface

Why set up a private mirror?

Setting up a server

What tools are used

Creating an LXC container on Proxmox

A local Tor proxy (SOCKS5) with obfs4 bridges

Install Tor

Configure Tor

ClamAV with automatic database updates

Install ClamAV

Configure freshclam

Web server with HTTPS

Prepare a directory for serving

Install NGINX

Copy certificates

A Linux Geek Tried PowerShell — It Feels Powerful Indeed