DEV Community: Berk

Split-Horizon DNS on AdGuard: One Domain, Two Networks, Two IP Addresses

Berk — Sat, 19 Apr 2025 23:47:29 +0000

Split-Horizon DNS with AdGuard Home

Let’s say you self-host an app on your home network. Your server’s local IP is 192.168.1.200, so you set app.example.com to point to that. Everything works fine—until you want to access it from outside your home.

You set up a VPN (like Tailscale, WireGuard, or OpenVPN) so your clients and server can talk securely. The server now also has a VPN IP, like 100.64.50.50. You configure your devices to use your home AdGuard DNS over VPN.

But here’s the problem: when you try to reach app.example.com over VPN, it still resolves to 192.168.1.200—a local IP that doesn’t work outside the LAN. So, you switch the DNS to return 100.64.50.50 instead.

Great—VPN clients can now connect. But now what about your devices at home? Do they also need to connect through VPN all the time? Do you really want encrypted VPN traffic on your local LAN? Should you run two separate DNS servers and sync them?

There’s a simpler way using split-horizon DNS with AdGuard Home.

What’s Split-Horizon DNS?

It’s when your DNS server returns different IP addresses depending on where the request comes from. For example:

If you're on VPN → return VPN IP (100.64.50.50)
If you're on home LAN → return local IP (192.168.1.200) ### How to Set It Up in AdGuard Home

Log in to your AdGuard Home dashboard.
Go to Filters → Custom Filtering Rules.

Add these two rules:

||split.example.com^$dnsrewrite=NOERROR;A;100.64.50.50,client=100.0.0.0/8
||split.example.com^$dnsrewrite=NOERROR;A;192.168.1.200,client=192.168.1.0/24

The first rule responds with the VPN IP if the query comes from the Tailscale subnet (100.0.0.0/8).
The second rule responds with the local IP for devices on your home network (192.168.1.0/24).

📝 Note: These rules are just an example for testing!

Be sure to replace split.example.com with your actual domain name,

100.64.50.50 with your server’s VPN IP,

and 192.168.1.200 with your server’s local IP.

Testing

From home network:

nslookup split.example.com

Server: 192.168.1.235
Address: 192.168.1.235#53

Non-authoritative answer:
Name: split.example.com
Address: 192.168.1.200

From VPN:

nslookup split.example.com

Server: 100.100.100.100
Address: 100.100.100.100#53

Non-authoritative answer:
Name: split.example.com
Address: 100.64.50.50

Result

Now you’ve got one domain, one DNS server, and two responses—depending on where the request comes from. It’s a clean and easy solution for accessing your app from both home and VPN.

Achievements:

One domain name for all environments
No need to run multiple DNS servers
Easy to manage and update from the AdGuard Home dashboard
Works great with VPNs like Tailscale, WireGuard, or OpenVPN
No need to keep VPN enabled all the time
Devices without VPN can still connect on the home network

Fixing Unknown SMART Status for External SSD/HDD on Proxmox

Berk — Sun, 30 Mar 2025 13:52:14 +0000

Overview

If you are using an External SSD/Harddisk and the Proxmox can't detect the SMART status, this post might help you.

In a previous blog, I showed how to use smartctl tool with external SSDs/Harddisks. The problem with in previous blog post was that smartctl was not automatically detecting my external SSD controller and the SMART query command failed. The fix is very easy. You should detect the controller chip and tell smartctl with -d flag. For more details, you can head to the previous blog post.

TDLR:

Problem was default command can't detect the device type:

$ smartctl -i /dev/sdb

/dev/sdb: Unknown USB bridge [0x0bda:0x0bda (0x2001)]
Please specify device type with the -d option.

Fixed with specifying the correct device type:

$ smartctl -i -d sntrealtek /dev/sdb

=== START OF INFORMATION SECTION ===
Model Number:                       KBG50ZNV256G KIOXIA
...

If you can't query your SSD's SMART values with the default command, Proxmox can't too. In this blog, we will wrap the smartctl so Proxmox won't have problem while querying the SMART status of my external SSD.

First, login to the Proxmox host server.
- I am using the root account directly.
- If you are not using root, please add sudo to beginning of the commands below.
- Note that you need sudo privileges for this solution.
Create a bash script somewhere in your server.

vim ~/smartctl.external

Copy and paste the script below to the file.
- What this script basically do is adding -d sntrealtek flag if the queried device is /dev/sdb. It doesn't do anything to another devices or flags.
- ⚠️ Please change the /dev/sdb and -d sntrealtek according to your requirements.

#!/usr/bin/env bash

SMARTCTL=/usr/sbin/smartctl.orig
OPTIONS=("$@")

for((i=1; i<$#; ++i)); do
    # Check for /dev/sdb and add the Realtek option
    if [[ "${OPTIONS[i]}" == "/dev/sdb" ]]; then
        realtek_option="-d sntrealtek"
        # Add the "-d sntrealtek" option to the list of options
        OPTIONS=($realtek_option "${OPTIONS[@]}")
    fi
done

exec $SMARTCTL "${OPTIONS[@]}"

Now rename the original smartctl tool.

cp /usr/sbin/smartctl /usr/sbin/smartctl.orig

Replace the wrapper script with the original smartctl

cp ~/smartctl.external /usr/sbin/smartctl

Set the permissions of the wrapper script.

chmod 755 /usr/sbin/smartctl

(Optional) Change the attribute of the wrapper script so it won't be changed by updates or by mistake.

chattr +i /usr/sbin/smartctl

Now make sure that the default command returns the SMART values:

$ smartctl -i /dev/sdb

=== START OF INFORMATION SECTION ===
Model Number:                       KBG50ZNV256G KIOXIA
...

Return to the Proxmox and click on the reload on the Disks page to reload the SMART values.
The UNKNOWN status should be PASSED now.
You can also view the full SMART details.

Enabling Secure Boot with Linux and Windows Dual-Boot Setup

Berk — Mon, 10 Feb 2025 21:57:27 +0000

Overview

This comprehensive guide demonstrates how to enable Secure Boot on a dual-boot system running both Linux and Windows. While maintaining the ability to boot into both operating systems, this setup ensures UEFI Secure Boot verification for enhanced system security.

Although demonstrated using Arch Linux and a Gigabyte motherboard, these procedures are applicable across most Linux distributions and motherboard manufacturers with minor variations in UEFI interface layouts.

This guide was specifically created to enable Secure Boot for applications requiring stringent security measures, such as FACEIT Anti-Cheat (used in Counter-Strike), while preserving full Linux functionality. Many modern gaming anti-cheat systems and enterprise applications mandate Secure Boot for enhanced security.

Important Prerequisites

Before modifying your UEFI firmware settings, complete these essential preparation steps:

Backup Your UEFI (BIOS) Configuration
- The procedure requires clearing existing and generating new Secure Boot keys
- While most modern motherboards provide options to restore factory Secure Boot keys, this capability varies by manufacturer
- Document all current UEFI settings thoroughly, especially boot sequence and security options
Locate Essential UEFI Settings
- Navigate through your motherboard's UEFI interface to familiarize yourself with its layout
- Identify these critical security settings:
  - Secure Boot activation toggle
  - Secure Boot key management interface (for key deletion and restoration)

Step-by-Step Implementation

1. Disable Secure Boot

Initially, disable Secure Boot in your UEFI settings. This step is necessary because Linux cannot boot with the default Secure Boot keys, and we need to establish our own custom keys.

2. Delete Secure Boot Keys

Navigate through your UEFI settings to prepare for custom key enrollment:

First, switch to "Custom" secure boot mode. This enables granular control over Secure Boot keys.

Next, remove all existing Secure Boot keys to prepare for custom key enrollment. This step places the system in Setup Mode.

Verify the keys have been deleted and boot into Linux. The system should now be in Setup Mode.

3. Install and Configure `sbctl`

Install sbctl, the Secure Boot key management utility. For installation instructions specific to your distribution, consult the official README file.

For Arch Linux users:

sudo pacman -S sbctl

4. Check Status

Verify the current Secure Boot configuration:

sudo sbctl status

Expected output indicating proper setup mode:

Installed:  ✘ Sbctl is not installed
Owner GUID: bbbfcaf8-3102-47f9-a921-2e5245da7e9f
Setup Mode: ✗ Enabled
Secure Boot:    ✗ Disabled
Vendor Keys:    none

5. Generate and Enroll Keys

Generate a new set of custom Secure Boot keys:

sudo sbctl create-keys

Upon successful key creation, you'll see:

Created Owner UUID a9fbbdb7-a05f-48d5-b63a-08c5df45ee70
Creating secure boot keys...✔
Secure boot keys created!

Enroll both your custom keys and Microsoft's keys (required for Windows boot compatibility):

sudo sbctl enroll-keys --microsoft

Successful enrollment confirmation:

Enrolling keys to EFI variables...
With vendor keys from microsoft...✓ 
Enrolled keys to the EFI variables!

6. Configure GRUB Bootloader

Install GRUB with TPM support and security modules enabled:

sudo grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=GRUB --modules="tpm" --disable-shim-lock

7. Sign Boot Files

First, check for unsigned boot files:

sudo sbctl verify

The system will identify unsigned files:

Verifying file database and EFI images in /boot...
✗ /boot/EFI/BOOT/BOOTX64.EFI is not signed
✗ /boot/EFI/GRUB/grubx64.efi is not signed
✗ /boot/grub/x86_64-efi/core.efi is not signed
✗ /boot/grub/x86_64-efi/grub.efi is not signed
✗ /boot/vmlinuz-linux is not signed
✗ /boot/vmlinuz-linux-lts is not signed

Sign all boot files with your custom keys:

sudo sbctl sign-all

Successful signing confirmation:

✓ Signed /boot/EFI/BOOT/BOOTX64.EFI
✓ Signed /boot/EFI/GRUB/grubx64.efi
✓ Signed /boot/grub/x86_64-efi/core.efi
✓ Signed /boot/grub/x86_64-efi/grub.efi
✓ Signed /boot/vmlinuz-linux
✓ Signed /boot/vmlinuz-linux-lts

Verify all files are now properly signed:

sudo sbctl verify

Expected verification output:

Verifying file database and EFI images in /boot...
✓ /boot/EFI/BOOT/BOOTX64.EFI is signed
✓ /boot/EFI/GRUB/grubx64.efi is signed
✓ /boot/grub/x86_64-efi/core.efi is signed
✓ /boot/grub/x86_64-efi/grub.efi is signed
✓ /boot/vmlinuz-linux is signed
✓ /boot/vmlinuz-linux-lts is signed

8. Enable Secure Boot

Return to UEFI settings to activate Secure Boot with your new keys:

Reboot and enter UEFI settings
Navigate to Secure Boot configuration
Enable Secure Boot
Save changes and exit

9. Verify Configuration

After booting into Linux, confirm Secure Boot is properly enabled:

sudo sbctl status

Expected configuration status:

Installed:  ✓ sbctl is installed
Owner GUID: bbbfcaf8-3102-47f9-a921-2e5245da7e9f
Setup Mode: ✓ Disabled
Secure Boot:    ✓ Enabled
Vendor Keys:    microsoft

Troubleshooting Guide

Common Issues and Solutions

Boot Failures
- If boot fails, temporarily disable Secure Boot through UEFI settings
- Boot into Linux and verify all boot files are correctly signed
- If problems persist, consider regenerating and re-enrolling keys
Windows Boot Problems
- Verify Microsoft keys were properly enrolled using the --microsoft flag
- Ensure Windows Boot Manager maintains proper signatures
Key Management Issues
- For corrupted keys, use UEFI options to restore factory Secure Boot keys
- Repeat the key generation and enrollment process from the beginning

Maintenance Notes

Sign new kernel images after system updates: sudo sbctl sign-all
Monitor Secure Boot status regularly, especially following system updates
Store custom key backups securely in a separate location

Security Considerations

Custom Secure Boot keys enhance security only when properly managed
Implement a strong UEFI administrator password
Consider enabling TPM for additional hardware-based security features

How to Add Domains and IP Addresses to Docker Containers

Berk — Sat, 19 Oct 2024 14:52:57 +0000

Overview

Docker users often face challenges with custom domain and IP address mappings in their containers. The inability of containers to read the host machine's /etc/hosts file creates significant obstacles.

In this practical guide, we'll tackle this common Docker networking challenge: how to make containers resolve custom domains and IP addresses without modifying each container individually.

We'll cover two methods: integrating dnsmasq with NetworkManager and configuring dnsmasq without NetworkManager. By the end of this post, you'll be able to easily manage custom domain resolutions in your containers.

By the end of this post, you'll be able to:

Easily add custom domain resolutions to all your Docker containers
Streamline your development and testing workflows
Avoid the headache of managing /etc/hosts files for each container

Install Dnsmasq
Configure Dnsmasq
1. Dnsmasq Standalone Usage
2. Dnsmasq with NetworkManager
Configure Docker
Add Test Domain Names to /etc/hosts
Test the Custom Domain Names

Install Dnsmasq

Make sure dnsmasq is installed.

which dnsmasq

If not installed, install it

# For Debian / Ubuntu
sudo apt install dnsmasq

# For RHEL / CentOS / Fedora
sudo dnf install dnsmasq

Configure Dnsmasq

Dnsmasq Standalone Usage

In this section, we'll configure dnsmasq if you are not using NetworkManager.

Create a new configuration file for dnsmasq:

sudo vim /etc/dnsmasq.conf

Add the following configuration options:

# Listen on the Docker bridge interface
interface=docker0

# Specify a default DNS server to forward queries
server=8.8.8.8  # Example: Google DNS

Start the dnsmasq service and enable it to run on boot:

sudo systemctl stop dnsmasq && \
sudo systemctl enable --now dnsmasq && \
sudo systemctl status dnsmasq

Dnsmasq with NetworkManager

If you are using NetworkManager, you can use dnsmasq as its plugin.

Open NetworkManager main configuration file with an text editor:

sudo vim /etc/NetworkManager/NetworkManager.conf

Find the [main] section and add dns=dnsmasq

[main]
dns=dnsmasq

Create dnsmasq configuration file in the config daemon directory.

sudo vim /etc/NetworkManager/dnsmasq.d/split-dns.conf

# Listen on the Docker bridge interface
interface=docker0

# Specify a default DNS server to forward queries
server=8.8.8.8  # Example: Google DNS

Check if NetworkManager is running without no problem.

sudo systemctl restart NetworkManager && \
sleep 5 && \
sudo systemctl status NetworkManager

If there are problems, check the NetworkManager logs.

journalctl -u NetworkManager -r

Configure Docker

Check the IP address of the docker0.

ip a show docker0

4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:a9:72:e1:3a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:a9ff:fe72:e13a/64 scope link
       valid_lft forever preferred_lft forever

Configure Docker to use the Dnsmasq

sudo vim /etc/docker/daemon.json

{
  "dns": ["172.17.0.1"]
}

Restart the docker service

sudo systemctl restart docker

Add Test Domain Names to `/etc/hosts`

For testing purposes, add a domain to /etc/hosts.

sudo bash -c "echo '192.168.1.12 nonexisting.example.com' >> /etc/hosts"

Restart the dnsmasq service to get updates.

sudo systemctl restart dnsmasq

Test the Custom Domain Names

In the /etc/hosts file, we defined nonexisting.example.com as 192.168.1.10.

Lets test if the configuration works as expected.

docker run --rm --name dns-test busybox nslookup nonexisting.example.com

The output should be:

Server:     172.17.0.1
Address:    172.17.0.1:53

Non-authoritative answer:

Non-authoritative answer:
Name:   example.com
Address: 192.168.1.10

Viola, you can set any custom domain name in the /etc/hosts file on the host machine and containers will use it by default.

For More

If you like the post, you can head to my personal blog site and read more about DevOps, Linux, system engineering, and self-hosted applications for homelab!

Burak Berk Keskin

Blog of a DevOps & System Engineer

burakberk.dev

Sonatype Nexus Usage 101 | Best Practices & Example Use Cases

Berk — Sun, 06 Oct 2024 20:26:39 +0000

Overview

In my previous blog post, we embarked on a journey to set up Sonatype Nexus with an Nginx reverse proxy. We covered everything from the initial server preparation to accessing the Nexus Web UI securely via HTTPS. Now that you've got Nexus up and running, it's time to roll up our sleeves and dive into the nitty-gritty of using this powerful tool effectively.

Nexus Dashboard

Upon logging into the Nexus web UI, you'll be greeted by a dashboard showcasing several pre-defined repositories.

For those of us who prefer a clean slate and better organization, we're going to start by decluttering. We'll remove all the pre-defined repositories and the default blob storage. This approach allows us to create a more tailored setup, with dedicated repositories and blobs for each category we'll be working with.

Let's begin by logging in with the admin user and navigating to the administration section.

Clearing the Slate

First, we'll tackle the repositories. One by one, we'll select each repository and hit that Delete repository button.

Hit Delete and confirm your choice.

Next, we'll turn our attention to the default blob storage.

Head over to the Blob Stores section and locate the default blob store.

Hit Delete and confirm your choice.

Building Our Custom Setup

Now that we've cleared the decks, it's time to start building our custom setup. We'll create new blob stores, each dedicated to a specific type of repository. This approach allows for better organization and management of our assets.

Let's start by creating a new blob store for our Maven repositories. Navigate to the administrator blob stores menu and click on Create blob store.

Select File as the type and name it maven. This will be our dedicated storage space for all things Maven.

Repeat this process to create blob stores for other types of repositories you'll be working with. In our case, we'll set up separate stores for:

Maven
NPM
Docker
Yum

By the end of this process, you'll have a clean, organized foundation for your Nexus setup.

Maven Repository

Now that we've laid the groundwork, it's time to set up our Maven repositories. Maven is a staple in the Java ecosystem, and having a well-configured Maven repository in Nexus can significantly streamline your development process.

Maven Proxy Repository

We're going to create two Maven proxy repositories. We'll set up one for the central Maven repository and another for the Google Maven repository.

Let's start by navigating to the repositories administrator menu and clicking that inviting Create repository button.

Select maven2 (proxy) from the list.

Now, let's fill in the details for our central Maven repository:

Name: maven-central
Remote Storage: https://repo1.maven.org/maven2/
Blob Store: maven

Hit Create to create the repository.

Great! Now let's repeat this process for the Google Maven repository. Use these values:

Name: maven-google
Remote Storage: https://maven.google.com/
Blob Store: maven

Maven Group Repository

Now that we have our individual Maven proxy repositories set up, it's time to group them together. You can give clients the single URL of the group repository and manage the group repository on the Nexus side without having to change the repository URL.

Head back to the repositories administrator menu and click Create repository again. This time, select maven2 (group).

Select maven2 (group).

Let's set it up with these details:

Name: maven
Blob: maven

In the group section, click on the previously created repositories and use the right arrow to move both repositories to the members column.

Hit that 'Create repository' button, and voila! Your Maven repository is ready to serve.

Raw Proxy for Gradle Zip File

If you're planning to use this Maven repository for Android development, there's one more piece of the puzzle we need to address. The gradlew tool needs to download the Gradle distribution zip file, and we want to ensure it does so through our Nexus repository.

Go to the repositories administrator menu and click Create repository.

Select raw (proxy).

Fill in the details:

Name: gradle-distributions
Remote storage: https://services.gradle.org/distributions/
Blob store: maven

Now maven repositories are ready to use. You can head to the [[#Maven Usage Examples]] if you want to skip other repository configuration examples.

NPM Repository

Next up, let's set up our NPM repository to manage JavaScript packages efficiently.

NPM Proxy Repository

We'll create two NPM proxy repositories: one for the main npm registry and another for the Yarn package registry. This dual setup helps reduce potential downtime and provides a fallback option.

Navigate to the repositories menu and create a new repository, selecting npm (proxy).

For the npm registry, use these settings:

Name: npmjs
Remote Storage: https://registry.npmjs.org
Blob: npm

Hit "Create repository".

Repeat the process for the Yarn registry:

Name: yarnpkg
Remote Storage: https://registry.yarnpkg.com
Blob: npm

NPM Group Repository

Now, let's group these repositories for easier management:

Now NPM repositories are ready to use. You can head to the [[#NPM Usage Examples]] if you want to skip other repository configuration examples.

Docker Repository

Last but not least, let's set up Docker repositories to manage container images efficiently.

We'll create proxy repositories for Docker Hub, GitHub Container Registry, and Red Hat Registry.

For Docker Hub:

Name: dockerhub
Allow anonymous docker pull: true
Remote Storage: https://registry-1.docker.io
Docker Index: Use Docker Hub 
Blob: docker

Hit Create repository button.

Repeat the same process for GitHub container registry

Name: github-container-registry
Allow anonymous docker pull: true
Remote Storage: https://ghcr.io
Docker Index: Use proxy registry
Blob: docker

[!important]
We didn't specify any HTTP or HTTPS port for the dockerhub and github-container-registry repository since we will group them and publish as groupped.

Now create another Docker proxy repository for Red Hat registry but with the values below:

Name: redhat-registry
HTTP: Enable and enter `8083`
Allow anonymous docker pull: true
Remote Storage: https://registry.access.redhat.com
Docker Index: Use proxy registry
Blob: docker

[!important]
We are publishing the Red Hat registry directly, without a group. I am configuring the ports this way since my Nginx configuration is ready for that. You can check the previous blog for the configuration.

Container Registry Group Repository

We will group dockerhub and github-container-registry repositories.

Name: cr
HTTP: enabled on port `8082`
Blob store: docker

Hit create repository.

Maven Usage Examples

To use your Maven repository, update your build.gradle or pom.xml files with the Nexus repository URL. For Android projects, also update the Gradle wrapper properties.

Check the Repository URLs

Go to the repositories section and check the public URL of the maven and gradle-distributions repositories.

Copy the URL.

In my example, the repository URLs:

maven: https://registry.burakberk.dev/repository/maven/
gradle distributions: https://registry.burakberk.dev/repository/gradle-distributions/

Usage For an Android Gradle App

Create an react native application just to test android app.

npx @react-native-community/cli init myreactnativeapp && \
cd myreactnativeapp/android

Edit the build.gradle file and add custom maven url. Remove the google() and other maven repositories.

vim build.gradle

buildscript {
    ext {
        buildToolsVersion = "34.0.0"
        minSdkVersion = 23
        compileSdkVersion = 34
        targetSdkVersion = 34
        ndkVersion = "26.1.10909125"
        kotlinVersion = "1.9.24"
    }
    repositories {
      maven {
        url 'https://registry.burakberk.dev/repository/maven'
      }
    }
    dependencies {
        classpath("com.android.tools.build:gradle")
        classpath("com.facebook.react:react-native-gradle-plugin")
        classpath("org.jetbrains.kotlin:kotlin-gradle-plugin")
    }
}

apply plugin: "com.facebook.react.rootproject"

Also edit the gradle wrapper properties file to download the zip from the Nexus raw repository. Also update the timeout to a higher rate.

vim gradle/wrapper/gradle-wrapper.properties

cat gradle/wrapper/gradle-wrapper.properties
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://registry.burakberk.dev/repository/gradle-distributions/gradle-8.8-all.zip
networkTimeout=120000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

Usage for other Java Applications

Edit the pom.xml file of the project.

vim pom.xml

Change or edit the repositories section. Add the public URL you copied from the nexus.

<repositories>
    <repository>
        <id>nexus-repo</id>
        <url>https://registry.burakberk.dev/repository/maven/</url>
    </repository>
</repositories>

Browse the Maven Repository From Nexus UI

Check the maven repository if there are any downloaded and cached packages to make sure your configurations are correct and the packages are being downloaded from the Nexus.

NPM Usage Examples

Update your package-lock.json or yarn.lock files to use the Nexus NPM repository URL:

Head to the Nexus UI and NPM group repository named npm if you followed the guides above.

Note the repository URL

Sample NodeJS Application

Create a new react application to test the Nexus repository.

mkdir my-app && cd my-app/ && \
npx create-react-app . my-app --template typescript

Usage for NPM

This command will replace all https://registry.npmjs.org URLs in your package-lock.json file with your own Nexus repository URL. It will also backup the original file before replacing the URLs.

NEXUS_NPM_REGISTRY_URL=https://registry.burakberk.dev/repository/npm  && \
DEFAULT_NPM_REGISTRY_URL=https://registry.npmjs.org && \
sed -i '.backup' "s#${DEFAULT_NPM_REGISTRY_URL}#${NEXUS_NPM_REGISTRY_URL}#g" package-lock.json

Install packages.

npm install

If you want to be sure about the downloaded packages are being downloaded from the Nexus URL, you can add --verbose flag to the npm install command.

npm install --verbose

Usage for Yarn

This command will replace all https://registry.npmjs.org URLs in your yarn.lock file with your own Nexus repository URL. It will also backup the original file before replacing the URLs.

NEXUS_NPM_REGISTRY_URL=https://registry.burakberk.dev/repository/npm  && \
DEFAULT_NPM_REGISTRY_URL=https://registry.yarnpkg.com && \
sed -i '.backup' "s#${DEFAULT_NPM_REGISTRY_URL}#${NEXUS_NPM_REGISTRY_URL}#g" yarn.lock

Install packages.

yarn install

If you want to be sure about the downloaded packages are being downloaded from the Nexus URL, you can add --verbose flag to the npm install command.

yarn install --verbose

Browse the NPM Repository From Nexus UI

Check the NPM repository if there are any downloaded and cached packages to make sure your configurations are correct and the packages are being downloaded from the Nexus.

Docker Usage Examples

CR registry: Group of Docker Hub and Github Container Registry

Try to pull a standart image like nginx or ubuntu.

$ docker pull cr.burakberk.dev/nginx
Using default tag: latest
latest: Pulling from nginx
302e3ee49805: Pull complete
d07412f52e9d: Pull complete
9ab66c386e9c: Pull complete
4b563e5e980a: Pull complete
55af3c8febf2: Pull complete
5b8e768fb22d: Pull complete
85177e2c6f39: Pull complete
Digest: sha256:d2eb56950b84efe34f966a2b92efb1a1a2ea53e7e93b94cdf45a27cf3cd47fc0
Status: Downloaded newer image for cr.burakberk.dev/nginx:latest
cr.burakberk.dev/nginx:latest

Successfully pulled the image from the Nexus repository. The image is downloaded from the Docker Hub proxy registry.

Now lets try to pull a Github container registry specific image.

docker pull cr.burakberk.dev/github/super-linter:latest

Successfully pulled ✅

Red Hat Registry

Run the docker pull command on two different registries and see if you can pull the images from the Nexus installation.

For example, pulling the ubi:8.10-1088 image from the redhat-registry.

$ docker pull redhat-registry.burakberk.dev/ubi8/ubi:8.10-1088
8.10-1088: Pulling from ubi8/ubi
148a3ed2f70e: Pull complete
Digest: sha256:a965f33ee4ee57dc8e40a1f9350ddf28ed0727b6cf80db46cdad0486a7580f9d
Status: Downloaded newer image for redhat-registry.burakberk.dev/ubi8/ubi:8.10-1088
redhat-registry.burakberk.dev/ubi8/ubi:8.10-1088

Successfully pulled ✅

Also browse the Nexus repository and check if the image exists.

Conclusion

Congratulations! You've now set up a comprehensive Nexus repository manager capable of handling Maven, NPM, and Docker packages. This setup will provide faster access to packages and greater control over your dependencies.

Remember to regularly maintain your Nexus instance, keeping an eye on storage usage and updating proxy repositories as needed.

Installing Sonatype Nexus With Nginx Reverse Proxy & HTTPS

Berk — Sun, 06 Oct 2024 20:25:17 +0000

Introduction to Sonatype Nexus

Sonatype Nexus is a powerful repository manager that has become an essential tool in modern software development ecosystems. It serves as a central hub for storing and managing various types of artifacts, including Maven dependencies, NPM packages, Docker images, and more.

But for a container image registry, I prefer Harbor:)

Why Use Sonatype Nexus?

Centralized Management: Nexus provides a single point of control for all your software artifacts, simplifying dependency management across projects.
Improved Build Performance: By caching dependencies locally, Nexus significantly reduces build times and network usage.
Enhanced Security: Nexus offers fine-grained access control and can act as a firewall between your organization and public repositories.
Support for Multiple Repository Formats: Nexus can handle various repository types, including Maven, NPM, Docker, and raw repositories, among others.
CI/CD Pipeline Optimization: For organizations running Continuous Integration and Continuous Deployment (CI/CD) pipelines, Nexus acts as a local cache for dependencies. This caching mechanism substantially improves pipeline performance by reducing network-dependent downloads, leading to faster and more reliable builds. ## Installation with HTTPS

In this guide, we'll walk through the process of installing Nexus and configuring Nginx as a reverse proxy, all containerized using Docker. This setup will provide HTTPS access to your Nexus instance.

Prerequisites

Docker and Docker Compose installed on your system
A domain name (for HTTPS configuration)
SSL certificate and key for your domain

Step 1: Set Up the Directory Structure

First, let's create the necessary directory structure for our Nexus installation:

mkdir nexus && cd nexus && \
mkdir -p config/ssl

Step 2: Create Docker Compose File

Create a docker-compose.yaml file in the nexus directory with the following content:

vim docker-compose.yaml

services:
  nexus:
    image: "sonatype/nexus3:3.72.0"
    volumes:
      - "nexus-data:/nexus-data"
    restart: always
  proxy:
    image: "nginx:alpine"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./config/nginx.conf:/etc/nginx/nginx.conf
      - ./config/ssl/:/etc/nginx/ssl
    restart: always
volumes:
  nexus-data:

This configuration sets up two services:

nexus: The Sonatype Nexus service
- There is no ports published from the nexus service since we will use Nginx for all requests.
proxy: An Nginx service acting as a reverse proxy

Step 3: Configure Nginx

Create an Nginx configuration file at config/nginx.conf with the following content:

vim config/nginx.conf

worker_processes 1;

events {
  worker_connections 1024;
}

http {
    proxy_send_timeout 120;
    proxy_read_timeout 300;
    proxy_buffering    off;
    proxy_request_buffering off;
    keepalive_timeout  5 5;
    tcp_nodelay        on;

    server {
        listen 80;
        server_name registry.burakberk.dev;
        location / {
            return 301 https://$host$request_uri;
        }
    }

    server {
        listen 443 ssl;
        server_name  registry.burakberk.dev;
        ssl_certificate /etc/nginx/ssl/server.crt;
        ssl_certificate_key /etc/nginx/ssl/server.key;
        client_max_body_size 2G;
        # optimize downloading files larger than 1G - refer to nginx doc before adjusting
        #proxy_max_temp_file_size 2G;
        location / {
            proxy_pass http://nexus:8081;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto "https";
        }
    }

    server {
        listen 443 ssl;
        server_name cr.burakberk.dev;
        ssl_certificate /etc/nginx/ssl/cr.crt;
        ssl_certificate_key /etc/nginx/ssl/cr.key;
        client_max_body_size 2G;
        # optimize downloading files larger than 1G - refer to nginx doc before adjusting
        #proxy_max_temp_file_size 2G;
        location / {
            proxy_pass http://nexus:8082/;
            proxy_set_header Host $host;
            proxy_set_header X-REAL-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto "https";
        }
    }

    server {
        listen 443 ssl;
        server_name redhat-registry.burakberk.dev;
        ssl_certificate /etc/nginx/ssl/redhat-registry.crt;
        ssl_certificate_key /etc/nginx/ssl/redhat-registry.key;
        client_max_body_size 2G;
        # optimize downloading files larger than 1G - refer to nginx doc before adjusting
        #proxy_max_temp_file_size 2G;
        location / {
            proxy_pass http://nexus:8083/;
            proxy_set_header Host $host;
            proxy_set_header X-REAL-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto "https";
        }
    }
}

This configuration sets up HTTP to HTTPS redirection and proxies requests to the Nexus container.

In Nexus, generally container image registries run on different ports. So if you want to host a container image registry, I recommend to create virtual servers as you need in the Nginx configuration above.

For a brief,

I will create 3 docker proxy registry.
- Docker hub
- Red Hat registry
- GitHub container registry
Group the Docker hub and Github container registry under a common name
- Publish it on the port 8082
- Access with the domain cr.burakberk.dev
Standalone Red Hat registry
- Publish it on the port 8083
- Access with the domain redhat-registry.burakberk.dev

[!tip]
I will share how to configure and use the Nexus repositories in another blog post.

Step 4: Set Up SSL Certificates

You can obtain the SSL certificates from LetsEncrypt for free.
I used self-signed certificates that created with crtforge.

[!note]
If you don't have an SSL certificate, you can obtain one from Let's Encrypt for free if you own a public domain name.
Alternatively, you can create your self-signed SSL certificates by following the Creating Fullchain Self Signed SSL With 3 Commands blog post.

Copy the SSL certificates and keys to the config/ssl directory:

For registry.burakberk.dev:

Certificate: config/ssl/server.crt
Key: config/ssl/server.key

For cr.burakberk.dev:

Certificate: config/ssl/cr.crt
Key: config/ssl/cr.key

For redhat-registry.burakberk.dev:

Certificate: config/ssl/redhat-registry.crt
Key: config/ssl/redhat-registry.key

[!info]
It is best practice to copy fullchain certificate into the server.crt file.

The overall file structure should be:

$ tree
nexus
├── config
│   ├── nginx.conf
│   └── ssl
│       ├── server.crt
│       └── server.key
└── docker-compose.yml

Step 5: Start the Services

Run the following command to start Nexus and Nginx:

docker compose up -d

Step 6: Retrieve Initial Admin Password

To access the Nexus web UI, you'll need the initial admin password. Retrieve it with the following command:

docker compose exec nexus cat /nexus-data/admin.password

Step 7: Access Nexus Web UI

Open your web browser and navigate to https://registry.burakberk.dev (replace with your actual domain). Log in with the following credentials:

username: admin
password: the password you retrieved in Step 6

Follow the initial setup wizard to change the admin password and configure anonymous access if desired. I will enable anonymous access in my case.

Move to the Nexus Usage

You can head over to the my next post to read Sonatype Nexus 101 to see how to configure Maven, NPM and container image repositories. You can also see some use cases and tests in the blog post.

Free, Self-Hosted & Private Copilot to Streamline Coding

Berk — Sat, 31 Aug 2024 07:15:30 +0000

Overview

In today's fast-paced development landscape, having a reliable and efficient copilot by your side can be a game-changer. However, relying on cloud-based services often comes with concerns over data privacy and security. This is where self-hosted LLMs come into play, offering a cutting-edge solution that empowers developers to tailor their functionalities while keeping sensitive information within their control.

The Advantages of Self-Hosted LLMs

Self-hosted LLMs provide unparalleled advantages over their hosted counterparts. By hosting the model on your machine, you gain greater control over customization, enabling you to tailor functionalities to your specific needs. Moreover, self-hosted solutions ensure data privacy and security, as sensitive information remains within the confines of your infrastructure.

Imagine having a Copilot or Cursor alternative that is both free and private, seamlessly integrating with your development environment to offer real-time code suggestions, completions, and reviews. This self-hosted copilot leverages powerful language models to provide intelligent coding assistance while ensuring your data remains secure and under your control. A free self-hosted copilot eliminates the need for expensive subscriptions or licensing fees associated with hosted solutions.

Getting Started with Ollama

In this article, we will explore how to use a cutting-edge LLM hosted on your machine to connect it to VSCode for a powerful free self-hosted Copilot or Cursor experience without sharing any information with third-party services. We will utilize the Ollama server, which has been previously deployed in our previous blog post. If you don't have Ollama or another OpenAI API-compatible LLM, you can follow the instructions outlined in that article to deploy and configure your own instance.

Pre-requirements

Before we dive into the installation process, ensure that you meet the following pre-requisites:

VSCode installed on your machine.
Network access to the Ollama server. If you are running the Ollama on another machine, you should be able to connect to the Ollama server port.
If you want to test if you can connect to the Ollama server on another machine, you can use the following command:

$ curl http://192.168.1.100:11434
Ollama is running

If you want to test if you can connect to the Ollama server on the local machine, you can use the following command:

$ curl http://127.0.0.1:11434
Ollama is running

If you don't have Ollama installed, check the previous blog.

Installing the Continue plugin

To integrate your LLM with VSCode, begin by installing the Continue extension that enable copilot functionalities.

Configure the Continue Plugin

In MacOS and Linux, you can find the configuration file in ~/.continue/config.json. In Windows, it is in %userprofile%\.continue/config.json.

Edit the file with a text editor. Here I will show to edit with vim.

vim ~/.continue/config.json

In the models list, add the models that installed on the Ollama server you want to use in the VSCode. In the example below, I will define two LLMs installed my Ollama server which is deepseek-coder and llama3.1.

{
  "models": [
    {
      "title": "Llama 3",
      "provider": "ollama",
      "model": "llama3"
    },
    {
      "title": "Ollama",
      "provider": "ollama",
      "model": "AUTODETECT"
    },
    {
      "title": "Local Ollama DeepSeek-Coder 6.7b",
      "provider": "ollama",
      "model": "deepseek-coder:6.7b-instruct-q8_0",
      "apiBase": "http://192.168.1.100:11434"
    },
    {
      "title": "Local Ollama LLama3.1",
      "provider": "ollama",
      "model": "llama3.1:8b-instruct-q8_0",
      "apiBase": "http://192.168.1.100:11434"
    }
  ],
...
}

Save and exit the file. If you use the vim command to edit the file, hit ESC, then type :wq! and hit enter.

Test the Ollama LLMs on VSCode with Continue

Open the VSCode window and Continue extension chat menu.

Check if the LLMs exists that you have configured in the previous step. Send a test message like "hi" and check if you can get response from the Ollama server.

You can use that menu to chat with the Ollama server without needing a web UI.

Using Continue as Copilot Alternative

To use Ollama and Continue as a Copilot alternative, we will create a Golang CLI app.

Lets create a Go application in an empty directory.

mkdir aitest
cd aitest

Open the directory with the VSCode.

code .

Create a file named main.go.

Hit command (ctrl on Windows/Linux) + I to open the Continue context menu.

Copy the prompt below and give it to Continue to ask for the application codes.

Create a command-line interface (CLI) application that meets the following requirements: The application should display its version, provide a help message, and include usage examples. The CLI should accept one positional argument, which will be a number. Upon execution, the application should print numbers from 0 up to the provided number. Additionally, while printing each number, the application should display a progress bar indicating the percentage of completion. The progress bar should use the symbol "#" to represent the progress and also display the percentage as a number. The application should wait 3 milliseconds before printing each number to simulate the progress. The output should always be on one line, and each new log should overwrite the previous one. Do not use any external libraries for this task.

You can see the performance and quality of the code from the video below:

Running Ollama and Open WebUI Self-Hosted With AMD GPU

Berk — Sat, 25 May 2024 09:04:54 +0000

Why Host Your Own Large Language Model (LLM)?

While there are many excellent LLMs available for VSCode, hosting your own LLM offers several advantages that can significantly enhance your coding experience. Below you can find some reasons to host your own LLM.

Customization and Fine-Tuning
Data Control and Security
Domain Expertise
Easy Switching Between Models

Prerequisites

To host your own Large Language Model (LLM) for use in VSCode, you'll need a few pieces of hardware and software in place.

Hardware Requirements

For this example, we'll be using a Radeon 6700 XT graphics card and a Ryzen 5 7600X processor on Linux. However, you can also host an LLM on Windows or macOS machines with compatible hardware.

A modern CPU (at least quad-core) with high-performance capabilities
A suitable graphics card with OpenCL or HIP support (Radeon or NVIDIA)
At least 16 GB of RAM for smooth performance

Software Prerequisites

To get started, you'll need to install the packages you need on your Linux machine are:

Docker
GPU drivers.
Nvidia Container Toolkit (if you use Nvidia GPU)

[!tip]
If you don't have docker installed already, please check the Docker Installation document.

It doesn't matter if you are using Arch, Debian, Ubuntu, Mint etc. Since we will use containers, the environment will be the same.

Note that we won't be training our own LLM models; instead, we'll focus on hosting and running pre-trained models. This means you won't need a high-performance GPU or specialized hardware for model training.

With these prerequisites in place, you're ready to start setting up your LLM hosting environment!

Deploying the AI

We will deploy two containers. One for the Ollama server which runs the LLMs and one for the Open WebUI which we integrate with the Ollama server from a browser.

To deploy Ollama, you have three options:

Running Ollama on AMD GPU (Recommended)

If you have a AMD GPU that supports ROCm, you can simple run the rocm version of the Ollama image.

docker run -d --restart always --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm

If your AMD GPU doesn't support ROCm but if it is strong enough, you can still use your GPU to run Ollama server. I use that command to run on a Radeon 6700 XT GPU.

docker run -d --restart always --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama -e HSA_OVERRIDE_GFX_VERSION=10.3.0 -e HCC_AMDGPU_TARGET=gfx1030 ollama/ollama:rocm

[!info]
If you run LLMs that are bigger than your GPUs memory, then they will be loaded partially on the GPU memory and RAM memory. This will cause a slow response time in your prompts.

Running Ollama on CPU Only (Not Recommended)

If you run the ollama image with the command below, you will start the Ollama on your computer memory and CPU.

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

[!warning]
This is not recommended if you have a dedicated GPU since running LLMs on with this way will consume your computer memory and CPU.

Also running LLMs on the CPU are much slower than GPUs.

Verifying Installation

After you have deployed the Ollama container, you can manually check if the Ollama server is running successfully.

docker exec ollama ollama list

You won't see any LLM in the list because we haven't downloaded any.

Deploying Web UI

We will deploy the Open WebUI and then start using the Ollama from our web browser.

Since our Ollama container listens on the host TCP 11434 port, we will run our Open WebUI like this:

docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main

After the container is up, you can head to your browser and hit the http://localhost:8080 if Open WebUI is running on your own computer. If it's on another computer, you can use http://ip-address-or-domain:8080 to access Open WebUI from browser.

Pulling LLM AI Models

Ollama provides LLMs ready to use with Ollama server. To view all the models, you can head to Ollama Library.

Since my GPU has 12GB memory, I run these models:

Name: deepseek-coder:6.7b-instruct-q8_0 , Size: 7.2GB: I use that LLM most of the time for my coding requirements.
Name: llama3:8b-instruct-q8_0 , Size: 8.5 GB: I use that LLM to chat if I am writing emails or etc.
Name: deepseek-coder:33b-instruct-q4_0 , Size: 18 GB: I use that LLM for challenging coding requirements.

You can paste the LLM name into the red box to pull the LLM image.

Performance

If you want to see how the AI is performing, you can check the i button of response messages from AI.

At the first message to an LLM, it will take a couple of seconds to load your selected model.
As you can see below, the LLM took 9 seconds to get loaded. Then answered in 1 second.

The messages after the first one doesn't took that load time.

As you can see below, it started to response after 0.5 seconds and all the answer took 7.7 seconds.

Also you can check your GPU statics if you want to be sure about where the LLM is running.

For the AMD GPUs, you can use radeontop

For Nvidia GPUs, you can use nvidia-smi.

This is my radeontop command outputs while a prompt is running:

For More

If you want to use the deployed Ollama server as your free and private Copilot/Cursor alternative, you can also read the next post in the series!

Free, Self-Hosted & Private Copilot to Streamline Coding

Berk ・ Aug 31 '24

#githubcopilot #ai #llm #cursor

If you like the post, you can head to my personal blog site and read more about DevOps, Linux, system engineering, and self-hosted applications for homelab!

Burak Berk Keskin

Blog of a DevOps & System Engineer

    <div class="color-secondary fs-s flex items-center">
        <img
          alt="favicon"
          class="c-embed__favicon m-0 mr-2 radius-0"
          src="https://burakberk.dev/content/images/size/w256h256/2024/02/website-logo2.jpeg"
          loading="lazy" />
      burakberk.dev
    </div>
  </div>
</div>

How to Create VPC and EC2 on AWS for Free Tier With Terraform

Berk — Wed, 03 Apr 2024 19:58:38 +0000

Overview

In this post, you will learn how to deploy resources to AWS without exceeding the free limits with Terraform.

Once you follow this tutorial, you will have:

A VPC with 10.10.0.0/16 CIDR,
A public subnet with 10.10.20.0/24 CIDR on eu-central-1a AZ,
- A public subnet with 10.10.40.0/24 CIDR on eu-central-1b AZ,
- A private subnet with 10.10.21.0/24 CIDR on eu-central-1a AZ,
- A private subnet with 10.10.41.0/24 CIDR on eu-central-1b AZ,
- Security groups for SSH, HTTP(S) for ingress and all for egress,
A Public Elastic IP,
An Ec2 Instance,
- Ubuntu Linux
- t2.micro type,
- 25GB storage,

All these resources are free if you are on AWS free tier!

You can see an example diagram of the resources.

Pre-Requirements

An AWS account.
A Linux or macOS computer to run Terraform.
Git

Install Terraform

You can skip this section if you already have Terraform installed on your computer.

If you are reading this tutorial, you know what Terraform is. Terraform is an open-source Infrastructure as Code (IaC) tool that enables users to define and provision infrastructure efficiently across various cloud providers.

Installation on MacOS

If you have Homebrew installed on your computer, you can use brew to install Terraform.

Open a terminal and run these commands:

brew tap hashicorp/tap && \
brew install hashicorp/tap/terraform

Once the installation is finished, you can check the Terraform version and verify the installation.

terraform --version

Installation on Windows

You can head to the official Terraform installation document and download the latest Terraform version.

But my recommendation is to use Linux with WSL.

Installation on Linux

You can head to the official Terraform installation document and click on the tab for your distro.

If you are using Arch Linux and have yay tool installed , you can simply install Terraform with the commands below.

yay -S terraform

Create an AWS Service Account for Terraform

If you have a service account for the terraform, you can skip this step.

To create a service account, you can follow the tutorial below step by step.

Creating a Service Account on AWS For Terraform

burakberk.dev

Create an SSH Key For Linux Server Authentication

You can skip this section if you already have an SSH key to use on a Linux server.

To enhance Linux server security, generate an SSH key pair using the ssh-keygen command, enabling secure authentication by pairing a private key on the client with the server's authorized public key.

A SSH key pair consists of 2 files, public and private. You should keep the private one safe and secure. You can think that a private file opens the locks. You should use the public one to lock something like a Linux server.

In summer, we will use a public key to lock our server SSH on our Terraform configuration. After the server is up and running, we will use the private one to connect to the server.

To create a SSH key pair, you can use this command on your macOS or Linux computer.

-f ~/.ssh/aws-ec2: The file path and name where you want to create the key pair.
-C "contact@burakberk.dev": The comment can be anything.
Once you run the command below, it will ask you to enter a passphrase. This is password protection for the private file. You should use a secure and strong password.

ssh-keygen -t ed25519 -f ~/.ssh/aws-ec2 -C "contact@burakberk.dev"

To view the private file that we will use to connect to the server, you can run:

cat ~/.ssh/aws-ec2

To view the public file that we will use on Terraform, you can run:

cat ~/.ssh/aws-ec2.pub

Create AWS Credentials Environment Variables

Since Terraform must be authenticated to your AWS account, we will use the access keys we created earlier.

There are multiple ways to use those keys. But in this tutorial, I will use the environment variables method, which is unsafe for the long term.

For more details about AWS authentication, you can read the AWS Terraform documentation.

We could run the export command below directly, but this will save the secret access keys in the shell history, which is very unsafe. So we will create a file and put the keys into it.

Create a file named terraform-aws.sh with vim

vim terraform-aws.sh

Copy this template and paste it into the file.

#!/bin/bash
export AWS_ACCESS_KEY_ID=YourAccessKeyId
export AWS_SECRET_ACCESS_KEY=YourSecretAccessKey

Press the i button, move the cursor with the arrow buttons on your keyboard, and edit the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables with yours.

Press esc, colon (:) and write wq! and hit enter on your keyboard to save the changes and exit the editor.

❗Keep this file safe and secure. Whoever gets access to the file controls all resources on your AWS account.

Now you can use this file to use your keys.

source terraform-aws.sh

📚Note: If you close your terminal or open a new one, you should run the source command each time before running terraform commands.

If your AWS variables get lost and you run terraform commands, you will get an error, like in the example below:

Planning failed. Terraform encountered an error while generating this plan.

Error: configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.

Download the Sample Terraform Files From Github

You should create a couple of files. Instead of creating them manually, you can just download them from the Github repository.

Open a terminal and change the directory to the home directory. You can choose anywhere you want.

cd $HOME

Then you can download the Terraform files from the Github repository.

git clone --depth 1 --branch blog-tutorial https://github.com/burakberkkeskin/personal_infra.git && \ 
cd personal_infra/main

Once the download is finished and you are in the Terraform configuration directory, you can run terraform init to see if Terraform configuration is ready to be used.

❗Be aware that you are in the personal_infra/main directory. The terraform commands below should also run in this directory too.

terraform init

Configure and Customize Terraform Variables

Add the SSH Key

Print the public key you created earlier and copy it.

cat ~/.ssh/aws-ec2.pub

Open the variables.auto.tfvars file with a file editor.

vim variables.auto.tfvars

Press the i button, move to a new line with the arrow buttons on your keyboard, and create a new variable named ec2_public_key like in the example below:

// general variables
 project_name = "main"
 aws_region = "eu-central-1"
 aws_zone = "eu-central-1a"

 ec2_public_key = "ssh-ed25519 SuperSecretKeylZDI1NTE5AAAAINa3hu5qHgD3cS7dxMlO6I3dTTx/AFU5HqhNZnM6rnEJ contact@burakberk.dev"

 ...

Press esc, colon (:) and write wq! and hit enter on your keyboard to save the changes and exit the editor.

You can also configure other variables as you wish, like the aws_region, project_name.

Terraform Plan

To see if everything is ready to use, you can run terraform plan command to see the resources that will be created.

terraform plan

If you see that there are resources that will be created as a plan, it means you have configured terraform correctly ✅

Terraform Apply

Once you are ready to create the resources you saw in the terraform plan output, you can run terraform apply command.

terraform apply

It will ask you to be sure.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

Write yes and hit enter. Within a few minutes, your resources will be ready to use.

Apply complete! Resources: 23 added, 0 changed, 0 destroyed.

Outputs:
ec2_instance_private_ip = "10.10.20.136"
ec2_instance_public_ip = "35.158.26.83"

To check from the AWS Console, you can head to the EC2 instance. You can also see the IP address of the instance.

Ta da 🥳 Your Ubuntu Linux server is ready to use.

SSH Into the Server For Testing

You have already seen the IP address of the instance. To connect, you can open a terminal and run ssh command:

-i ~/.ssh/safaws: The path to the private SSH key.
ubuntu: Default username of the Ubuntu AMI on the AWS.
35.158.26.83: Public IPv4 address.

ssh -i ~/.ssh/safaws ubuntu@35.158.26.83

If it is the first time that you are connecting to a server, it will ask if you are sure to connect. Write yes enter hit Enter.

Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

If you have followed the tutorial until now, congratulations, You are on the Linux server!

You can use this server however you like. For example, you can host a website, make it a VPN server, a database server, or anything else you want.

Example Nginx Container

In this step, you will see how to deploy a nginx container. The docker should be already installed in the instance. All you need to do is run the docker run command.

Let's run a Nginx container.

sudo docker run --name "website" --rm -d -p "80:80" nginx

To test if everything works, you can simple open a browser and head to the public IPv4 address like:

http://35.158.26.83

Destroying the All Resources

If you want to destroy all the resources for some reason, like the end of the free tier, you can destroy all resources easily.

First, you should change the directory to the Terraform you downloaded earlier.

cd ~/personal_infra/main

You should activate the AWS keys again for authentication.

source terraform-aws.sh

Now, it is time to destroy all resources.

terraform destroy

It will ask you to be sure. Write yes and hit Enter.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

Once all resources are destroyed, you will see an output like below.

Destroy complete! Resources: 23 destroyed.

Re-Use on Another Account

If you want to use the exact same configuration, you can just change the AWS access keys in the terraform-aws.sh. file and apply the same configuration. With that, you can change AWS accounts under 10 minutes!

For More

If you like the post, you can head to my personal blog site and read more about DevOps, Linux, System Engineering!

Burak Berk Keskin

Blog of a DevOps & System Engineer

burakberk.dev

Creating a Service Account on AWS For Terraform

Berk — Sun, 03 Mar 2024 10:31:52 +0000

Overview

For Terraform to create resources on your AWS account, it should get some access keys to authenticate to your account. In this step, we will create a service account for Terraform to use.

We will create an admin user group, then create a Terraform user and add it to the admin group. Then create access keys for the Terraform user. You can follow the steps below to create them all.

When you are in the AWS Console, search for the IAM service.

Head to "User groups" and click on the "Create group" button.

Enter your group name, search for administratorAccess and select it, and hit the "Create group" button.

Click on the "Users" menu, then click the "Create user" button to create a user for Terraform.

Enter the "User name" and hit the "Next" button.

Add the new user to the group previously created. So the user will have "Administrator Access" permissions.

Review the user and hit "Create user" to continue.

To create a access keys for the new user, hit the username.

Change the tab to "Security credentials" and hit the "Create access key" button.

Change the "Other" type and hit the "Next" button.

You can write a description of why you have created this access key. Hit the "Create access key" button to continue.

Now you can see the access keys. Copy both of them and paste them somewhere else securely. Hit "Done" to continue.

You can use those keys in the Terraform configuration.

❗Keep these access keys safe and secure. Whoever gets access to the keys controls all resources on your AWS account.

If you enjoyed this DevOps article and crave more insights into System Administration, visit my blog for additional resources and tutorials. Thank you for exploring the world of efficient software development with me, and stay tuned for more on my blog!

https://burakberk.dev

Creating Fullchain Self Signed SSL With 3 Commands

Berk — Thu, 21 Dec 2023 09:38:19 +0000

Overview

I wrote about how to create a fullchain ssl certificate manually in a previous post. Today, I will show you how you can create fullchain ssl cert with only 3 commands.

We will use a tool named crtforge which is free and open-source application that you can find on Github.

Install `crtforge`

To install crtforge, you need Linux or macOS. You can also use the crtforge on Windows with wsl.

sudo curl -L -o /usr/bin/crtforge https://github.com/burakberkkeskin/crtForge/releases/latest/download/crtforge-$(uname -s)-$(uname -m) && \
sudo chmod +x /usr/bin/crtforge

These two command will install crtforge binary to your system.

Create a fullchain cert.

You can directly run crtforge with a appname and domains that app needs.

crtforge websiteApp myapp.com app.myapp.com

Congratulations 🎉, you will have your fullchain with 3 commands in total.

You can get the certs from the $HOME/.config/crtforge/default/websiteApp directory.

For detailed usage like custom root CA, custom intermediate CA and trusting the root CAs, you can check the readme of the repository.

How to Test SMTP Settings under 5 Minutes

Berk — Thu, 21 Dec 2023 09:35:42 +0000

Overview

In this tutorial, you will see how to test SMTP settings with a tool named gomtp on Linux and macOS cli.

gomtp is a open-source and free software to test SMTP settings. You can see the source code on the Github page.

Install the gomtp tool.
Configure the SMTP settings.
Test the configuration.

Install gomtp

To install the gomtp, you can simply run the command below:

sudo curl -L -o /usr/local/bin/gomtp "https://github.com/burakberkkeskin/gomtp/releases/latest/download/gomtp-$(uname -s)-$(uname -m)" && \
sudo chmod +x /usr/local/bin/gomtp

SMTP configuration.

To test SMTP settings, you first need to configure a yaml file.

Create a file named gomtp.yaml.

vim gomtp.yaml

❗The file name must be gomtp.yaml by default.

Configure the SMTP settings for your needs.

##### Gmail Example #####
 username: 'from@gmail.com'
 password: 'appPassword'
 from: 'from@gmail.com'
 to: 'to@example.com'
 host: 'smtp.gmail.com'
 port: 587
 ssl: false
 tls: true
 auth: 'LOGIN'
 subject: 'Testing Email'
 body: |
   this is line 1
   This is line 2

ℹ You can find some other example configurations under the repository.

Test the Configuration

Now it's time to test the configuration.

Be sure that you are in the same directory with the gomtp.yaml file you have just created previously.
Run the gomtp directly with no argument.

gomtp

If the configuration is valid, you will see this output.

$ gomtp
Email sent successfully!

If the configuration is invalid, you will see an error message that describe the problem.

$ gomtp
2023/12/21 11:44:33 535 5.7.8 Error: authentication failed: Invalid user or password! 1703148273-WiJ7lX77XKo0

DEV Community: Berk

Split-Horizon DNS on AdGuard: One Domain, Two Networks, Two IP Addresses

Split-Horizon DNS with AdGuard Home

What’s Split-Horizon DNS?

Testing

Result

Fixing Unknown SMART Status for External SSD/HDD on Proxmox

Overview

Enabling Secure Boot with Linux and Windows Dual-Boot Setup

Overview

Important Prerequisites

Step-by-Step Implementation

1. Disable Secure Boot

2. Delete Secure Boot Keys

3. Install and Configure sbctl

4. Check Status

5. Generate and Enroll Keys

6. Configure GRUB Bootloader

7. Sign Boot Files

8. Enable Secure Boot

9. Verify Configuration

Troubleshooting Guide

Common Issues and Solutions

Maintenance Notes

Security Considerations

How to Add Domains and IP Addresses to Docker Containers

Overview

Table of Contents

Install Dnsmasq

Configure Dnsmasq

Dnsmasq Standalone Usage

Dnsmasq with NetworkManager

Configure Docker

Add Test Domain Names to /etc/hosts

Test the Custom Domain Names

For More

Burak Berk Keskin

Sonatype Nexus Usage 101 | Best Practices & Example Use Cases

Overview

Nexus Dashboard

Clearing the Slate

Building Our Custom Setup

Maven Repository

Maven Proxy Repository

Maven Group Repository

Raw Proxy for Gradle Zip File

NPM Repository

NPM Proxy Repository

NPM Group Repository

Docker Repository

Container Registry Group Repository

Maven Usage Examples

Check the Repository URLs

Usage For an Android Gradle App

Usage for other Java Applications

Browse the Maven Repository From Nexus UI

NPM Usage Examples

Sample NodeJS Application

Usage for NPM

Usage for Yarn

Browse the NPM Repository From Nexus UI

Docker Usage Examples

CR registry: Group of Docker Hub and Github Container Registry

Red Hat Registry

Conclusion

Installing Sonatype Nexus With Nginx Reverse Proxy & HTTPS

Introduction to Sonatype Nexus

Why Use Sonatype Nexus?

Prerequisites

Step 1: Set Up the Directory Structure

Step 2: Create Docker Compose File

Step 3: Configure Nginx

Step 4: Set Up SSL Certificates

Step 5: Start the Services

Step 6: Retrieve Initial Admin Password

Step 7: Access Nexus Web UI

Move to the Nexus Usage

Free, Self-Hosted & Private Copilot to Streamline Coding

Overview

The Advantages of Self-Hosted LLMs

3. Install and Configure `sbctl`

Add Test Domain Names to `/etc/hosts`

Install `crtforge`