DEV Community: Blacknight318

Setup Blinko Notes with Ollama

Blacknight318 — Wed, 07 May 2025 11:00:00 +0000

Another note app?

In the past we've looked at, and used, Obsidian and Joplin. While both are great note-taking apps I'd been looking for one that had a responsive webui and possibly the ability to use a local LLM like Ollama or Exo. Blinko is a little like a callback to Mem but self-hosted and able to use local LLM's, adding a level of control over your data. This guide will be a quick setup using Docker, with a few tweaks to avoid a few headaches in getting things working with Ollama(or any AI).

Docker Compose Setup

While there is an install.sh script that works, I tend to prefer having a docker-compose.yml file with a few tweaks to bolster security. Below are the contents of that docker-compose.yml file. Note the lines that have changeme in them, these need to be unique. For those who don't already have a running Ollama server, and don't intend to use a GPU, you can uncomment the Ollama lines to start a dockerized isolated instance.

Let's open a shell and create a folder for Blinko, along with a data folder, and the docker compose file.

cd ~
mkdir blinko
cd blinko
mkdir data
nano nano docker-compose.yml

Now let's paste the following into the file, being sure to make the above changes.

networks:
    blinko-network:
    driver: bridge

services:
    blinko-website:
        image: blinkospace/blinko:latest
        container_name: blinko-website
        environment:
        NODE_ENV: production
        # NEXTAUTH_URL: http://localhost:1111
        # IMPORTANT: If you want to use sso, you must set NEXTAUTH_URL to your own domain
        # NEXT_PUBLIC_BASE_URL: http://localhost:1111
        # IMPORTANT: Replace this with your own secure secret key!
        NEXTAUTH_SECRET: ChangeMe
        DATABASE_URL: postgresql://postgres:changeme@postgres:5432/postgres
        depends_on:
        postgres:
            condition: service_healthy
        # Make sure you have enough permissions.
        volumes:
            - /home/user/blinko/data/.blinko:/app/.blinko
        restart: always
        logging:
        options:
            max-size: "10m"
            max-file: "3"
        ports:
            - 1111:1111
        healthcheck:
        test: ["CMD", "curl", "-f", "http://blinko-website:1111/"]
        interval: 30s
        timeout: 10s
        retries: 5
        start_period: 30s
        networks:
            - blinko-network

    postgres:
        image: postgres:14
        container_name: blinko-postgres
        restart: always
        ports:
            - 5435:5432
        environment:
            POSTGRES_DB: postgres
            POSTGRES_USER: postgres
            POSTGRES_PASSWORD: CrthQDQK7k
            TZ: America/Chicago
        # Persisting container data
        # Make sure you have enough permissions.
        volumes:
            - /home/chief/blinko/data/.db:/var/lib/postgresql/data
        healthcheck:
            test: ["CMD", "pg_isready", "-U", "postgres", "-d", "postgres"]
            interval: 5s
            timeout: 10s
            retries: 5
        networks:
            - blinko-network

    # ollama: # Add the Ollama service
        # image: ollama/ollama
        # container_name: ollama
        # restart: always
        # volumes:
            # - /home/chief/blinko/ollama:/root/.ollama
        # No need to expose ports here if only accessed internally
        # networks:
            # - blinko-network

Now we can start up Blinko with the following command.

docker compose up -d

Getting Started with Blinko

Now that we're up and running we can connect to our Blinko instance and register the initial account on the login screen. Once that's done we'll want to click on your account name in the upper left-hand corner of the screen and go to settings and click on the AI header withing settings.

Here there be dragons

When setting up the AI settings for things like AI tagging, AI writing, and Chat(which may time out depending on your Ollama server), I ran into trouble getting things to talk correctly. After trying various combinations of address formats and models I found a closed issue on Blinko's Github Page stating that that person had to manully enter in the model name, after trying this things clicked into place and started working.

First, let's open a shell on our Ollama server and download a few of the models we'll start with.

ollama pull llama3.2:latest
ollama pull nomic-embed-text:latest
ollama pull linux6200/bge-reranker-v2-m3:latest

In the AI screen we previously opened let's change the following.

Check Use AI
Select Ollama for the Model Provider
For the Model type in llama3.2:latest

Set the Endpoint to http://ollama-server-ip:11323/api
Type into Embedding Model nomic-embed-text:latest

Type into Rerank model linux6200/bge-reranker-v2-m3:latest

In closing

With that, we should have a working instance of Blinko with a connection to an in-house Ollama instance. While testing I've found, at least with llama3.2, that AI writing was ok if a little error-prone, but the AI tagging was a great help.

If you found this post helpful please consider buying me a coffee. Till next time, fair winds and following seas.

Raspberry Pi 5 Ollama Llama3.x Performance

Blacknight318 — Mon, 03 Feb 2025 12:00:00 +0000

This post contains affiliate links. If you make a purchase through these links, I may earn a small commission at no extra cost to you. Thanks for supporting Salty Old Geek! It helps cover the costs of running the blog.

Why a Raspberry Pi 5

Back in April of this year, I posted an article about setting up an Ollam, Llama3, and OpenWebUI. In parts one and two I looked at trying to create a 1U unit with an external GPU using some adapters and setting up a server using an old gaming PC with the same external GPU(Nvidia Quadro K620). This did work but was under-spec'd with only 2GB VRAM. In the search for a cost-effective node structure I've narrowed it down to 2 options, the first of which we'll go over in this post, the Raspberry Pi 5 8GB RAM. Let's dive in!

Testing

To give a reference between certain setups I started with a prompt to work the CPU and provide a meaningful run time.

The test prompt and benchmark times

Test prompt
- "Write a detailed 500-word essay on the importance of renewable energy in combating climate change."
Llama 3.1 model
- Pi 5 time Stock Case: 8min
- Pi 5 time 3d case: 6 min
- Pi 5 time Argon Neo case: 5 min 30 sec
- Beast time(Ryzen 7 5800x w/32GB RAM): 1 min 30 sec
- M3 Bacbook Air 8GB RAM: Failed to complete, however MacBook Pro with 16GB did finish at or better than Beast's time
Lamma 3.2 model
- Pi 5 time Stock Case: 2 min 55 sec
- Pi 5 time 3d case: 2min 50 sec
- Pi 5 time Argon Neo case: 2 min 30 sec
- Beast time(Ryzen 7 5800x w/32GB RAM): 1 min
- M3 Bacbook Air 8GB RAM: 38 sec

NUMA, is it worth the patch?

While looking for ways to improve performance, if not with shorter times than with thermal performance, I came upon an article by Jeff Geerling "NUMA Emulation speeds up Pi 5 (and other improvements)" and decided to give that a try. Now, I tested this month after Jeff's post had been published, so the patch might now be standard in the Raspberry Pi kernel, and in my case, the patch made no meaningful difference.

TLDR

For the best value, the Raspberry Pi 5 with the Argon Neo NVME case is my second choice(only to an M series MacBook/Mac Mini with 16GB RAM). If you're comfortable with a little longer answer times or some prompt tweaking then running the Pi setup with Ollama and llama3.2:latest is the sweet spot. I may go into cost more in a future post. That said, if you wanted more power I'd go for the MacBook or Mac Mini, any M series with +16GB RAM and you'd still come in under a PC with NVIDIA GPU(s) equivalent for the job.

I hope you found this post helpful, if so consider buying me a coffee. till next time fair winds and following seas.

Sync Joplin Clients with Self-Hosted Server

Blacknight318 — Mon, 30 Dec 2024 07:00:00 +0000

Recap

It's been a minute since my last post, life happens, so here's a continuation of the previous post "Self-Host a Joplin Sync Server in Proxmox". In that post we set up a self-hosted Joplin sync server, here we'll use that server to sync our Joplin clients (be it Mac or Windows desktop, iPad, iPhone, or my favorite Android). We'll need the email and password we set up in the initial Docker Compose file on the server(or another user if you've added one).

Joplin client

(This post uses the MacOS client as an example)

Here are the steps to set up the MacOS Client.

Click on Joplin at the top left of the screen
Click on settings
Click on Synchronization
Fill in the highlighted fields with the IP/URL of the sync server and the email and password we set up previously.
Click Check synchronization configuration to make sure all works

Wrap-up

That's it, pretty simple! If you're still on the fence about Joplin check out this post over on XDA "9 reasons you should self-host Joplin on your Raspberry Pi".

Look forward to more posts coming up in the new year. If you found this post, or any of my other posts helpful, consider buying me a coffee. Til next time fair winds and following seas.

Self-Host a Joplin Sync Server in Proxmox

Blacknight318 — Thu, 10 Oct 2024 21:03:28 +0000

Intro

In a previous post, I had written a guide to set up Obsidian and CouchDB for sync, since then I've struggled to get in a groove and keep things running. This post is the first in a two-part series on setting Joplin and hosting your own sync server. The sync path is a little more straightforward than the Obsidian method. That being the case, and not really using Obsidian in a way that worked for me, I've switched. Let's get into setting up the server and in the next post I go over more on why I switched and how to set up the sync server in the Joplin clients themselves.

Creating a base LXC for Joplin

We'll want to run a few commands to get a base LXC for our server. Since my home lab is a hyper-converged Promxox Cluster, and this project is using Docker so I decided to use a stripped-down LXC, the script for which you can find on tteck's site.

Running the Proxmox Script to Set Up LXC

Run the following in the shell on your Proxmox node/server. While going through the prompts use advances and change the disk size to 5GB to start, say yes to docker compose, and the rest you can leave as the default(maybe change the name) then you're good to go.

bash -c "$(wget -qO - https://github.com/tteck/Proxmox/raw/main/ct/alpine-docker.sh)"

Joplin Sync Server

When researching this I found an article over on Vultr by Humphrey Mpairwe which served as my starting point.

Important note on reverse proxies

If you're using NPM(Nginx Proxy Manger) or Traefik you'll want to go through that to set up a reverse proxy. In my case, I added a subdomain and forwarder with Cloudflare Tunnels.

Bootstrapping the sync server itself

Run the following commands to get started with the prep work.

mkdir /opt/joplin
cd /opt/joplin
nano docker-compose.yml

Now we'll want to paste the following into the docker-compose.yml file.

version: '3'

services:
  db:
    image: postgres:13
    volumes:
      - ./data/postgres:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    restart: always
    environment:
      - POSTGRES_PASSWORD=strong-password
      - POSTGRES_USER=joplin-user
      - POSTGRES_DB=joplindb
  app:
    image: joplin/server:latest
    container_name: joplin-server
    depends_on:
      - db
    ports:
      - "8080:8080"
    restart: always
    environment:
      - APP_PORT=8080
      - APP_BASE_URL=https://joplin.example.com
      - DB_CLIENT=pg
      - POSTGRES_PASSWORD=strong-password
      - POSTGRES_DATABASE=joplindb
      - POSTGRES_USER=joplin-user
      - POSTGRES_PORT=5432
      - POSTGRES_HOST=db

Once that's in place and you've changed the app base URL and passwords, press ctrl+x then y to save the file. Now we have just one more command to get things going.

docker compose up -d

After a minute you should be able to go to the URL we set up earlier, the default username is admin@example.com and the password is admin, which you'll want to change.

Wrapup

If you found this useful consider following, or clapping if you're reading this on Medium, consider buying me a coffee.

Til next time fair winds and following seas.

Fixing JAMF Renaming Script Issues

Blacknight318 — Mon, 19 Aug 2024 13:00:00 +0000

Recap

In a previous post I wrote about creating a script to rename a Mac using the barcode1 field within JAMF. Since then we've done a few deployments and noticed a few tweaks/changes in JAMF. This post will go over what's new.

Users and Passwords

This came to me from another JAMF Pro user, at some point, JAMF had changed the minimum password length and disabled any accounts that didn't meet this requirement. We also decided to add read access API Integrations and API Roles. This wasn't a huge deal, but it did make troubleshooting the next issue more of a challenge.

Field Names and Variables

In the script we created, it calls fillable fields from JAMF, we used $4, $5, and $6. At some point it seems the field names in the script changed to variable names, this left the field names as $user instead of $5, etc. This was under Settings > Scripts > Options, after clearing the Parameters to default we can then change the fields under the Policies> Options > Scripts itself. With those out of the way, we should now be able to run the script.

Order of Operations

The last issue to tackle was a little trickier. We were seeing intermittent execution and completion of the script, we ruled out network issues and Prestage Enrollments. We finally narrowed it down to the script's priority within the policy. We changed it from after to before resolving the intermittent execution issue.

Wrapup

Correcting the Parameters fields, ensuring account permissions and passwords meet requirements, cleared up those issue. This is a good reminder to subscribe and read the Release Notes/Changelogs.

If you found this article helpful consider buying me a coffee. Till next time, fair winds and following seas.

Easily Split and Rename PDFs for Skyward

Blacknight318 — Mon, 29 Jul 2024 13:00:00 +0000

Why build it and what does it do

A few weeks ago my supervisor gave me a challenge to see if I could come up with a workflow for a particular problem we were having. We wanted to get Pre/ACT letters into our SMS(Student Management System), which in our case was Skyward. The problem we ran into is that Pre/ACT letters are in either a bulk PDF or per individual PDF, and to get into Skyward we would need to have a PDF for each student's name as their ID number. To accomplish this I decided to write a program in Python, using Streamlit for the UI.

Let's look at the problems we need to address, starting with the PDF. It made more sense just to grab the bulk single PDF export of the letters, this meant we needed to split up the bulk export into individual PDFs. While each letter is typically 2 pages that isn't always the case, so a simple break every other page is likely to be error-prone.

The second issue was reading each student's PDF and renaming it to the corresponding ID Number. This mostly hinged on a Regex pattern that pulled what I needed.

Since this was also a time challenge I worked with AI to help generate the code. NOTE: This is not a replacement for knowing the logic and language you are using. When writing this with AI/LLM I used the chain-of-thought approach, giving bite-sized chunks of what I wanted, and then debugging and testing each chunk before adding more. The code below is the final code that was used, I'll break each down section by section. If you're looking to implement this as a solution at your district see the TLDR are the end of this post.

Requirements and Imports

This part is fairly straightforward and is the foundation the program runs on.

Streamlit for our UI
pypdf2, pymupdf, and fitz for PDF manipulation

Content of requirements.txt

streamlit
pypdf2
fitz
pymupdf

The app.py imports

import PyPDF2
import fitz  # PyMuPDF
import re
from pathlib import Path
import concurrent.futures
import streamlit as st
import shutil
import zipfile
import os

Finding ID's

This next snippet is dealing with finding the IDs in the bulk PDF and creating a list of pages to be used to split them up, this is the part that hinges on the regex and may need to be changed for your situation.

def find_id_pages(input_pdf):
 doc = fitz.open(input_pdf)
 id_pages = []
 id_pattern = re.compile(r'\(ID#:\s*(\d+)\)')

    for i, page in enumerate(doc):
 text = page.get_text()
        if id_pattern.search(text):
 id_pages.append(i)

    return id_pages

Splitting the PDF's

As the title says, this is used to split up the PDFs. This will use a function for extracting the names for each individual PDF. You'll also notice that this splits them in parallel, up to 10 at a time, to improve performance.

def split_pdf(input_pdf, output_folder, progress_callback):
 input_path = Path(input_pdf)
 output_folder = Path(output_folder)
 output_folder.mkdir(parents=True, exist_ok=True)

    # Find pages with IDs
 id_pages = find_id_pages(input_pdf)

    if not id_pages:
 st.error("No ID pages found in the PDF.")
        return

 pdf_reader = PyPDF2.PdfReader(str(input_path))
 total_pages = len(pdf_reader.pages)
 temp_pdfs = []

    for i in range(len(id_pages)):
 start_page = id_pages[i]
 end_page = id_pages[i + 1] if i + 1 < len(id_pages) else total_pages

 pdf_writer = PyPDF2.PdfWriter()
        for j in range(start_page, end_page):
 pdf_writer.add_page(pdf_reader.pages[j])

 temp_pdf_path = output_folder / f'temp_{i}.pdf'
        with open(temp_pdf_path, 'wb') as output_pdf:
 pdf_writer.write(output_pdf)

 temp_pdfs.append(temp_pdf_path)
 progress_callback((i + 1) / len(id_pages))  # Update progress bar

    # Process renaming in parallel
    with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
 executor.map(lambda pdf_path: extract_and_rename_pdf(pdf_path, output_folder), temp_pdfs)

def extract_and_rename_pdf(pdf_path, output_folder):
 doc = fitz.open(pdf_path)
 text_first_page = doc[0].get_text()

    # Extract ID using a regex pattern for the format (ID#: 01234)
 match_first_page = re.search(r'\(ID#:\s*(\d+)\)', text_first_page)

    if match_first_page:
 id_value = match_first_page.group(1)
 new_pdf_path = output_folder / f'{id_value}.pdf'
 pdf_path.rename(new_pdf_path)
    else:
 new_pdf_path = output_folder / f'unknown_{pdf_path.stem}.pdf'
 pdf_path.rename(new_pdf_path)

Almost there

Next up are a couple of short functions, one to zip all the split PDFs (in case you want to run this on an internal server), and one to cleanup any temp files so there is no PII student information hanging around where it doesn't need to live.

def zip_output_folder(output_folder, zip_name):
 shutil.make_archive(zip_name, 'zip', output_folder)

def clean_up(output_folder, zip_name):
 shutil.rmtree(output_folder)
 os.remove(f"{zip_name}.zip")

Building the UI

The last bit of code is for the UI. Streamlit is a WebUI for versatility(yes you can run it solo). After a few attempts and considering usability. Keeping it simple I distilled it down to an upload button, an action button(ie split), and a download button to get the zipped PDFs.

# Streamlit App Portion
st.title("PDF Splitter and Renamer")

uploaded_file = st.file_uploader("Choose a PDF file", type="pdf")
output_folder = "output_folder"

if st.button("Split and Rename PDF"):
    if uploaded_file and output_folder:
        try:
            # Save uploaded file temporarily
            with open("temp_input.pdf", "wb") as f:
 f.write(uploaded_file.getbuffer())

 progress_bar = st.progress(0)
            def update_progress(progress):
 progress_bar.progress(progress)

 split_pdf("temp_input.pdf", output_folder, update_progress)

 zip_name = "output_pdfs"
 zip_output_folder(output_folder, zip_name)
 st.success("PDF split and renamed successfully!")

            with open(f"{zip_name}.zip", "rb") as f:
 st.download_button(
                    label="Download ZIP",
                    data=f,
                    file_name=f"{zip_name}.zip",
                    mime="application/zip"
 )

            # Remove temporary file
 Path("temp_input.pdf").unlink()
 clean_up(output_folder, zip_name)
        except Exception as e:
 st.error(f"An error occurred: {e}")
    else:
 st.error("Please upload a PDF file and specify an output folder.")

TLDR to get up and running

To get things up and running just use the following commands(this assumes Linux, WSL, and MacOS). and you'll be able to reach the app by going to http://localhost:8501.

git clone https://github.com/Blacknight318/act-to-sms.git
cd act-to-sms
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
streamlit run app.py

In Closing

If you're in a K12 school I hope you'll find this helpful. If so clap or consider buying me a coffee. Till next time, fair winds and following seas.

Rename macOS Devices with JAMF API Tokens

Blacknight318 — Mon, 22 Jul 2024 10:00:00 +0000

What is this tool and who is it for?

About 2 years ago the school I work for used a script they found, on JAMF Forums or StackOverflow probably, to rename static iMacs, Mac Minis, and MacBooks, through pre-stage enrollment using the serial number to lookup the barcode1 field with JAMF API. This worked slick, for a while.

Not giving it a thought we used the same script to prep machines this summer for a new lab rollout, at first it ran intermittently then not at all. Reaching out to JAMF support they sent me a couple of relevant posts on the forum showing how to get and use authentication tokens. After copying their script to request a token and modifying the script to use said token all was working again, for a bit. See the repeating theme?

Doing some more digging I found that the tokens were good for no more than 30 minutes, I'm struggling to understand why JAMF picked that time limit, by the way, to get the tokens you have to perform an API call using the username and password which is the only call you can make with them. Great! Time to add that in, which seems to defeat the intended purpose of security to me, just my opinion.

Below is the script I eventually came up with using the JAMF API documentation, scripting experience, and some help from AI to write the script. The $4, $5, and $6 variables refer to fields in the options section of your script in JAMF.

#!/bin/sh

jamfProURL=$4
user=$5
pass=$6

# request auth token
authToken=$(/usr/bin/curl --request POST --silent --url "$jamfProURL/api/v1/auth/token" --user "$user:$pass")

# parse auth token
token=$( /usr/bin/plutil \
-extract token raw - <<< "$authToken" )

# Pull serial number from system_profiler
serial=$(system_profiler SPHardwareDataType | awk '/Serial/ {print $4}')
if [ -z "$serial" ]; then
    echo "Failed to retrieve serial number"
    exit 1
fi
echo "Serial number: $serial"

# Pull computer name from Jamf Pro
response=$(curl --header "accept: text/xml" --silent --request GET --url "$jamfProURL/JSSResource/computers/serialnumber/${serial}/subset/general" --header "Authorization: Bearer $token")
if [ -z "$response" ]; then
    echo "Empty response from API"
    exit 1
fi

# Extracting the first barcode
barcode1=$(echo $response | /usr/bin/awk -F'<barcode_1>|</barcode_1>' '{print $2}')
if [ -z "$barcode1" ]; then
    echo "Failed to extract barcode_1"
    exit 1
fi
echo "Barcode 1 is: $barcode1"

# Setting computername
echo "Setting computer name..."
scutil --set ComputerName "$barcode1"
scutil --set HostName "$barcode1"
scutil --set LocalHostName "$barcode1"

echo "Computer name set successfully"

# expire auth token
/usr/bin/curl \
--header "Authorization: Bearer $token" \
--request POST \
--silent \
--url "$jamfProURL/api/v1/auth/invalidate-token"

# Perform JAMF Recon
sudo /usr/local/jamf/bin/jamf recon -verbose

exit 0

That's it, a quick script to request the auth token, pull the serial number and check it against JAMF API, parse out the barcode1 field from the return, and rename the machine. The jamf recon at the end ensures it's updated on JAMF as well.

Hopefully, you found this helpful(especially with deployments through JAMF). If so consider sharing this post and supporting the blog through Buymeacoffee, or by liking or clapping if you're reading this from one of the cross-posting sites. Till next time, fair winds and following seas.

Self-Hosting Perplexica and Ollama

Blacknight318 — Mon, 08 Jul 2024 03:57:59 +0000

Perplexica and Ollama Setup

Are you in the self-hosted camp, enjoying Ollama, and wondering when we'd have something like Perplexity AI but local, and maybe a bit more secure? I had been keeping an eye out when I came across an article on MARKTECHPOST about Perplexica. So I decided to take a crack at it. There were a few issues I encountered which we'll work around in the Perplexica setup, aside from config there was a call property that we need to address. Let's dive in.

Ollama Install and Setup

To begin with Ollama, follow these steps:

Run the installation script using

curl -fsSL https://ollama.com/install.sh | sh

Pull the latest version of Llama3 using
```
ollama pull llama3:latest
```
Pull the latest version of Nomic-Embed-Text using
```
ollama pull nomic-embed-text:latest
```
Edit the Ollama service file by running sudo systemctl edit ollama.service and adding the following lines
```
Copy Code
[Service]
Environment="OLLAMA_HOST=0.0.0.0"
```
Reload the systemd daemon using
```
sudo systemctl daemon-reload
```
Restart the Ollama service using
```
sudo systemctl restart ollama
```

Perplexica Setup

To set up Perplexica, follow these steps:

Clone the Perplexica repository using

git clone https://github.com/ItzCrazyKns/Perplexica.git

Copy the sample configuration file to a new file named config.toml
```
using cp sample.config.toml config.toml
```
Open config.toml in a text editor (such as nano) and make the following changes:
```
Change OLLAMA = http://server-ip:11434
```
Comment out the server for SEARXNG and press CTRL+X to exit and Y to save
Open ui/components/theme/Switcher.tsx in a text editor (such as nano) and make the following changes to line 10
```
const ThemeSwitcher = ({ className, size }: { className?: string; size?: number }) => {
```
Then press ctrl+x, then y to save the file
Open docker-compose.yml in a text editor (such as nano) and make the following changes

    SEARXNG_API_URL=http://server-ip:4000
    NEXT_PUBLIC_API_URL=http://server-ip:3001/api
    NEXT_PUBLIC_WS_URL=ws://server-ip:3001

Build and start the Perplexica container using
```
docker compose up -d --build
```
Access Perplexica by visiting http://server-ip:3000 in your web browser

That's it! With these steps, you should be able to set up both Perplexica and Ollama on your system. If you found this helpful please share this post, donate to my Buymeacoffee, or clap if you're reading this on Medium. Till next time fair winds and following seas!

Round Two: Enhancing the Ollama Cluster

Blacknight318 — Mon, 08 Jul 2024 03:43:01 +0000

Re-cap

Just over three weeks ago I wrote a post titled: Setting Up an Ollama + Open-WebUI Cluster, where I went over my first experiment in creating a entry barrier node for what was to become an Ollama cluster. In short, I could see the card but performance was negligible. Working on what I was previously able to accomplish it was time to bump things up.

Round Two Setup

Using the same Nvidia Quadro K620 and USB-C Power supply the only item left was the adapter and host machine. This time it was a Lenovo M700 Tiny and the ADT-Link M.2 NGFF NVMe Key M Extender Cable. Originally this was going to go into the Wifi card slot on the motherboard, what I didn't think to check was that while the NVME drive was M key the wifi card was A+E keyed, ok, let's try the NVME slot and temporary mode the drive to an external NVME-to-USB adapter. The drive booted with no problem, however NVIDIA card was not recognized at all, it did spin the fan as would be expected when signaled on, but no dice in lspci.

Adapt to M.2 A+E Key?

Sadly, after much research, any adapter to the A+E key would only support PCIe 1x, which is not worth the effort. There is a possible solution that might still work for this task, but that's another post. Keep watching for updates, and until next time, fair winds and following seas.

Recovering My Blog with Jekyll and Proxmox

Blacknight318 — Wed, 29 May 2024 15:21:22 +0000

The Server is(was) Down

Today I decided to try and update the Jekyll theme for this site, Chirpy. If you've watched the blog or gone to this blog's status page you probably noticed it was down for a few hours today. Needless to say, things didn't go as planned. It turns out that the last time I tried to update/recreate the blog site I chose the Chirpy Starter option instead of the Github Fork option, and in trying to update it the whole thing went sideways. No problem I'd just restore from the backup/snapshot in Proxmox, which also failed and destroyed the LXC, great!

After a few failed attempts to restore this zombie LXC, I decided to try spinning up and small shared instance of Ubuntu on Linode which updated at a crawl, to be fair this was a shared instance and I probably was trying to update it at peak usage time. Dedicated nodes aren't cheap, at least for a small blog like this. Eventually, after several attempts to stand up the blog again, I created a new LXC and cloned the theme for easier updates in the future(as long as I remember). This post is as much for me to document the process, should I need it again in the future, and help anyone else wanting to start one of their own. Time to roll up the sleeves and get to work.

Building the Blog

Building the LXC in Proxmox

When I first started using Proxmox there was a lot of experimentation to get things right. During that time I found an invaluable site to get things up and running quick, Helper Scripts by tteck. Let's use one of those to get the initial LXC up and running, in the shell of your Proxmox node paste in the following.

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/ubuntu.sh)"

You can follow the prompts on screen and use the defaults(you'll need to rename and adjust settings), or choose advanced and tweak things to your liking, here are the spec settings I used.

Ubuntu
Ubuntu version 24.04
Disable IPv6
2GB RRAM
2 Cores
15GB Storage

The reason for that RAM size and Cores was to allow overhead when initially building out or upgrading the theme, for day-to-day use you could halve those.

Now that that is out of the way we can continue with the rest of the setup for this LXC. We can click on our now LXC and then click on shell to get a few more things set up. Here are those commands.

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
adduser dave
usermod -aG sudo dave
usermod -aG docker dave

Now that docker is installed(we'll need this later), and permissions are set, we can start installing Jekyll.

sudo apt-get install ruby-full build-essential zlib1g-dev
echo '# Install Ruby Gems to ~/gems' >> ~/.bashrc
echo 'export GEM_HOME="$HOME/gems"' >> ~/.bashrc
echo 'export PATH="$HOME/gems/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc
gem install jekyll bundler

Setting up Chirpy

We've now got docker installed and Jekyll working, now we'll want to follow the steps for the fork method shown above. Once you've created a fork we'll want to pull that down and set up a few globals, including an access token for a password with Github.

Let's get that token, on Github do the following

Click on your profile
Click on Settings
Click on Developer Settings
Click on Personal Access Tokens
Click on Fine Grained Tokens
Click on Generate Token
Give the token a name
Click Only Select Repositories
Choose the forked repository
Click on Repository permissions
Change Commit Statuses to Read and Write
Change Contents to Read and Write
Change Pull requests to Read and Write 14 Click Generate Token
Copy the token and paste it into a temporary note (once you leave the page you won't be able to use it again)

Now we'll run the following commands to pull down our repo and set up Git. Either in the console through Proxmox, or an SSH session if you prefer, run the following.

sudo apt install git npm
git clone https://github.com/your-github-username/your-github-fork-repo
cd your-github-fork-repo
git config --global user.name "your username here"
git config --global user.password "your github token here"
git config --global user.email "github associated email here"
git config --global credential.helper store

Ok, we're almost there. It's time to initialize the theme (if you are pulling down an existing, fork with posts, to a new machine you can skip this step).

bash tools/init

Whether you are pulling down a new or existing fork run the following to pull all the Ruby Gems, if you don't you'll get error after error.

bundle

While we're at it let's change, at a minimum, these four lines in _config.yml so your blog shows as you.

title:
lang:
timezone:
tagline:
description:
url:
avatar:

We're set!

Building the posts

You can write a post saved under _posts folder, keep these two things in mind

Filename format 2024-05-28-title-here.md
Minimum Frontmatter at the top

  ---
  title: ""
  description: ""
  author: dave
  date: 2024-05-28 14:22:00 -0500
  categories: [Blogging]
  tags: [tag one, tag two]
  ---

To build the _site static site from these posts run the following command(you'll want to run this anytime you create a new post).

JEKYLL_ENV=production jekyll serve --config _config.yml

Running Nginx to serve the blog

Lastly, we're going to use a Dockerized Nginx instance to serve up our static site under_site folder. This is a simple one-liner.

docker run --name blog-server -v /home/dave/fork-folder-name/_site:/usr/share/nginx/html:ro -d nginx

We're done! you should now be able to navigate to the IP of the server with HTTP and see your site. Hopefully, this helps you get started or recreate a site if, like me, you have a server crash. Till next time, fair winds and following seas.

Time Series Analysis of Plausible Data

Blacknight318 — Wed, 22 May 2024 05:00:00 +0000

Purpose of the notebook

Early on with this blog I wanted a way to track its usage, just to see if anyone was reading it, and it needed to be GDPR compliant. Enter Plausible Analytics, it was GDPR compliant without needing the cookie or consent banners. This is great for simple metrics, especially for a small blog or site. Now, after a year of usage, there are a few features that would be nice to have. As of this writing the bounce rate is calculated by how many users navigate to another page on the site, this is something I'll be working on and sending out an updated post on that. This post will go over using the Plausible API to perform some time series analysis on data from Plausible. this will focus on visitors and pageviews. Let's dive in.

Prerequisites

To get things started let's set up a Python virtual environment and Jupyter Labs to work with our data. The following commands will do both.

mkdir plausible
cd plausible
python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install --upgrade jupyterlab ipywidgets

Libraries and Pulling Data

Now that we have a working environment we'll get the libraries installed that we'll need along with a function that pulls data from plausible into a dataframe we can use.

Libraries

Below is the code for the libraries we're going to use in this project, keep in mind that these are all the libraries for the final project.

import requests
import pandas as pd
from datetime import datetime, timedelta
import plotly.express as px
import plotly.graph_objects as go
from plotly.subplots import make_subplots
from statsmodels.tsa.seasonal import seasonal_decompose

We can install these libraries using pip with the following command.

pip install --upgrade pandas plotly statsmodels

Pulling Data from Plausible

We'll want to set a few keys/variables up for use when pulling the data.

site_id = "Your Plausible Site ID Here"
api_key = "Your Plausible API Key Here"

Now we'll want to pull data for our site from Plausible API, you'll want to go to Account Settings in Plausible and scroll down to API Keys, then generate a key to work with, we'll also want the Site ID which is the domain listed in the site settings. Ok, let's build this function.

# Function to get Plausible Analytics timeseries data
def get_plausible_timeseries_data():
    # Calculate the date range for the last 90 days
    date_to = datetime.today().strftime('%Y-%m-%d')
    date_from = (datetime.today() - timedelta(days=90)).strftime('%Y-%m-%d')

    # Setting the metrics we want to look at
    metrics='visitors,pageviews'

    # Actually pulling the data we want
    url = f"https://plausible.io/api/v1/stats/timeseries?site_id={site_id}&period=custom&date={date_from},{date_to}&metrics={metrics}"
    headers = {
        "Authorization": f"Bearer {api_key}"
    }
    response = requests.get(url, headers=headers)
    data = response.json()

    # Putting the data into a dataframe we can use for analysis
    results = data['results']
    df = pd.DataFrame(results)

    # Adjusting the date field so we can avoid future warnings and be more accurate
    df['date'] = pd.to_datetime(df['date'])

    return df

Data Analysis

Now that we have the data let's perform some analysis, looking for patterns/seasonality, and graph it out for easier consumption.

Seasonal Decomposition

While building this tool I had considered using code from an earlier project that leveraged Prophet and/or Neuralprohet engines, but after putting the data through 2 different AIs (ChatGPT 4o and Gemini 1.5) to get a quick analysis, I ended up going with the same process that ChatGPT 4o used, seasonal decomposition. There was a bit of tweaking to do to the original code to get it to work in the environment we're going to use, and changing the plots to Plotly from Matplotlib for more interactive data. Here's the end result code for performing the analysis.

# Function to perform seasonal decomposition
def plot_seasonal_decomposition(df, column, period=7):
    decomposition = seasonal_decompose(df[column], model='additive', period=period)

    fig = make_subplots(rows=4, cols=1, shared_xaxes=True, subplot_titles=('Observed', 'Trend', 'Seasonal', 'Residual'))

    fig.add_trace(go.Scatter(x=decomposition.observed.index, y=decomposition.observed, name='Observed'), row=1, col=1)
    fig.add_trace(go.Scatter(x=decomposition.trend.index, y=decomposition.trend, name='Trend'), row=2, col=1)
    fig.add_trace(go.Scatter(x=decomposition.seasonal.index, y=decomposition.seasonal, name='Seasonal'), row=3, col=1)
    fig.add_trace(go.Scatter(x=decomposition.resid.index, y=decomposition.resid, name='Residual'), row=4, col=1)

    fig.update_layout(
        height=800, 
        title_text=f'{column.capitalize()} - Seasonal Decomposition',
        template='plotly_dark'
    )
    fig.show()

Plotting the data

Here's the Plotly code used, besides being interactive I was able to use a dark theme that doesn't hurt the eyes as much, which also took some tweaking, here's that code.

# Function to plot day of the week trends
def plot_day_of_week_trends(df):
    df = df.reset_index() 
    df['day_of_week'] = df['date'].dt.day_name()
    day_of_week_stats = df.groupby('day_of_week').mean(numeric_only=True)[['pageviews', 'visitors']]
    days_order = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday']
    day_of_week_stats = day_of_week_stats.reindex(days_order)

    fig = go.Figure()
    fig.add_trace(go.Scatter(x=day_of_week_stats.index, y=day_of_week_stats['pageviews'], mode='lines+markers', name='Pageviews'))
    fig.add_trace(go.Scatter(x=day_of_week_stats.index, y=day_of_week_stats['visitors'], mode='lines+markers', name='Visitors'))

    fig.update_layout(
        title='Average Pageviews and Visitors by Day of the Week', 
        xaxis_title='Day of the Week', 
        yaxis_title='Count',
        template='plotly_dark'
    )
    fig.show()

Main Code and Running

Now we'll create a main function, this isn't strictly necessary but it's good practice especially if you might be using it as a class in the future. Let's take a look.

# Main function to load data and perform analysis
def main():
    df = get_plausible_timeseries_data()
    df.set_index('date', inplace=True)

    # Seasonal decomposition for pageviews and visitors
    plot_seasonal_decomposition(df, 'pageviews')
    plot_seasonal_decomposition(df, 'visitors')

    # Day of week trends
    plot_day_of_week_trends(df)

We then run it with the following code snippet.

if __name__ == "__main__":
    main()

It Lives

Once you have the code in your notebook be sure that you run all the cells(if you have them split up) and you should get several graphs.

Pageviews - Seasonal Decomposition
Visitors - Seasonal Decomposition
Average Pageviews and Visitors by Day of the Week

That last graph has been the most useful so far, giving me metrics by day of the week to find the best time to release new posts. In a future post, I hope to look into the script Plausible uses to collect metrics and see if we can add a check the timer and drop any that are on the page for more than 10 seconds from the bounce rate numbers(this is how Google Analytics and Posthog calculate bounce rate). That's a topic for another time. Till next time, fair winds and following seas.

Setting Up an Ollama + Open-WebUI Cluster

Blacknight318 — Thu, 16 May 2024 00:00:00 +0000

Why?

Having set up an Ollama + Open-WebUI machine in a previous post I started digging into all the customizations Open-WebUI could do, and amongst those was the ability to add multiple Ollama server nodes. This got me thinking about setting up multiple Ollama, and eventually Open-WebUI, nodes to load and share the work and make an internal cloud or cluster of sorts.

Before we build a cluster we first need a stable node(server/instance). We'll start by creating a BoM(Bill of Materials) to test. Here's my starter list(NOTE: This is not a shopping list, you'll see why in a moment).

Experimenting

The first thing I did was install Ubuntu 22.04 LTS on an external drive for testing, in the future this should be an onboard drive(SATA/NVME). This was fairly straightforward. After installing all updates I installed Ollama and OpenWebUI, see my post on setting that up here. After installing and testing we now had a base to start from. Getting the GPU to work was a little less straightforward.

Powering this card was not going to work with the provided SATA to 6 Pin adapter, this was because the laptop drives these Tiny's use only outputs 3v and 5v out, this is where that USB C PD supply comes in. This part is relatively straightforward, cut the SATA port end off, tie the yellow to positive and black to negative, NOTE: before plugging anything into the PCIe adapter plug in the USB C PD supply and use the button to select 12v, it will remember this going forward.

Fitting the Mini PCIe end into the Lenovo Tiny required cutting back the plastic strain relief on the cable, and snapping off the extension tabs on the PCB(perforation dots are on the bottom indicating where to break/cut)

Now that the card is connected, it was time to install the drivers. You can do this through the driver download page from Nvidia or using Ubuntu's method, be sure to use the 550 drivers. After I would recommend re-installing Ollama to ensure it sees the Nvidia Card. If you run into issues with it seeing the card check cables and connectors, make sure the power for the card is on, and run the following to see if the card is listed.

sudo lspci | grep nvidia

Results?

While I was able to see the Nvidia card in lspci and Ollama, and it was working through the GPU, it was as slow, if not slightly slower, than CPU-only mode. This is likely because there aren't enough PCIe lanes for this to make a meaningful benefit. That said, I still learned that we can adapt the card, power it externally, and see it in Ubuntu. I also learned that putting this card in an HP ProDesk 600 G2 SFF would not boot, this is likely due to not enough power from the built-in power supply.

Coming up in this series I'll be testing more hardware and looking for used hardware that gets us full use of our GPU and still be lite, power efficient, and still be scalable. Till next time, fair winds and following seas.