DEV Community: Luke Liukonen

Reclaiming Developer Velocity: The 5-Layer AI Optimization Stack

Luke Liukonen — Fri, 13 Nov 2026 00:24:40 +0000

Note: I've recently had the opportunity to present on this topic twice over the last two weeks—one talk diving deep into the loop engineering work and agentic iterations, and another covering the full 5-layer optimization stack. While the exact slides, internal metrics, and specific implementation details remain proprietary, I wanted to share the broader architectural framework and general best practices behind the core ideas.

We have all been there: you type a prompt, wait for an LLM to spin its wheels, and watch it confidently generate five paragraphs of polite narrative alongside a completely misaligned solution. Or worse, you spin up an autonomous agent loop to tackle a refactoring task, only to realize it has burned through a massive chunk of your token budget because it spent thirty iterations reading and rereading multi-megabyte log files.

The industry-wide knee-jerk reaction to these failures is usually to write longer prompts, dump more files into the window, or throw a larger, more expensive model at the problem. But if we look at AI integration through a systems engineering lens, these symptoms point to a structural flaw: we are treating prompt engineering as a silver bullet while ignoring the actual architecture of how we feed, house, and govern our models.

First, Respect the Token Mechanics

Before optimizing the stack, it helps to treat tokens as an engineering constraint rather than just an API billing line item. Token volume directly dictates inference latency, attention degradation, and hallucination rates.

More importantly, tokenization isn't spelling. LLMs don't read characters; they process numerical IDs mapped to statistical subword chunks. Because of how tokenizers parse text:

Code is denser than prose: Uncommon class names, UUIDs, leading whitespace, and syntax punctuation split into far more chunks than standard English sentences.
Logs are silent budget killers: Pasting identical multi-page stack traces repeatedly across turns burns context capacity exponentially.
Measure, don't guess: Every LLM interaction ultimately boils down to a fundamental REST contract. Inspecting the actual input/output token counts (prompt_eval_count and eval_count) and execution duration against your target production tokenizer is the only way to baseline real performance.

To move from guesswork to predictable, production-ready AI workflows, we need to organize our approach around a 5-layer optimization stack: Prompt, Context, Harness, Loop, and Meta.

Layer 1: Prompt Engineering (The Assembly Layer)

Prompt engineering sits at the absolute bottom of the optimization stack—not because it is the most important, but because it is the closest to "assembly language".

Just as writing raw assembly code gets you closest to the bare-metal CPU instructions—giving you maximum direct control over every register and byte, but requiring immense manual effort for anything complex—writing prompts puts you right at the metal of the model's token weights. Treating prompt engineering as the bottom layer means recognizing that trying to solve systemic engineering failures by writing longer, more complex assembly prompts is like trying to write an entire modern application in raw assembly. Direct control over your instructions matters, but treating prompts like magic incantations leads to bloated text and erratic behavior.

Drop the Persona Hype: Telling a model it is a "10x principal staff engineer" wastes tokens and adds zero deterministic value. Boring, explicit prompt contracts—defining the exact job, boundary fence, allowed modifications, and output shape—outperform elaborate persona prompts every single time.
Adopt "Caveman" Efficiency: Strip away conversational padding. Compare a polite, 35-token request ("Could you please look at the code I selected and refactor it to remove duplication...") against a direct, 24-token contract:

Refactor selected code. Remove duplication. Preserve behavior. 
Modify selected code only. Return code only. Minimal diff.

Saving 10 input tokens is nice, but preventing the model from generating 500 tokens of conversational narration around the code block is the real latency and cost win.

Match Models to the Task: Avoid extremes. Don't route simple JSON reformatting or log classification to a massive reasoning model, and don't force a tiny 3B local model to untangle complex multi-file architecture if it takes four failed retries to get right. Optimize for the cheapest completed job, not the cheapest individual token.

Layer 2: Context Engineering (Controlling the View)

One does not simply dump an entire repository into a prompt. Passing raw, unconstrained codebases creates signal-to-noise degradation, increases latency, and causes models to lose track of critical constraints buried in the middle of the window.

Understand What IDE Tools Actually Do: Modern editor integrations use workspace indexing, semantic vector search, inline grounding (open files and cursor selections), and diff awareness to surface relevant code. However, baked-in tools optimize context; they do not optimize intent. If your acceptance criteria are fuzzy, better semantic retrieval won't save the output.
Expand on Demand: Never start by feeding the model the whole project tree. Start tightly scoped: pass the selected code block, the exact failure log, the current git diff, and explicit acceptance criteria. Only pull in adjacent interfaces or upstream callers if the first pass lacks sufficient type or contract awareness.

Layer 3: Harness Engineering (Environmental Guardrails)

Harnessing is engineering the environment around the model so it doesn't start from zero context on every single execution. A harness doesn't require a heavyweight agent framework; it can be as straightforward as a structured instruction file, a local CLI validation script, and a strict JSON schema.

A strong harness relies on four pillars:

Prepopulate: Surface known facts—such as repository rules, architectural boundaries, and service connection maps—before the model has to ask or guess.
Structure: Enforce deterministic input and output schemas so downstream scripts can parse model responses reliably without fragile regex gymnastics.
Memory: Persist prior architectural decisions and known constraints so the model doesn't re-litigate settled patterns.
Tools: Give the model scoped, executable commands to inspect state, run linters, and validate its own work.

Think about cross-service architecture discovery: instead of letting an LLM burn tokens spelunking through dozens of repositories every time someone needs to trace message queues, APIs, or database connections, you build a lightweight scanner that extracts that dependency metadata into a clean, structured schema once. You pay the discovery cost programmatically once, and every subsequent AI interaction benefits from deterministic context.

Layer 4: Loop Engineering (Managing Iteration Safely)

Loop engineering takes those harness capabilities and allows an LLM to iterate autonomously over a task—measuring state, acting, verifying results, and repeating.

However, the most dangerous word in loop engineering is another. Just because an agent can keep iterating doesn't mean it should. Without strict architectural constraints, agentic loops easily enter a death spiral: a test fails, an orchestrator spawns a sub-agent to fix it, the sub-agent runs the entire test suite, hits the exact same failure, and spawns more workers that all pay to re-read identical multi-megabyte log files. Optimization becomes amplification when you pay AI to reread noise.

If you are building automated test-generation or refactoring loops, enforce four architectural guardrails:

Single Process Execution: A central orchestrator controller must own execution state. Sub-agents should generate code or patches, never trigger full environment runs independently.
Smallest Possible Verification Scope: During the inner loop, execute only the specific unit test file or module that was just modified. Reserve full test-suite runs for the final verification gate.
Useful Log Tails: Never pipe raw stdout/stderr streams directly back into the prompt. Write the full log to a flat file on disk, and pass only the exit status, coverage delta, and the last 10 relevant lines of the error stack to the model.
Deterministic Exit Code Routing: Use strict exit codes (0 = step passed, proceed; non-zero = parse error fingerprint and route to fix, retry, or abort) so control flow lives in deterministic code, not LLM guesswork.

Layer 5: Meta-Prompt Engineering (Governance and Control)

Meta-prompting sits at the top of the stack as a control and governance layer that optimizes how a task gets solved before heavy execution even begins.

Pre-Optimize Intent: Before routing a complex request to an expensive frontier model, use a lightweight classifier step to inspect the prompt, assess task complexity (low, medium, high), enforce strict output constraints, and route to the appropriate model tier with a capped token budget. Asking "What is the cheapest, fastest, most reliable path to a verified result?" upfront routinely cuts downstream token waste in half.
Enforce Repository-Wide Defaults: Relying on individual developers to remember prompt hygiene every time doesn't scale. Commit persistent rule files directly into your repositories to standardize task boundaries, coding conventions, and concise, filler-free output rules across the entire team automatically.
Govern Loop State: At the meta level, track failure fingerprints across iterations. If an agent produces the same error hash twice, kill the loop or escalate rather than letting it burn retries. Checkpoint valid states, reset stale context windows, and pass the git diff—never the entire universe.

Takeaway

Tokens equal control, but fewer tokens is not the ultimate goal—less useless work is the goal.

When an AI workflow feels slow, expensive, or unreliable, resist the urge to write a longer prompt. Look up the stack instead, find where you are forcing the model to repeatedly parse noise or rediscover things your deterministic tooling already knows, and cut that work out of the loop entirely.

Logging Frameworks... Moving from SEQ to f/Lo/G... and why I didn't pick ELK

Luke Liukonen — Sun, 21 Apr 2024 01:24:15 +0000

First, anyone with a "FLOG" app... not trying to take the name away or claim it, but I do think Flog is a neat little way of explaining the logging stack... similar to ELK... (FluentBit / Loki / Grafana)

the why

With the latest flare-ups of companies doing a "rug pull" on the licensing schemas, locking down functionality, and outright abandoning the people who put them in the spots they are in (looking at you, Redis)... I felt it was time to start migrating any proprietary solutions over to more free and open-source solutions. I'm not in any way saying DataLust is doing this with their SEQ product. I have used this for years as a log aggregator, and it has worked very well. I did have a few hiccups a bit back over looking at logs on my computer and phone at the same time due to limitations of the software, but overall, it has done what I wanted it to do, which is a very lightweight logging system that just works out of the box. Evaluating the options, though, was rough. The industry standard is ELK (or an OpenSearch variation), is the way to go. Having Elestisearch as your main engine, Logstash as the aggregator, and Kilbana as your main UI felt very heavy to me in terms of running something that does one thing... or in my mind does one thing. Also.. is it just me, or do setting up these solutions feel like you need a PHD in Logging to get them to work? So I did what any developer would do... google something for 5 minutes, give up, and asked ChatGPT what it would do. A lot of the recommendations it gave Im sure, are great, but for the full suite of things to run on something like a Raspberry Pi, most of my research pointed to Grafana Loki as the main product to use.

I've used Loki before, and it was OK, but Grafana was kind of overkill. I originally started with Grafana / Prometheus / Agents to monitor my systems. This solution is great, but very, very overkill for what I have, which is a basic home network with a handful of services I host. I prefer and still use Uptime Kuma as my main monitoring solution, but when using Loki, I was overwhelmed. Not so much in the data, but in the amount of data. I was using Promtail as my collector, and I felt the system was very, very slow... especially with the amount of data I collected. See, I was collecting all of my Docker container logs, and some of my home lab services are very chatty to the console log. That said, I figured I'd give it a fresh start and use something other than promtail. To me, the concept of something like Promtail that would either have to read the data live or query it every so many minutes felt like a waste. When it comes to my applications, I want live data coming into one and only one location, and limited data.

I could have used Lokis rest API for my solution. I hate adding complexity to a project, but I do love performance. For me, FluentBit felt like a good solution for this. I can configure my endpoints and how I want to connect to them, which I did. I picked a TCP connection since it is a level down in the network stack from http, and doesn't need a handshake or the overhead of establishing a connection to transmit the data. Meaning my log event is as light as possible. It also gave me a similar trio of services as the ELK platform, each serving the same use case. The 3rd reason I went with FluentBit was if I ever needed to switch over to ELK or Opensearch, I could. FluentBit is a great, quick middleware platform and is recommended for people who don't want to use Logstash in their ELK platform.

Final thought on this... In terms of resources used, looking them up on my Portainer instance, it appears my "Flog" stack uses just a few more resources than my SEQ instance. This is good since I was worried that by cutting over to a multi-app stack, somehow I'd be pushing the hardware limits on my Raspberry Pi.

the code

The cutover. Im at the time doing a proof of concept more than anything, but I found that Serilog for DotNet was probably the best bet in cutting over. My ASP.NET services already use the Microsoft Logging platform, and my original service was calling in the service.cs a simple

services.AddLogging(loggingBuilder =>
 { loggingBuilder.AddSeq(Configuration.GetSection(SEQ));

With Serilog, it took a bit, but I was able to get it fairly close to the same code. I needed to go to Nuget and install the serilog.aspnetcore and serilog.sinks.network packages, and really, really tried using the serilog.configuration package to make the imports easy, but settled without the automatic configuration bindings. This left me with

 services.AddLogging(loggingBuilder =>
 {
    var logger = new LoggerConfiguration()
      .Enrich.WithProperty("App", name)
      .MinimumLevel.Debug()
      .WriteTo.Console()
      .WriteTo.TCPSink(loggerConnection, new CompactJsonFormatter(), Serilog.Events.LogEventLevel.Warning)
      .CreateLogger();
    loggingBuilder.AddSerilog(logger);
});

Not as clean as Seq importing, but to be fair, not too bad either. I can reduce it by not writing to the console if I don't want to, and I am adding an extra app attribute to the JSON structure so that I can tell what app is causing the error message.

Now to the server-side stuff.
I am a huge fan of Docker Compose files. This stack is no exception. for it, the setup is as follows (note: anything with an _ is a folder)

.
├── docker-compose.yml
├── fluent-bit.conf
├── loki-config.yaml
├── _fluentbit
│ └── _logs
├── _loki
│ ├── _loki
│ └── _wal
└── _grafana

I prefer having my configs under the same directory as my root, but each their own. Folders are needed for storing the live data. Im not a fan of volumes in the Docker sense, and just mount to the folder that exists in the project.

The first step in the process is fluent. The config I have set is

[INPUT]
    name http
    listen 0.0.0.0
    port 24224   

[INPUT]
    Name            tcp
    Listen          0.0.0.0
    Port            12201

[OUTPUT]
    Name        grafana-loki
    Match       *
    Url         http://loki:3100/loki/api/v1/push
    RemoveKeys  source,container_id
    Labels      {job="fluent-bit"}
    LabelKeys   container_name
    BatchWait   1s
    BatchSize   1001024
    LineFormat  json
    LogLevel    info

I give 2 endpoints, on 2 different ports (12201, 24224)
Input 1 is an HTTP endpoint that I can make post-calls to. This will come in handy for basic bash shell scripts I have, and logging their items. The second input is the TCP input. This one I will use for any application-based logging I do. The output is configured specifically for Loki, and the log levels are set to info. While I could have gone higher, I think having that limitation in the application fits better than in the logger (at least for my scenario, which is hosted 100% internally and has a lesser likelihood of being breached).

From FluentBit, we go to the Loki Configuration

auth_enabled: false
server:
  http_listen_port: 3100
  grpc_listen_port: 9095

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 15m
  max_chunk_age: 1h
  chunk_target_size: 1048576
  chunk_retain_period: 30s


schema_config:
  configs:
    - from: 2020-02-25
      store: boltdb
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

storage_config:
  boltdb:
    directory: /var/lib/loki/index

  filesystem:
    directory: /var/lib/loki/chunks

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: true
  retention_period: 30d

compactor:
  working_directory: /var/lib/loki/boltdb-shipper
  shared_store: filesystem

ruler:
  storage:
    type: local
    local:
      directory: /var/lib/loki/rules
  rule_path: /var/lib/loki/rules-temp

Im sure there is some cleanup I can do with this. For one, Im not using grpc for any of my logging, so I don't think I need this (could be wrong though) My goal for this though is to keep data for about a month before throwing it away (which means persistence with the Loki instance) I've yet to run this any longer then a few hours, so there might end up being some corrections to these files.

Third and last is the compose file. I could have set up Loki right off the bat in the compose, or something like that, but it takes a few seconds to add Loki (HTTP://loki:3100) to the sources in Grafana.

version: '3'

services:
  grafana:
    image: grafana/grafana:latest
    ports:
      - "3003:3000"
    volumes:
      - ./grafana:/var/lib/grafana
    networks:
      - logging_network

  loki:
    image: grafana/loki:latest
    command: -config.file=/etc/loki/local-config.yaml
    ports:
      - "3100:3100"
    volumes:
      - ./loki/loki:/var/lib/loki/
      - ./loki-config.yaml:/etc/loki/local-config.yaml
      - ./loki/wal:/wal
    networks:
      - logging_network

  fluent-bit:
    image: grafana/fluent-bit-plugin-loki:latest
    #image: fluent/fluent-bit:latest
    container_name: fluent-bit
    environment:
      - LOKI_URL=http://loki:3100/loki/api/v1/push
      - LOG_PATH=/var/log/*.log=value
    ports:
      - "24224:24224"
      - "12201:12201"
    volumes:
      - ./fluentbit/log:/var/log
      - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf
    networks:
      - logging_network

networks:
  logging_network:

Im currently using the Grafana/fluent-bit-plugin container, which only runs on x64 at the moment. I do plan on trying to cut this over to the official Fluentbit container, which supports ARM-based CPUS.

As with any of my posts... if you see any improvements I can make or ways to clean things up, let me know. Im still in the middle of my project, but at a point right now where publishing a quick little how-to / why article felt ok to do.

Building Microsofts contender for a Redis (Garnet) on the Raspberry Pi

Luke Liukonen — Mon, 25 Mar 2024 03:18:05 +0000

When finding out about Redis going from a BSD to a dual license, I felt a bit betrayed. I was still a holdout of using Redis locally and became a huge fan of its speed and support. While there were competitors on the market, such as KeyDB and DragonFly DB, Key had no native Pi abilities that I could tell, and DragonFly also had a Business License over an open-source license. I used Redis because not only was it the top name in a quick in-memory database, but because it was open source, meaning there was a community of engineers and businesses willing to support the product.

I just, like 2 hours ago, found out about Microsofts recent open source offering, and had to give it a go. With a larger name backing the product, and being more of a "dot net" shop in my house, I wanted to see what this product could do, and see if it was a decent drop-in replacement for my Redis server, which I had just ported to my Raspberry Pi 5

I don't see myself playing with this too much tonight, I did want to post at least my Dockerfile that I used to get this running

I am curious though, and could use some help... I don't know exactly where the files are stored for the Tsavorite DB, as I'd like to keep that on a volume for rebuilds and upgrades... I'm sure it's buried somewhere in the documentation.

So... without further ado... Here is the dockerfile I used... it is slightly tweaked from what Microsoft has in the repo but doesn't require a full download of the git repo to build. I also specifically targeted the arm64-based image instead of the x64. I also am using a chiseled version of the runtime, to help reduce surface attack area, and have a smaller overall image. The final image output appears to be 101 MB in size, which isn't too bad, but still more than double that of the official 7.2.4-alpine image (41.6 MB) I'm using from Redis

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /source

# Copy files
RUN curl -L https://github.com/microsoft/garnet/archive/refs/tags/v1.0.1.tar.gz -o garnet.tar.gz \
    && tar -xzf garnet.tar.gz --strip-components=1 \
    && rm garnet.tar.gz

RUN dotnet restore
RUN dotnet build -c Release

# Copy and publish app and libraries
WORKDIR /source/main/GarnetServer
RUN dotnet publish -c Release -o /app -r linux-arm64 --self-contained false -f net8.0

# Final stage/image
# FROM mcr.microsoft.com/dotnet/runtime:8.0
FROM mcr.microsoft.com/dotnet/runtime:8.0-jammy-chiseled
WORKDIR /app
COPY --from=build /app .

# Run GarnetServer with an index size of 128MB
ENTRYPOINT ["/app/GarnetServer", "-i", "128m"]

For a Docker-compose file, I went with something simple

version: '3.3'
services:
  garnet:
    restart: always
    build:
      context: .
    image: garnet
    container_name: garnet
    ports:
      - "6379:3278"
    volumes:
      - /etc/localtime:/etc/localtime:ro

I map the port of entry for Garnet to the same port Redis uses. Let me know if this helps you, or if you have any recommendations!

Side... as I play with this more, I'll be sure to update this post with my findings. My current use cases are using the Pub/Sub functionality, as well as a simple key-value memory cache for my services.

updates 2023-04-05

So.. after getting some info back from github, I was able to get my docker instance to persist. so I have some updates to my files... for starters, I migrated my configs out of the entry point, and into a file. This way, I could keep persistence on the data I have. There is some cleanup work or optimizations I'd like to do, but I might reserve that for when I try out ValKey... the Linux Foundations version of Redis they are developing.

Here is what my dockerfile now looks like.. if you're running this on an Intel or AMD based machine, you'll want to drop the -r linux-arm64 from the publish command

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /source

RUN curl -L https://github.com/microsoft/garnet/archive/refs/tags/v1.0.2.tar.gz -o garnet.tar.gz \
    && tar -xzf garnet.tar.gz --strip-components=1 \
    && rm garnet.tar.gz

RUN dotnet restore
RUN dotnet build -c Release

# Copy and publish app and libraries
WORKDIR /source/main/GarnetServer
RUN dotnet publish -c Release -o /app -r linux-arm64 --self-contained false -f net8.0

# Final stage/image
FROM mcr.microsoft.com/dotnet/runtime:8.0
WORKDIR /app
COPY --from=build /app .
COPY garnet.config /app/garnet.conf
VOLUME /data
RUN mkdir -p /data/checkpoint /data/logs
ENTRYPOINT ["/app/GarnetServer", "-i", "128m", "--config-import-path", "/app/garnet.conf"]

my docker-compose.yml has also slightly changed

version: '3.3'
services:
  garnet:
    restart: always
    build:
      context: .
    image: garnet
    container_name: garnetpersist
    ports:
      - "6399:3278"
    volumes:
      - ./data:/data

as noted below, I dropped the time syncing between the container and system time in this container, however, I do have a data directory in the same directory as the volume... I map my folders in docker, but if you want to use proper "volumes", it should be a hard change to invoke.

For my config, to save persistence, I am using the following config file, which is pulled into my image at the time of compile. It wouldn't also be that hard to map, but in my simple case, I just pull it in from the dockerfile build script

garnet.config

{
  "MemorySize": "128m",
  "Port" : 3278,
  "Address": "0.0.0.0",
  "CheckpointDir": "/data/checkpoint",
  "EnableAOF": true,
  "Recover": true
}

It was also recommended to me that I call SAVE or BGSAVE then data will be checkpointed... this way I'm not running every write operation to the system through AOF. I have to thank Badrish Chandramouli and kingcomxu on Github for the recommendations! here is a link to our discussion of it on Github

Unleashing the Power of Developer AI: A Journey into Hosting a Private LLM/Code Assistant locally

Luke Liukonen — Fri, 22 Dec 2023 01:53:05 +0000

Introduction

In the ever-evolving landscape of software development, the quest for efficient coding tools led me to Twinny, a VSCode extension promising to bring GitHub Copilot-like capabilities to the local environment. Eager to play around with something that promises copilot-like capabilities without the cost, I set out to host this private GitHub Copilot alternative on my machines. I'll take you through the highs and lows, the challenges faced, and the eventual win on this fascinating journey.

As a note. I am not affiliated with the twinny project in any way.

Discovery

When I first encountered Twinny, the prospect of having GitHub Copilot's insights, right within the confines of my local development environment, was, well, a bit mind-blowing. The promise of streamlined coding, tailored to my needs, I had to see extension had to offer.

The Hardware Landscape: Two Machines, Two Stories

Machine 1 - Framework Laptop

Processor: 11th Gen Intel Core i7-1165G7 @ 2.80GHz
Memory: 32GB RAM
Storage: 1TB NVMe SSD
Graphics: Onboard Graphics
OS: Windows 11
Containerization: Rancher Desktop over Docker

Machine 2 - Custom-built Desktop

Processor: AMD Ryzen 5 5600X 6-Core
Memory: 32GB RAM
Storage: 1TB SSD
Graphics: NVIDIA GeForce RTX 3060 TI
OS: Windows 11
Containerization: Docker Desktop

The Pros and Cons: Navigating the Local Deployment Terrain

Advantages of Local Deployment:

Cost Efficiency: Running GitHub Copilot-like capabilities without incurring cloud costs was a game-changer.
Privacy Control: Keeping code and data on-premises provided an added layer of security.
Customization: While I'm stuck using Ollama as my backend/host for LLMs, there are more LLMs available than 1.

Disadvantages of Local Deployment:

Hardware Requirements: The demand for powerful GPUs for optimal performance proved to be a consideration.
Setup Complexity: While not too bad, having to have an LLM running all the time and the complexity of a client/server model is a bit more than just installing a plugin
Maintenance Responsibility: Regular updates and maintenance became my responsibility, adding a layer of ownership.

Machine 1: No luck

While Ollama and the LLM were installed and running, when calling the API using Postman, it would just spin. Calling directly from within the terminal appeared to work, but was really, really slow. In my IDE, I would just see a spinning wheel where my twinny icon should appear. My CPU would spike, and that's it.

A Glimpse Into Success on Machine 2

Not surprisingly, it was on my gaming PC, a custom-built powerhouse with an AMD Ryzen processor and NVIDIA GeForce RTX 3060 TI, where Twinny truly came to life. The performance boost, notably attributed to GPU acceleration, turned what initially seemed like a challenge into a success story.

Key Observations:

GPU acceleration played a pivotal role in Twinny's optimal performance.
Comparisons of Docker Desktop configurations between machines shed light on factors influencing responsiveness.

Installation Instructions: (Windows)

get the backend running
- Have a container-based system or WSL. I prefer Containers since they are easy to spin up or down
- as you saw above, I have Rancher Desktop and Docker desktop.
- Run the following command to download and start up your container... the gpus=all is needed to access my NVidia GPU. My Framework laptop however did not have the same luck. As a side, for my Framework laptop, I did try both with and without this flag.

  docker run --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollamagpu ollama/ollama

Run the following command to install the LLM needed for Twinny

docker exec -it ollama ollama run codellama:7b-code

I validated things "should work" by running the following curl command

  curl --location 'http://localhost:11434/api/chat' \
--header 'Content-Type: application/json' \
--data '{
  "model": "codellama:7b-code",
  "messages": [
    { "role": "user", "content": "why is the sky blue?" }
  ]
}'

Install the extension link

From there in the bottom right-hand corner, you should see an extension for Twinny. As you start typing, hopefully, you'll see IntelliSense kick in with automatic code-generation recommendations.

Conclusion: Unveiling the Potential

In conclusion, self-hosting a private "GitHub Copilot"-like tool locally with Twinny has been a bit of a journey with some definite learnings. The challenges faced on the Framework Laptop were met with at least a win on the gaming PC, showcasing the potential benefits and potential alternatives to the bigger players in the game. I'm excited about this proof of concept, and I like the idea that soon organizations and individuals will have the capability to run code assistants from within their organization and reduce the fear of 3rd parties getting a hold of their code.

Automating a Backup Server and Gaming PC with Wake on Lan, PowerShell, Bash Shell Scripts, and a Raspberry Pi

Luke Liukonen — Wed, 06 Sep 2023 02:47:44 +0000

Introduction:
Efficiency has always fascinated me as a software engineer. I'm constantly seeking ways to optimize existing technologies. In 2020, I built a gaming PC equipped with a mid-tier Zen 3 CPU, 16 GB of RAM, and an Nvidia 3060. However, it often took a back seat to my Framework Laptop due to its compatibility with my kitchen workstation setup, and the ease of hot-swapping using USB-C with my work laptop, an M1 Apple MacBook.

With a decent setup in my kitchen area, because of the whole, work from home privilege many of us have seen over the last few years, my gaming PC seldom saw use. When it did, it was usually for video transcoding, light gaming, or overdue Windows updates. Meanwhile, my home lab, running on Ubuntu Server, saw substantial power consumption reduction over the years, thanks to moving from a AMD 5350 to an Intel J1900 CPU/motherboard combo and Raspberry Pi devices.

I knew about Wake on LAN but never explored it until I had two idle PCs: the old home lab and Windows Gaming PC.

Problems to Solve:
Going through the above, I had three problems I wanted to solve:

How to keep my backup Linux PC in sync with my home lab so if my machine ever went down, cutover would be nearly seamless.
How to keep my Windows PC up to date with software updates and apps.
How to automate video transcoding seamlessly.

Solution:
This led me to utilize the "efficiency cores" of a Raspberry Pi 3 for automation. My goals, driven by cron jobs, were clear:

Every day at 6:30 am, check for videos to encode on my network share. If found, wake up the gaming PC, run the encoding script, and shut down when finished.
Once a week, wake up my backup Linux server, perform a backup, and shut down.
Once a week wake up the gaming PC, execute system updates, and shut down.

Wake on LAN played a crucial role in achieving these goals, offering substantial energy savings. (just the AMD server alone, Im looking at just over $42 a year by not running it 24/7)

Script Description:

This script serves as the core component of our automation process. Its primary objective is to initiate the wake-up and execution sequence for a target machine, such as a Windows PC, using a Raspberry Pi. Here's how it works:

Setting the Target: We begin by specifying the target machine's IP address, which allows us to precisely identify the system we want to awaken and interact with.

Checking the Samba Share: Our script first verifies the contents of a designated Samba share folder. This step is critical for determining whether there are any pending tasks or files to process. If the folder contains subfolders or files, it signals the need for the target machine's activation.

Waking Up the Target: To wake up the target machine, we employ the Wake-on-LAN (WoL) mechanism. This involves sending a "magic packet" to the machine, causing it to power up from a sleep or powered-off state. The unique MAC address of the target machine is used to identify it within the local network.

Ensuring Connectivity: Once the target machine is awakened, we establish a connection check loop. We repeatedly ping the machine to confirm its availability. This process continues for a predefined duration, ensuring that we don't proceed until a successful connection is established.

Executing Tasks Remotely: Upon successful connection, we leverage SSH to establish a secure connection to the target machine. From there, we execute a PowerShell script (typically used for tasks like video encoding). You can customize this step by replacing 'mainuser' with the correct username and '~\encodevideo.ps1' with the actual script path you want to run.

Exit and Reporting: If the connection and task execution are successful, we exit the script gracefully. In the event of a failed connection within the designated time frame, the script provides a notification that the connection attempt has failed.

This script streamlines the process of waking up and automating tasks on a target machine remotely. It is particularly useful for scenarios where you need to conserve power by keeping the target machine in a low-power state until specific tasks require execution. To use this script effectively, ensure you've installed the 'wakeonlan' app and 'sambaclient' on your Raspberry Pi.

Feel free to customize the script to suit your specific requirements and automate a wide range of tasks on your target machine.

#!/bin/bash

# Set the target IP address of the machine you want to wake up.
target_ip="127.0.0.1"

# Define the Samba share you want to check for files.
samba_share="//NAS/hdd/tmp"

# List the contents of the Samba share folder for video backups.
# Replace 'videos/backup' with the correct path if needed.
samba_contents=$(smbclient -A smb.conf -c "cd videos/backup; ls" //NAS/HDD | grep D | grep -vF \.| grep -v -e '^$')
echo "Contents of the Samba share folder $samba_share:"
echo "$samba_contents"

# Check if there are subfolders in the Samba share.
if [ -n "$samba_contents" ]; then
  echo "Subfolders exist in the Samba share folder $samba_share."
else
  echo "No subfolders found in the Samba share folder $samba_share. Exiting."
  # Exit the script, as there's no need to wake up the PC.
   exit 1
fi

# Wake up the target PC using its MAC address.
# Replace 'AA:AA:AA:AA:AA:AA' with the actual MAC address.
/usr/bin/wakeonlan -i 1$target_ip AA:AA:AA:AA:AA:AA

# Set a timeout for checking if the target PC is awake (5 minutes).
timeout=$((SECONDS+300))

while [ $SECONDS -lt $timeout ]; do
    if ping -c 1 -W 1 $target_ip &> /dev/null; then
        echo "Connection to $target_ip successful."
        # Connect to the Windows PC over SSH and run a PowerShell script.
        # Replace 'mainuser' with the correct username and '~\encodevideo.ps1' with the actual script path.
        ssh mainuser@$target_ip "pwsh ~\encodevideo.ps1"
        exit 0
    else
        sleep 20
    fi
done

echo "Connection to $target_ip failed after 5 minutes."

Conclusion:
My experience with Wake on LAN has been fascinating. The ability to put PCs to sleep or shut them down, only to wake them when needed, aligns with my vision of efficiency. I'd love to hear others' ideas or projects in this realm. In the back of my mind, I can envision small-scale data centers with empty Kubernetes clusters benefiting from this approach. Has anyone used K8ts or something similar to auto-scale on-demand locally? Share your thoughts and experiences!

Transitioning from Vue 3 to Svelte: Unleashing Performance with Vanilla JavaScript (my take)

Luke Liukonen — Wed, 12 Jul 2023 03:58:49 +0000

Introduction:
In the ever-evolving landscape of web development, staying up to date with the latest frameworks and technologies is crucial. Vue.js has gained significant popularity due to its ease of use and powerful features. Because of this, a while back I went from vanilla javascript with Jquery and a template-based system over to Vue2 and eventually Vue3. However, for those seeking even greater performance gains and a more native development experience, transitioning from Vue 3 to Svelte can be a game-changer. In my case, going from a lighthouse performance score of 85 to 100. I want to be 100% honest though, this may not be an exact apples-to-apples comparison, as with my changes, I made some fairly large refactors, such as not loading my video file in the background until the play button is invoked, or using session caching on API calls to reduce network IO. In this article, I'll explore the struggle I faced while converting from my Vue 3 homepage, written using a CDN and a sprinkling of vanilla javascript, to a Svelte-based homepage. I'll try to help uncover the challenges, benefits, and the joy of embracing vanilla JavaScript in the form of Svelte, while making comparisons along the way.

The Appeal of Performance:
When it comes to web development, performance is a top priority. The Virtual DOM (VDOM) approach, employed by Vue.js and React, offers great flexibility and ease of development. However, it can come at the cost of performance. Svelte, on the other hand, takes a different approach by compiling components at build time, resulting in a smaller and more efficient bundle size. This shift from a VDOM-based framework to a more native one was a major appeal for me to undertake the transition. In comparison, Angular, while highly performant, brings a heavier framework with a steeper learning curve.

The Conceptual Struggle:
Vue.js, React, and Angular each have their own unique paradigms, which can make transitioning between them challenging. Moving to Svelte required a conceptual shift since it follows a different approach to building components. While Vue.js and React rely on a declarative approach, Svelte embraces a more imperative approach by directly manipulating the DOM. This shift challenged me to rethink my code structure and understand Svelte's component-based architecture. However, the effort was well worth it, as I discovered the true power of a lean, framework-agnostic approach.

Embracing Vanilla JavaScript:
One of the key aspects of Svelte that appealed to me was its use of vanilla JavaScript. Unlike Vue.js, React, and Angular, which introduce their own abstractions and syntax, Svelte leverages the full power of JavaScript without any additional layers. This made the transition easier for someone like me, who prefers working with vanilla JavaScript. I could utilize my existing knowledge and avoid the learning curve associated with a new framework-specific syntax. While React and Angular allow using JavaScript, they still come with additional concepts and patterns to learn.

Assembling the Webpage:
In Vue.js, React, and Angular, the process of assembling a webpage often involves defining components, writing templates, and wiring them together using directives or JSX. Svelte simplifies this process by allowing developers to work directly with HTML, CSS, and JavaScript, resulting in a more intuitive development experience. The shift from Vue's template-centric approach or JSX in React and Angular to a markup-driven approach in Svelte felt right to me. I found myself enjoying the freedom to write clean, concise code that was easily maintainable and highly performant.

Balancing Performance and Maintainability:
While I loved programming in a more vanilla JavaScript way, I initially felt pressure to use an assembled webpage due to the native performance gains and more maintainable code it offered. I was already using somewhat of a framework-driven approach using Vue3's application framework, but felt I was losing performance in having unminimized code, and a framework I'd have to load every time my site was visited. Svelte's compilation process eliminates the need for a runtime framework, resulting in smaller bundle sizes and faster load times. Additionally, the markup-driven approach and absence of a virtual DOM in Svelte simplified the codebase and made it more maintainable. It allowed me to focus on the core functionality of my application without the overhead of managing a complex framework or having to load multiple javascript files in order to have my site work.

Conclusion:
Transitioning from Vue 3 to Svelte was not without its challenges, but it was a decision I made after thorough research and consideration. As someone who prefers programming in a more vanilla JavaScript way, I explored various frameworks like Next.js, Solid, and Qwik (all of which look like great options). However, after careful evaluation, I found that the transition to Svelte felt the most intuitive for converting my JavaScript/Vue 3 homepage.

Svelte's blend of performance, simplicity, and familiarity resonated with me. The conceptual shift and the freedom to work directly with HTML, SASS (after importing the Sass library), and JavaScript made the development process more enjoyable. The native performance gains and the more maintainable codebase were significant factors that influenced my decision.

While React and Angular have their strengths and are popular choices for many developers, Svelte provided the optimal balance for my needs. It empowered me to deliver a performant website without sacrificing the simplicity and familiarity of vanilla JavaScript. Ultimately, the choice of a framework is subjective, and it's essential to thoroughly research and evaluate various options before making a decision. For me, the transition from JavaScript/Vue 3 to Svelte felt like the natural and intuitive choice, combining the performance gains, maintainability, and the joy of programming in a more familiar way.

Footnote:
Yes... this article was "co-published" as I used an AI to help assist in writing this up. Sometimes asking for help on expanding sentences and topics can lead to some interesting sentences and word choices that you may not ever think of. Feel free to comment in the comments on your thoughts on Svelte, React, Angular, Vue, Solid, NextJs or Qwik... Please keep it kind though. Coming from a background of mostly vanilla JS, and the bit of Angular, and Vue I've used in other projects, I feel like we are in the next round of the frontend framework wars, please dont make this thread a "battleground".

Why ChatGPT is close but not all there yet

Luke Liukonen — Fri, 30 Jun 2023 00:39:45 +0000

This 4-hour journey started with a Youtube video. "Encrypt your DNS traffic using Pi-Hole and a pf-sense router" may not have been its exact title, but in essence, it's what the topic was. This YouTuber indicates DNS server lookups go over an unencrypted line between you and the DNS provider, and how some of this traffic, can be watched by your ISP, even if you are using a private DNS provider like Google, Cloudflare, or Quad9. My current setup wasn't far off. I have instances of Pi-Hole running on my network, and my Pi-Hole instances pointed straight to Quad9 and a secondary provider. So I had half of the hardware but was missing pf-sense. Watching the video through, I found it really wasn't pf-sense, but a product called unbound.

Some backstories on what some of the technologies are for those interested.
Pi-Hole acts as a passthrough DNS server for your network. A DNS server you can think of as the yellow pages to the internet. Everyone connects to one and it translates "google.com" over to the IP addresses that the string represents. Pihole does the lookups from the provider, and filters anything it knows is an advertisement.

Unbound is an open-source, recursive DNS resolver that provides secure and efficient resolution of domain names. It is designed to enhance privacy and security by implementing features such as DNSSEC validation and DNS over TLS. Unbound is known for its high performance and flexibility, making it a popular choice for individuals, organizations, and even internet service providers looking for a reliable and customizable DNS resolver. It can be used as a standalone resolver or integrated into various systems and applications to improve DNS resolution capabilities. In this case, it acts as a DNS over TLS passthrough between myself and Quad9.

Like the Pen-Pineapple-Apple-Pen guy... I knew that it was only a matter of setting up a docker-compose file, that would pair up my Pi-Hole system with Unbound, and create a decent encrypted DNS out-the-door system.
Essentially traffic would go
PC -> Router -> PiHole -> unbound -> Quad9

Awesome... I had a game plan... now to implement. I went straight to ChatGPT, and asked it to set up the above for me... and within seconds I had a docker-compose file. The issue was, it directed me to a 3rd party project called Stubby.

Now I'm not going to bash Stubby or the developers who wrote it. ChatGPT recommended it to me because it was going to do what I needed without the extra overhead. Like using MS Word when all you need is notepad, sometimes it's better to use the right tool for the job. So down a rabbit hole, I went. First I found that the image ChatGPT sent didn't exist... like.. ever. Did some searches for any reference and came up blank. Ok, I thought... will find a different image. So for about 2 hours, I spent either trying to find a base image that looked reputable or even attempting to build my own. This is for security, I thought, I wasn't going to add a base image that was extremely outdated.

After two hours of failed compiles and images that were either too old or too sketchy, I caved. rewatched the video to catch the name of the product they used, unbound, and started searching Google myself. Well... to be honest, I asked chatGPT who hesitantly sent me to ANOTHER container that did not exist.

So the next two hours, I found a docker container that fits the bill, with the only exception that they used Cloudflare instead of Quad9. I still leveraged ChatGPT, but with more remedial tasks, and after 4 hours I got one of the two Pi-Holes I have running with full DNS over TLS encryption, and in the meantime, learned a lot about DNS over TLS, DNS over HTTPS, and even the new DNS over Quic. I have to reference this as the repo that got me up and running. I took their Docker-compose files and tweaked them slightly (pointing my custom unbound2.conf to /etc/unbound/unbound.conf)in order to point strictly to Quad9, and not Google or Cloudflare, as well as implement IP6 DNS Lookups as well.

So, to the Moral of the story, and the title of the article. ChatGPT helps in a lot of ways, and for some tasks, can slay. However, it is still a long way off from replacing developers' jobs, and sometimes knowing how to google your problems is still the better option.

Why I am taking a break

Luke Liukonen — Sun, 02 May 2021 03:01:03 +0000

It sucks. I've been enjoying my time writing my ideas on a community such as Dev.to, however, I feel like I am being forced to take a break. See, I just had to sign paperwork (as did everyone else) expressing ownership and IP. I do plan on staying at my current location (for now), however, this is definitely a strike against them, as the documentation indicated that they had the right to my IP, and any open source work I worked on would also need their blessing before being distributed. Now, I am wise enough to know not to work on any personal items on anything they provide. And when I use the equipment, it stays on its own subdomain and connects directly to its network. I won't mix personal with professional. What I do find discouraging, is that there was no verbiage in the contract indicating my own development outside of working hours. The wording around it seemed to indicate that they essentially own all development I do no matter what time of day. I signed off on it anyway assuming what most do... development time outside of work is not the same as at home, and if I have an idea that has nothing to do with the industry I work in, I'm safe. That said, Google, as with all things, put the fear of God into me as I read story after story of companies (not the one I am currently working at!) suing and going after IP of developers and engineers who worked on their side hustles. I'm sure if I talk with my direct manager, he would be more than ok with what I work on... that's not the issue... it's always when things change, and all the work you put into your own projects somehow gets swallowed up by some new manager who thinks that somehow your Red Square will magically fit the hole of the triangle and tries taking credit for your work (again, not saying where I am working... horror stories on Reddit and other forums are enough for me). So while I will be passively reading new and interesting things on this forum, my contributions are now going to have to be limited to reading and the comments section.

Where does that leave my Github?
That is still up for debate I guess. All my work can be dated back well before I started working at this new place, and all have licenses attached to them. For now, I feel I am safe leaving my work open and available, however, I refuse to leave it as "abandonware". Meaning all my work will receive any needed patches and bug fixes. That said, again, this stays off the company computers. It is hard to say where it goes though... there might be one day I am asked to remove it or somehow give the IP rights to code I wrote almost 20 years ago to them.

How did we get to this point?
I, unfortunately, have no idea how we let our IP, our thoughts, become the ownership of a company... especially during the time we are not working. Again... have to stress this enough!, NotMyCompanyYet... however, there have been, and will be cases where a company decides that they want ownership of something one of their employees do in their free time. Imagine if MITS wouldn't have allowed Bill Gates and Paul Allen to have their partnership Micro-Soft... or if HP would have not let Steve Wozniak work on his one-off computers where we would be today. At some point, we let the companies take control and we look to them as being the innovators while in reality, a company is only as strong as the employees that work there. And while I'm sure MITS and HP would have loved to have "thought of it back then".. the truth is, many of us, most of us, have fantastic ideas... and some of these ideas could and might be the next big thing! However, when put in a circumstance where our ideas are not our own, but that of a company, many of us will just not do it! We know that they will distort and twist the original vision to fit the mold of what they want. Removing its oringal purpose and leaving it an empty shell of something that could have been better. And with a trend of underappreciation and not having recognition of one's work (Again Not my company)(I have worked at many crap jobs during my college days to know that many companies do NOT care about you, only what you can do for them) how many of us will have our brilliant idea, only to not run with it, holding everyone back in the process.

Developer Tools - Terminal

Luke Liukonen — Sun, 25 Apr 2021 03:01:58 +0000

I love my terminal setups. I don’t know what it is about it. It reminds me of my first PC, where I was running Windows 3.1 / MS-Dos 6, except, well, its better. It also reminds me of when I first tried out linux. And I don’t mean a bloated full blown distro… this was back when I was in highschool and I found a distro that could boot off a floppy disk. The amount of power you have in the terminal is outstanding, and yet, a lot of people shy away from it. I think seeing a blinking cursor and a c:\ can be a bit intimidating. That said, as a developer, I have a terminal window open much of the time. I find switching git repos and getting latest a lot smoother using the git command then most GUI based tools. That said I’ve been using some form of terminal / prompt for many moons, and even I get nervous using it. To make my life easier however, I have “prettyfied” my terminal, and will show you how you can too.
So, I use the phrase terminal gets thrown around a LOT. I am not going to get into the nuances of terminal vs shell, since Scott Hanselman covers it very well here… https://www.hanselman.com/blog/whats-the-difference-between-a-console-a-terminal-and-a-shell

As a Windows Developer -

First and foremost is I download Windows Terminal. Second is Powershell Core. I also run the latest Powershell Core and Powershell shells. Windows Terminal gives you the option of quickly switching to Command Prompt, Powershell, Powershell core, Azure, or any installs WSL linux distros on your machine. From there I install NerdFonts, powerline, and Oh My Posh. Directions to install can be found here..https://docs.microsoft.com/en-us/winddows/terminal/tutorials/powerline-setup
This not only makes my terminal super nice looking, it gives me quick access to git information such as number of changes made, if it’s in sync, and which branch I am in.

As a Mac or Linux Developer -

Ok, so I haven’t tried this in a shell only linux distro, but in Gnome using xterm. For Mac, I have used terminal and Term2 and between the two, I haven’t seen a reason not to use Macs default terminal. For linux, I install Zsh shell. Mac already has Zsh installed (most recent versions). Once I have it installed, as above, I install a nerdfont based font for the GUI. This gives the nice fancy Icons such as the Git Icon and Linux / Apple logo in the corner. Oh my Zsh (https://ohmyz.sh) is then installed to get the fancy tools for git. For a theme, I installed Powerlevel10k. This gives me a GUI very similar to powerline on windows. Install directions can be found at its website, https://github.com/romkatv/powerlevel10k. This gives me the same bells and whistles I use on my Windows PC

How to use your Cellphone as a development machine*

Luke Liukonen — Sun, 18 Apr 2021 00:37:12 +0000

Yes... You see that right. After playing around with my Samsung Note 9, I have been able to create a full fledged machine and develop code on it... But it's not entirely running on my local machine, so there is the giant astric next to it... Here are the tools I am using
Hardware

My samsung galaxy note
A raspberry pi or other machine to connect to. You could even set up a docker instance using AWS or Azure
A usb multiport usb type c adapter, to hook up a keyboard, mouse and monitor

For software

Dex. Since I am in the Samsung ecosystem, it gives me a fantastic desktop environment
A browser
(optional) Termius. I was dumbfounded when I fired up an instance against my local machine and saw it worked locally (and run linux commands directly against my phone), but in this instance, it is to ssh into your remote machine for install and maintenance
Code Server. I am running this in a container on my remote machine, but essentially it is a instance of VS code in a web browser. This is my remote ide for development

This is a proof of concept that I confirmed works. I did run into some trouble with not having node installed, but there is an accessable terminal in code Server to install those dependencies, as it's built in Debian. Overall I am fairly happy with it. Let me know if anyone is interested in the setup / config directions on how to set it up.

But do you Terminal?

Luke Liukonen — Sat, 10 Apr 2021 23:52:18 +0000

With my new job comes a new PC... well, a new machine. I have run Windows for so long that going to a mac makes me feel like a new user all over again, and not in a good way. Goodbye is the print screen key, Ctrl C, Ctrl V, (I found how to bring that back by remapping the command key to ctrl) as well as Windows Terminal. I have been a HUGE fan of Windows Terminal and OH-MY-POSH and used them on a daily basis. If I need a Powershell, Command prompt or Ubuntu/Debian shell, it was a click away! That said, I don't know how many devs even touch the command line anymore. I'd like to think it's either still widely used or even making a coming back, especially with Git, Node, DotNet, and other cli tools, but I was surprised to see how many people use 3rd party tools or plugins! Ok, so VS code is fantastic with its built-in Git process... but at some point, I would imagine devs have to use shell at some point. Mac has its terminal, but, wow is it bare bones! Luckily, I know that Macs use ZSH, and Oh my zsh did the trick. Well, "Oh my ZSH" and Powerlevel 10k (along with a Nerdest font). There are some glitches when resizing the window, but does it feel good seeing git information in the active directory I am working in.

Handling outages gracefully

Luke Liukonen — Sun, 04 Apr 2021 03:42:47 +0000

Lately, my Spectrum internet has been on the fritz. I really can't tell why either. It will just cut out and cut back in again. I've seen this occurring almost like clockwork when I was working from home, and expect to see more of it in the upcoming future, as I'll be working from home again. It does have me thinking though about how we as developers have to handle the rare case that the user can't connect.
Here are some ideas on what we can do as developers to help in the event that something goes down....

Service Workers. Writing progressive web apps, the service worker has been both a blessing and a nightmare. If configured improperly, expect none of your JS files to ever be upgraded or it to do nothing for you. That said, the ideal in at least my world right now is to use the Service worker to cache an instance of my Javascript, but always try for the latest. More info about service workers can be found here.
Data caching. This can be client or server-side. Most data needs to come from somewhere. Many times it's either a SQL server or Web API. I am a huge fan of implementing caches on frequently called data. Especially on calls that will always return the same data. Implementing caches not only reduces the potential of a bad user experience in the event of a network outage but in many cases, can speed up the response time of your site.
Use a CDN. CDN's are like giant global internet caches. They help with response time by serving up data closer to the user and behave like a bit internet cache. As well many of them cost nothing or little to use. There can be downsides, like if a CDN acts up, but all in all, the pros to using one outweigh the cons.
Have a backup box. In the event of an outage, either have a fallback / secondary box to hit, or use a service like a load balencer if one machine acts up. In the case of my chatbot, if my box goes down, the code SHOULD fall back to Heroku (that said, there may be a bug or two in my logic on that... something I need to investigate)
Investigate based on your platform. Going to last year's virtual Mongo DB conference, I found they were putting a lot of time and effort into "local Mode", where in the event you can't connect to the DB, it keeps it local until you can. I'm sure other platforms either have their own or similar concepts you can use to help with this.

I'm keeping this week's post a bit light, with the holiday going on (Happy Easter).Last note... Keep the mindset that while we live in a connected world, there are still dead zones. Let me know what you think, or if you have any additional recommendations on the topic