DEV Community: Hanswillem

Getting CrewAI to run local, the prep work of the infra always exceeds the actual learning work

Hanswillem — Fri, 08 Aug 2025 14:38:14 +0000

Wanted to play around with agents. Decided to go for one of the more popular frameworks, crewai.

Stubborn decided to install it local. Took me an afternoon bug fixing and library checking to get all up and working.

Findings:
CrewAI runs on LiteLLM, this interacts easier with ollama as my preferred LM studio. Running now also Ollama, maxing out my laptop harddisk by having now some models double (luckily apple is so cheap with storage and memory :-D)

Got down the rabbit hole in debugging, expected the issue to be in LiteLLM. Got strange errors on that one related to a module on calculation which didn't know the model name. Switched to one of the standard models, but only solved part of the problem.

Faced the biggest challenge with an error of LiteLLM:ERROR: litellm_logging.py RuntimeError: can't register atexit after shutdown . First tried to vibe code and vibe ask myself out of this, but unfortunately my models are not strong enough to have the mighty answer. Went back to 1999 and used good old stack overflow. Found here the answer: https://stackoverflow.com/questions/65467329/server-in-a-thread-python3-9-0aiohttp-runtimeerror-cant-register-atexit-a

not really related to the python version is was running nor liteLLM, but adding the

# Added these two imports
import concurrent.futures.thread
import concurrent.futures.process

# and this is just the standard code from the liteLLM side
from litellm import completion

print("start test")
response = completion(
            model="ollama/llama2",
            messages = [{ "content": "Hello, how are you?","role": "user"}],
            api_base="http://127.0.0.1:11434"
)
print(response.object)
print("done test")

Going back to crewai, it worked again. Learned again a lot of stuff on vent's, python uninstall and installs.

To be continued for the crewai learnings

PCB - Assembly flashbacks?

Hanswillem — Thu, 17 Jul 2025 11:19:11 +0000

I really really liked my bachelors wrt programming. However there was one course which didn't got me energized; low level programing. This included writing assembly code an inject it direct into PCBs.

Recently I decided to create my own CO2 censor to steer the air quality in home (air quality is fine, but this is just a fun project to envision). For this I want to create something with ESP32 and sensors.

Created a following plan of approach:

Start with Adruino, first the basics with a breadboard
Move to raspberry for the software
create an endpoint based on esp32 and sensor, probably with some folding involved (3d printing for the case?)

Just received the Strex Adruino clone with tooling, lets see which things we can play with it.

MCP setup - Huggingface

Hanswillem — Sun, 13 Jul 2025 12:33:43 +0000

there is a tendency to think that every AI company is from either the US or china, but there are some with (semi) European roots. One of them is hugging face.

They have the MCP course, one often advised as play around course. For my first steps with MCP I don't want to run all local for the first time and first play a bit around in a more sandboxed environment.

https://huggingface.co/learn/mcp-course/unit2/introduction

One of the fun things is that I again come back to my apple renewably state. Python is not directly installed from the command line, thus learning first all new apple stuff (homebrew, uv). Linux is sometimes so much easier :)

After installing home-brew it was installing VU & python.
My python skills are a bit rusty, seems the virtual setup of python is not standard installed, thus also python -m venv .venv didn't work. In the end it seems its python3 -m venv .venv which did the trick (thanks to the super python RTFM :D)
https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/

Setting up a new home lab for arp poisoning

Hanswillem — Thu, 12 Jun 2025 13:14:53 +0000

As mentioned earlier I use a Kali version with Websploit for trying out my CEH labs.

Setting is Kali running with multiple containers to play around.
Do want to try now something new, made a new homelab to play around with things like ARP poisoning.

[placeholder]

Setup was easy in VM box with 1 ubuntu and one cloned, but then the network part started.

While I've chosen clone instead of create new, I was smart enough to check the option " create new Mac"
unfortunately after running ifconfig it showed both of the times the same ip address, with different Macs...

Lesson learned, there first needed to be a virtual NAT to have both systems in the same network

[ image virtual nat]

while there was some tweaking needed the lab succeeded in running ettercap and Wireshark to do an arp poisoning from the attacker box and sniffing the traffic of the victim :-)

[ image of Wireshark side by side to be added ]
Lab which I am conducting:
http://www.csc.villanova.edu/~enwafor/cps_security/documents/lab1_mitm_update.pdf

WebSploit containers don't like ARM

Hanswillem — Wed, 04 Jun 2025 17:04:21 +0000

As mentioned in an earlier post, I am preparing for my CEH foundation exam by following the CEH course on Oreilly.

Previous post explained how I got kali running, all smooth on VirtualBox with an arm image. However, I was cheering too fast. Websploit is using containers which are made for x86 infra, resulting in not able to run on my kali version.

Some googling learned that more people faced this issue. Seemed some tried to change the install procedure (dunno how that would solve the build issue...), at least no easy fix seemed to be available when googling. Then I was retinking, there is virtualization but also emulation. With emulation I could run x86 on the Mac. Enter UTM.

[post will be updated later on]

LLM on MacOS: XLM support

Hanswillem — Fri, 30 May 2025 06:56:11 +0000

One of the reasons to go for Mac was the unified memory setup. With the RAM shared between CPU and GPU apple silicon has a (theoretical) cost and power efficiency over a nVidia setup. Not talking about raw power of a 5900 setup, but compared to my earlier lab with a virtualized setup on a Ryzen xxx I do expect grand improvements.

Base case: old lab

[placeholder to set numbers]

Setting up LM Studio

First things first, I had an old setting with Ubuntu running on AMD with laptop GPU. As bit of a geek I prefer terminal over GUI for hobby projects thus Ollama was a logical starting point. But on Mac there is off course a more user friendly (less geeky) setup possible with LM Studio. For my projects are hobbies I like to play a bit more over ease of use, thus terminal and Ollama it is (possible I add mystify as GUI later on for fun).

Unfortunately I Ollama was at the time or writing not 100% stable supporting MLX (apple's CUDA to make it oversimplified). As I've read that MLX can give 30% speed bums I decided to not geek and go for LM Studio :-)

20 Tokens/Second on deepseek qwen 3 -8b, that's not bad

Moving (back) to Mac & setting up CEH Labs

Hanswillem — Thu, 29 May 2025 18:53:45 +0000

Moving after years back to a Mac. MacBook Air M4 as it has unified memory, let's see what the output is.

Gut feeling sooner or later I will move to a separate server with a high end nVidia.

First challenge was getting my labs up and running the labs for my Certified Ethical Hacker training. Via O'reilly I've started the course of Omar Santos (https://www.oreilly.com/search/?q=Omar%20Santos&rows=10).

Forgot apple moved during the years we've got kids from x86 to ARM, thus couldn't use the standard VirtualBox images. When reading a bit on it I stumbled upon some horror stories on getting Kali up and running on apple silicon.... however for me the install went smoothly. Used as reference this blog; https://kskroyal.com/kali-linux-virtualbox-apple-silicon/

Now happy to have a smooth setting for the labs, next step to see if websploit is able to run & get the coding tools on the main macOS image up and running.

Setting up my home lab

Hanswillem — Mon, 05 May 2025 13:53:57 +0000

The Christmas holiday period always brings some opportunity to sneak in some time for hobby projects. Either its because the rest of the family is still sleeping in during the morning or to just sit with the laptop next to the kids while they look a Christmas movie.

During recent years end period there was a lot of excitement about the deepseek model, released open source, so i decided to give it a try to run local.
Unfortunately i don''t have the state of the art hardware anymore which i had as a kid (3df FX Voodoo in SLI) nor didn’t i continue running everything on macbooks and mac mini's as during my student time. Currently all I have running is a not that state of the art windows laptop... so i knew it wouldn’t have good results in token processing time, but that shouldn’t spoil the fun.

First setting up the environment. From my ethical hacker lab I was already used of using oracle box (with Kali), so I continued in that space.
As Kali was for me a bit too specialized, I decided to spin up an Ubuntu vm and followed the tutorial of Pavan:
https://dev.to/pavanbelagatti/run-deepseek-r1-locally-for-free-in-just-3-minutes-1e82

Setting up went fast, although there where as usual some extra libraries to upgrade. Also made the rookie mistake to give the VM insufficient disc space :-D

First impression: slooooooooooooooooooooooooooooooooooow but also what a fun to use!
Didn’t had real experiences with a reasoning model, liked it a lot.

Planned three homelabs as follow-up:

Using python scripts to call the local LLM (update May: done, see next blog post)
Play around with different LLMs to measure performance
Install stable diffusion to create pictures for this blogpost (if hardware allows it...)