DEV Community: Kornel Rzeczycki

Own AI chatbot in terminal

Kornel Rzeczycki — Sun, 21 Jun 2026 19:40:08 +0000

Lately I was looking for an opportunity to expand my knowledge in algorithms. Particularly for sharpening designing skills for optimisation purposes.
Hence I found it useful to incorporate my PC powers to give me some boost in this matter.
I did a bit of research and it turns out that there is plenty of options having Apple M2 Silicon on board with 16GB of RAM.
My choice was Qwen2.5-7B. Why You could ask? Well first and foremost it has excellent reasoning particularly in Mathematics and logical exercises which I find useful in tasks I am about to throw at it.
Secondly it is reliably small which makes it maintainable in long term. Also it does not eat up too much RAM memory to avoid swap overworking.
So once You have this covered You can download the model. I was fully aware it needs quantisation by some well renowned party.

So I went under this address:
https://huggingface.co/bartowski/Qwen2.5-Math-7B-Instruct-GGUF/tree/main
And downloaded the model below: Qwen2.5-Math-7B-Instruct-Q4_K_M.gguf

I chose Qwen2.5-Math-7B-Instruct because of its strong performance in mathematical reasoning and logical problem-solving, which aligns well with algorithmic practice tasks. Its relatively small size also makes it suitable for local execution on Apple Silicon machines with 16GB of unified memory, without excessive reliance on disk swapping.

To run it efficiently, I used a quantised GGUF version. Quantisation reduces the precision of model weights (for example, to 4-bit integers), significantly lowering memory usage while maintaining acceptable output quality. In this case, the Q4_K_M variant reduces the model size to approximately 4–5 GB.

The goal is to ensure the model fits entirely in unified memory, avoiding swap usage, which would severely degrade inference speed and responsiveness.

So why is it that important? Ideally the size of model should fit in the free RAM memory not being used. Because Apple Silicon uses a unified memory architecture, it is important that the model fits in available memory to avoid swapping to disk. What this brings us to? Well you want the model to generate text in reasonable time to conversation to go fluently. Secondly you want to avoid situation where the model overflows the fast RAM memory and leak into SWAP memory (that is the part of You physical hard drive which serves as additional RAM) that is painfully slow and increase wearing out of your SSD.

Having all this set up we need to think about how would we use our brand new model on PC. You need an engine to run it on the pc. My choice was llama.cpp because I usually like to tinker with things and it gives endless possibilities in that terms.

What you will need here at the beginning is a cmake. It is a tool to build programs in C/C++ language.
You can easily install it by running: brew install cmake

Then I strongly advise to compile the source code from source. Without Metal enabled, the model runs entirely on CPU, which significantly reduces performance.

So now you need a separate folder and in there You should execute commands below:
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build -DGGML_METAL=ON cmake --build build --config Release

Now engine is set up and you can give LLM a go.
To do this assuming that you stored a model under models directory the command should look something like this:

And this brings us to another point. You can now talk to the chatbot and have the whole university library university in one place! This makes local experimentation much more accessible.

Where did all my disk space space go?

Kornel Rzeczycki — Sat, 20 Jun 2026 18:37:42 +0000

In today's post I wanted to give some basic introduction of the free program that I discovered.
GrandPerspective gave me the ability to analyze disk usage and identify files and folders that were taking up more storage space than I realized. This was especially useful because I am preparing my Mac for an offline LLM deployment.
Its strong feature is the visuals. You can visualise Hard Disk usage just like that in the form of visually appealing blocks just as you were viewing computer memory from Grand Perspective!

It will definitely have its place in my tool belt!
Secondly it is 100% Mac compatible software including Apple Silicon Macs.
Most importantly it has a very useful version which is totally free!
However, you can purchase a paid version from Apple App Store whenever you want.
Check out the link below.

grandperspectiv.sourceforge.net

First App built with Google AI studio

Kornel Rzeczycki — Sun, 29 Mar 2026 12:13:59 +0000

This post is my submission for DEV Education Track: Build Apps with Google AI Studio.

What I Built

I built an app to generating full box cover of retro game at a snap of the finger. It includes the front being an image generated with Imagen from Google and game text description on the back of the cover. The master prompt I used:
"Please create an app to generate new unique Retro Game cover picture using Imagen for the visuals, and Gemini to create the text descriptions for the game."

Demo

https://ai.studio/apps/c1aa2a41-6d3c-46c7-9bc4-a6fb1357fc1f?fullscreenApplet=true

My Experience

I was surprised how easy it was to generate an app in just a few minutes from successful prompt. Also I was curious about how new Google AI studio can handle such tasks which to me does it surprisingly well. I've got a bit of react along the way to read through generated code.