DEV Community: Alexey

Как я впихивал три модели в 6 ГБ видеопамяти

Alexey — Sat, 31 Oct 2026 14:11:28 +0000

Пару дней назад мы с другом участвовали в хакатоне. Кейс — веб-сервис для видеолекций: загружаешь запись или ссылку на YouTube, получаешь расшифровку, субтитры, конспект с оглавлением и поиск по смыслу. Чтобы студент написал «где он доказывал теорему Пифагора» и попал на нужную секунду видео.

На всё было три дня. Заняли пятое место.

Проект называется Lectio, код на GitHub. Но пост не столько про хакатон. Он про одну проблему, которая съела у меня неожиданно много времени: заставить всё это работать на моём компьютере.

Железо

Ryzen 5 5600
32 ГБ DDR4
GTX 1660 Super на 6 ГБ

1660 Super вышла в 2019 году. Это Turing без тензорных ядер. Для игр нормально. Для нейросетей — скажем так, скромно.

И всё должно было работать офлайн. Никаких облачных API, всё локально.

Три модели, одна маленькая карта

Обработке лекции нужны три модели:

Whisper large-v3-turbo (через faster-whisper) превращает речь в текст с таймкодом каждого слова.
BGE-M3 превращает куски расшифровки в векторы для поиска по смыслу.
Qwen3 4B (через Ollama) пишет конспект, оглавление и тест.

По отдельности каждая влезает в 6 ГБ. Все три вместе — нет.

А 6 ГБ — это на самом деле не 6 ГБ. Рабочий стол забирает 1–1,5 ГБ ещё до того, как я что-то запустил. Дискорд забирает ещё полгига. Это я узнал, когда Ollama вдруг начала гонять 8–17% модели на процессоре, а причиной оказался Дискорд в трее.

Первый заход

Самый простой вариант — загрузить все модели один раз, держать в памяти и обрабатывать лекции. В 6 ГБ это не влезает, и первым по памяти упал Whisper: Ollama после вопроса в чате ещё держала модель, и места не осталось.

Дальше было интереснее самого падения. Модели очень по-разному ведут себя, когда памяти не хватает.

Ollama подстраивается. Если карта занята, она тихо выносит часть слоёв на процессор и продолжает работать. Медленнее, но работает.

Whisper и эмбеддер не подстраиваются. Они пытаются выделить память, не могут и роняют всю задачу.

Получалось, что вежливой модели доставалось всё, а строгие падали. Ровно наоборот. Значит, память надо отбирать у Ollama до того, как грузятся мои модели, а не наоборот.

По этапам

Теперь воркер обрабатывает одну лекцию за раз, и на каждом этапе карта целиком принадлежит одной модели:

Попросить Ollama выгрузить модель.
Загрузить Whisper, расшифровать, выгрузить Whisper.
Ещё раз попросить Ollama выгрузиться — вдруг чат успел её загрузить.
Загрузить BGE-M3, посчитать векторы, выгрузить BGE-M3.
Теперь вся карта у Ollama, она пишет конспект.

Всё это висит на одной настройке, RELEASE_MODELS_BETWEEN_STAGES=true. На карте побольше её выключаешь, и ничего не выгружается.

Ollama не отпускает

По умолчанию Ollama держит модель в памяти пять минут после последнего запроса. То есть если кто-то спросил что-то в чате по лекции, а через минуту в обработку пришла новая лекция, Whisper стартовал при занятой карте.

У Ollama для этого есть ручка: запрос с keep_alive: 0, и модель выгружается сразу. В OpenAI-совместимом API её нет, поэтому зову родной:

async def unload(self) -> None:
    native_url = self._base_url.removesuffix("/v1") + "/api/generate"

    try:
        async with httpx.AsyncClient(timeout=10.0) as client:
            await client.post(
                native_url,
                json={"model": self._model, "keep_alive": 0},
            )
    except httpx.HTTPError:
        # Other OpenAI-compatible servers don't have this endpoint.
        return

Если LLM не Ollama, запрос просто тихо не проходит, и обработка идёт дальше.

Выгрузить модель сложнее, чем кажется

Модели создаются один раз и кешируются:

@cache
def get_transcriber() -> Transcriber:
    return FasterWhisperTranscriber()


def release_transcriber() -> None:
    get_transcriber.cache_clear()
    gc.collect()
    torch.cuda.empty_cache()

Сбросили кеш, запустили сборщик мусора, попросили PyTorch отдать память. Вроде всё.

Не освобождалось ничего.

Проблема была в коде, который модель использовал:

model = get_transcriber()
try:
    transcript = model.transcribe(audio_path)
finally:
    release_transcriber()

В момент вызова release_transcriber() локальная переменная model всё ещё указывает на модель. Кеш сброшен, но объект жив, сборщик мусора его не трогает, и память остаётся занятой.

Исправление выглядит странно, но работает. Не держать модель в переменной вообще:

# No local reference to the model: release_* can only free memory
# once nothing else points at it.
try:
    transcript = await asyncio.to_thread(
        lambda: get_transcriber().transcribe(audio_path, progress),
    )
finally:
    release_transcriber()

Комментарий в коде нужен, чтобы я потом это «не почистил».

fp16, но правильно

BGE-M3 в fp32 для этой карты жирновата, поэтому она работает в половинной точности. Первая версия была очевидной:

model = SentenceTransformer("BAAI/bge-m3", device="cuda").half()

Памяти освободилось меньше, чем ожидал. Модель сначала грузилась в fp32 и только потом конвертировалась, а fp32-копия оставалась висеть в кеше CUDA-аллокатора PyTorch.

Грузить надо сразу в fp16:

self._model = SentenceTransformer(
    settings.embedding_model,
    device=device,
    model_kwargs={"dtype": torch.float16} if device != "cpu" else None,
)

Размен, которого я не ожидал

API нужна своя копия BGE-M3, чтобы превращать поисковые запросы в векторы. Ещё одна модель на карте.

Я перенёс её на процессор, и цифры получились такие:

поиск стал медленнее примерно на 40 мс на запрос, этого никто не заметит;
Qwen3 теперь целиком помещается на видеокарту и выдаёт 68 токенов в секунду вместо 39.

Поиск на 40 мс медленнее за почти вдвое более быструю LLM. Одна строчка в .env:

SEARCH_DEVICE=cpu

Мелочи

Ещё несколько настроек дали заметную разницу:

WHISPER_COMPUTE_TYPE=int8_float16
WHISPER_BATCH_SIZE=4
LLM_MODEL=qwen3:4b-instruct

Веса Whisper в int8 с вычислениями в fp16, маленький батч и модель на 4B вместо 8B. Для Ollama ещё включил OLLAMA_FLASH_ATTENTION=1 и OLLAMA_KV_CACHE_TYPE=q8_0 — кеш контекста становится вдвое меньше.

И одна штука, в которой я так и не успел разобраться. faster-whisper работает на CTranslate2, а не на PyTorch. Но CTranslate2 находил cuBLAS и cuDNN, только если перед ним был импортирован torch. Поэтому транскрайбер начинается так:

import torch  # noqa: F401
from faster_whisper import BatchedInferencePipeline, WhisperModel

Импорт нигде не используется. Уберёшь его — Whisper перестаёт видеть CUDA.

Цифры

Вечером перед защитой я прогнал через конвейер лекцию на 1 час 25 минут. От загрузки до «готово» ушло 14 минут 16 секунд:

Этап	Время
Извлечение аудио (ffmpeg)	7 с
Расшифровка (Whisper)	5 мин 2 с
Индексация (BGE-M3)	39 с
Конспект (Qwen3 4B)	8 мин 27 с

Сам Whisper расшифровал 85 минут звука за 4 минуты 56 секунд. Это примерно в 17 раз быстрее реального времени, на карте без тензорных ядер.

Поиск работает уже через шесть минут. Всё, что дальше, — это 4B-модель пишет конспект. Это 60% всего времени, и именно это сильнее всего ускорит карта побольше или модель посильнее.

Что я из этого вынес

Я думал, сложными будут сами модели. Нет. Модели хорошие, библиотеки хорошие, и каждая нормально работает, пока она одна на карте.

Сложно было делить. Какая модель вежливо отдаёт память, а какая падает. Кто держит ссылку, про которую ты забыл. Какая библиотека что кеширует и сколько держит.

В документации к моделям этого нет, потому что каждая описана так, будто она единственная на твоей видеокарте.

В 6 ГБ так не бывает никогда.

How I squeezed three models into 6 GB of VRAM

Alexey — Sat, 31 Oct 2026 14:10:31 +0000

A few days ago my friend and I took part in a hackathon. The case was a web service for video lectures: upload a recording or a YouTube link, get a transcript, synced subtitles, a summary with a table of contents, and search by meaning, so a student can type "where did he prove the Pythagorean theorem" and land on the right second of the video.

We had three days. We took fifth place.

The project is called Lectio and the code is on GitHub. This post isn't really about the hackathon though. It's about one problem that ate a surprising amount of my time: making all of it run on my own PC.

The hardware

Ryzen 5 5600
32 GB of RAM
GTX 1660 Super with 6 GB of VRAM

The 1660 Super came out in 2019. It's a Turing card without tensor cores. It's fine for games. For running neural networks it is, let's say, modest.

And the whole thing had to work offline. No cloud APIs, everything local.

Three models, one small card

The processing pipeline needs three models:

Whisper large-v3-turbo (through faster-whisper) turns speech into text with a timestamp for every word.
BGE-M3 turns chunks of the transcript into vectors for semantic search.
Qwen3 4B (through Ollama) writes the summary, the table of contents and the quiz.

Each of them fits into 6 GB on its own. All three together don't.

And 6 GB isn't really 6 GB. My desktop takes 1–1.5 GB before I start anything. Discord takes another half a gigabyte. I found that out when Ollama suddenly started running 8–17% of the model on the CPU, and the reason turned out to be Discord sitting in the tray.

The first attempt

The simplest version is to load every model once, keep it in memory and process lectures. On 6 GB that doesn't fit, and Whisper was the first to run out of memory: Ollama was still holding the LLM after a chat request, and there was no room left.

What I noticed next was more interesting than the crash itself. The models behave very differently when there's not enough memory.

Ollama adapts. If the card is busy, it quietly offloads part of the layers to the CPU and keeps working. Slower, but it works.

Whisper and the embedder don't adapt. They try to allocate memory, fail and take the whole job down with them.

So the polite one was getting everything and the strict ones were crashing. That's backwards. The fix was to take memory back from Ollama before my own models load, not the other way around.

Stage by stage

The worker now processes one lecture at a time, and every stage gets the card to itself:

Ask Ollama to unload its model.
Load Whisper, transcribe, unload Whisper.
Ask Ollama to unload again, in case the chat loaded it in the meantime.
Load BGE-M3, build the vectors, unload BGE-M3.
Now Ollama gets the whole card for the summary.

It's all behind one setting, RELEASE_MODELS_BETWEEN_STAGES=true. On a card with more memory you turn it off and nothing gets unloaded.

Ollama doesn't let go

By default Ollama keeps a model loaded for five minutes after the last request. So if someone asked the lecture chat a question, and a minute later a new lecture went into processing, Whisper would start with the LLM still occupying the card.

Ollama has an endpoint for this: send a request with keep_alive: 0 and it unloads the model right away. It isn't part of the OpenAI-compatible API, so I call the native one:

async def unload(self) -> None:
    native_url = self._base_url.removesuffix("/v1") + "/api/generate"

    try:
        async with httpx.AsyncClient(timeout=10.0) as client:
            await client.post(
                native_url,
                json={"model": self._model, "keep_alive": 0},
            )
    except httpx.HTTPError:
        # Other OpenAI-compatible servers don't have this endpoint.
        return

If the LLM is something other than Ollama, the request just fails quietly and processing continues.

Unloading a model is harder than it sounds

Models are created once and cached:

@cache
def get_transcriber() -> Transcriber:
    return FasterWhisperTranscriber()


def release_transcriber() -> None:
    get_transcriber.cache_clear()
    gc.collect()
    torch.cuda.empty_cache()

Clear the cache, run the garbage collector, tell PyTorch to give the memory back. Looks fine.

It didn't free anything.

The problem was in the code that used the model:

model = get_transcriber()
try:
    transcript = model.transcribe(audio_path)
finally:
    release_transcriber()

At the moment release_transcriber() runs, the local variable model still points at the model. The cache is cleared, but the object is alive, so the garbage collector can't touch it and the memory stays taken.

The fix looks weird, but it works. Never keep the model in a variable:

# No local reference to the model: release_* can only free memory
# once nothing else points at it.
try:
    transcript = await asyncio.to_thread(
        lambda: get_transcriber().transcribe(audio_path, progress),
    )
finally:
    release_transcriber()

The comment in the code exists so I don't "clean it up" later.

fp16, but the right way

BGE-M3 in fp32 is too much for this card, so it runs in half precision. My first version was the obvious one:

model = SentenceTransformer("BAAI/bge-m3", device="cuda").half()

Memory usage dropped less than expected. The model was loaded in fp32 first and converted afterwards, and the fp32 copy stayed in PyTorch's CUDA allocator cache.

It has to be loaded in fp16 from the start:

self._model = SentenceTransformer(
    settings.embedding_model,
    device=device,
    model_kwargs={"dtype": torch.float16} if device != "cpu" else None,
)

The trade-off I didn't expect

The API needs its own copy of BGE-M3 to turn search queries into vectors. Another model on the card.

I moved it to the CPU, and the numbers turned out like this:

search got slower by about 40 ms per query, which nobody will notice;
Qwen3 now fits on the GPU entirely and generates 68 tokens per second instead of 39.

A 40 ms slower search for an LLM that's almost twice as fast. One line in .env:

SEARCH_DEVICE=cpu

The small stuff

A few more settings made a real difference:

WHISPER_COMPUTE_TYPE=int8_float16
WHISPER_BATCH_SIZE=4
LLM_MODEL=qwen3:4b-instruct

Whisper weights in int8 with fp16 computation, a small batch, and a 4B model instead of 8B. For Ollama I also turned on OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0, which makes the context cache half as big.

And one thing I didn't have time to dig into. faster-whisper runs on CTranslate2, not PyTorch. But CTranslate2 only found cuBLAS and cuDNN when torch had been imported before it. So the transcriber starts with this:

import torch  # noqa: F401
from faster_whisper import BatchedInferencePipeline, WhisperModel

The import isn't used. Remove it and Whisper stops finding CUDA.

The numbers

The evening before the defense I put a 1 hour 25 minute lecture through the pipeline. From upload to "ready" it took 14 minutes 16 seconds:

Stage	Time
Extracting audio (ffmpeg)	7 s
Transcription (Whisper)	5 min 2 s
Indexing (BGE-M3)	39 s
Summary (Qwen3 4B)	8 min 27 s

Whisper itself took 4 minutes 56 seconds for 85 minutes of audio. That's about 17 times faster than real time, on a card with no tensor cores.

Search works after six minutes. Everything after that is the 4B model writing the summary. That's 60% of the total time, and that's what a bigger card or a stronger model would speed up the most.

What I took from this

I expected the hard part to be the models. It wasn't. The models are good, the libraries are good, and each of them works fine as long as it's alone on the card.

The hard part was sharing. Which model gives memory back politely and which one crashes. Who holds a reference you forgot about. Which library caches what, and for how long.

None of it is in the model docs, because each model is documented as if it's the only thing running on your GPU.

With 6 GB it never is.

I Moved From Go to FastAPI. It Felt Suspiciously Easy.

Alexey — Fri, 07 Aug 2026 19:46:03 +0000

For quite a while, most of my backend code was written in Go.

I built projects like Logiflow, a logistics backend with PostgreSQL, Redis, WebSockets, OSRM and monitoring, and Sentinel, an event-driven fraud detection project using Kafka and PostgreSQL.

At some point I also got tired of setting up the same Go project structure over and over again, so I built gofro, a small scaffolding CLI for generating the boring parts.

I got used to the way Go backend projects feel.

You create the server, configure the router, initialize the database, construct dependencies, pass them around, handle errors and gradually connect everything together.

Then I started building LinkUp with Python and FastAPI.

My first impression was simple:

This felt suspiciously easy.

Not because the project itself was easier, but because a surprising amount of code I was used to writing simply disappeared.

After spending some time with FastAPI, I started to understand where that complexity actually went.

Where did all the plumbing go?

A typical Go project of mine has a fairly explicit structure.

cmd/
internal/
    api/
    config/
    database/
    handler/
pkg/
migrations/
Dockerfile
Makefile

I like this approach.

If something exists in the application, I can usually find where it was created and where it was passed.

Routes are registered somewhere. Dependencies are constructed somewhere. Database connections are opened somewhere.

When I started using FastAPI, the same things looked very different.

For example, you can define a Pydantic model:

class RegisterRequest(BaseModel):
    email: EmailStr
    password: str = Field(min_length=8)

Then use it directly in a route:

@router.post("/register")
async def register(data: RegisterRequest):
    ...

FastAPI understands that data comes from the request body. Pydantic validates it, FastAPI uses the same information for OpenAPI, and serialization is handled for you.

When I first saw this, I had to stop for a moment and figure out how FastAPI knew what I wanted.

In Go I was used to seeing more of this process explicitly.

FastAPI did not remove request parsing or validation. It just moved them behind conventions built around Python type annotations.

That became a recurring theme while learning it.

Less visible code does not necessarily mean less happening.

Then I found `Depends`

Dependency injection felt even stranger at first.

In LinkUp I have dependencies for things like the database session, Redis and the currently authenticated user.

A database dependency can be represented roughly like this:

SessionDep = Annotated[
    AsyncSession,
    Depends(get_session),
]

Then another route or dependency simply asks for SessionDep.

FastAPI resolves it.

Coming from Go, I was much more used to something like this:

server := NewServer(db, redis, logger)

Everything is visible right there.

With FastAPI, you look at a function parameter and the framework basically says:

"Yeah, I know a guy."

At first this felt like magic.

After using it for a while, it started making more sense. The dependency graph still exists. FastAPI just lets me describe part of it declaratively instead of wiring every dependency manually.

I still prefer understanding where those dependencies come from, but I no longer find the approach strange.

Most of the time.

`async def` does not make everything asynchronous

This was probably the most important adjustment.

Go gave me a fairly straightforward application-level model of concurrency.

If I want to start another concurrent task, I can write:

go doSomething()

The runtime obviously does a lot behind that simple line, but goroutines are a normal part of how Go programs are written.

Python async works differently.

In FastAPI, it is very easy to look at this:

async def login(...):
    ...

and mentally classify everything inside it as asynchronous.

That is not how it works.

I ran into a good example while implementing authentication in LinkUp.

Passwords are hashed with Argon2. Argon2 is intentionally expensive. Making password hashing computationally expensive is part of its job.

But Argon2 is still synchronous CPU work.

Putting that call inside an async def does not suddenly make it cooperate with the event loop.

If the hashing operation runs directly on the event loop thread, the loop has to wait for it to finish.

So in LinkUp, password hashing and verification are moved into a thread pool:

async def hash_password(password: str) -> str:
    return await run_in_threadpool(
        password_hash.hash,
        password,
    )

This was a useful reminder that async is not a property that spreads through the entire function.

An async PostgreSQL driver can yield while waiting for network I/O.

Redis can yield.

An HTTP client can yield.

CPU-bound synchronous work cannot do that automatically.

It sounds obvious once you understand the model, but an endpoint full of async keywords can make it easy to forget what is actually happening underneath.

SQLAlchemy was familiar and unfamiliar at the same time

The database side was another interesting part of the transition.

The concepts were already familiar.

A request needs a database session.

Something fails, so the transaction should be rolled back.

The session needs to be closed afterwards.

None of that was new.

The Python way of expressing it was.

A simplified version of the LinkUp session dependency looks like this:

async def get_session():
    async with session_factory() as session:
        try:
            yield session
        except Exception:
            await session.rollback()
            raise

FastAPI uses the code before yield as setup and the remaining part as teardown.

I understood why the lifecycle worked this way before the syntax itself started feeling natural.

That happened quite a lot while moving to Python.

I was not relearning what a database transaction was. I was learning how the Python ecosystem usually expresses the same idea.

That is a very different experience from learning backend development for the first time.

The actual backend problems did not change much

After the first few days, something else became obvious.

Once I got past the syntax and framework conventions, most of the important questions were the same ones I already knew.

LinkUp uses:

Argon2 password hashing
short-lived JWT access tokens
opaque refresh tokens stored in Redis
refresh token rotation
revocation on logout
HttpOnly cookies
PostgreSQL persistence

The implementation is Python, but the questions are not Python-specific.

When should a transaction commit?

What happens if two registration requests try to create the same email?

Where should refresh sessions live?

Can a refresh token be reused?

How should logout invalidate a session?

What happens if a valid access token references a user that no longer exists?

Python did not remove those problems.

FastAPI just removed a lot of the repetitive framework code around them.

That was probably the biggest change in how I looked at the framework.

At first, fewer lines of code made the whole thing look simpler.

Later I realized that the engineering was still there. The ratio between framework code and application code had changed.

Go made me notice hidden complexity

I think starting with Go affected the way I reacted to FastAPI.

Go made me used to seeing plumbing.

My Go projects tend to be fairly explicit about HTTP handlers, database access, context propagation, errors and infrastructure.

Sentinel goes even further with Kafka consumers, explicit delivery semantics, worker pools and transactional persistence.

I eventually wrote gofro because I had repeated enough of the usual setup that generating part of it became convenient.

Then I moved to FastAPI and suddenly request validation, dependency injection, OpenAPI generation and quite a lot of HTTP plumbing required very little code.

My first instinct was to look for what I was missing.

In practice, the abstraction is usually fine. I just need to know what it is doing.

An async def endpoint can still contain blocking code.

An ORM can still produce an inefficient query.

A dependency can have the wrong lifecycle.

A Pydantic schema can model the wrong contract.

A transaction can still be committed at the wrong time.

FastAPI does not solve backend engineering.

It mostly saves me from writing some of the repetitive parts of it.

I like that distinction much more than simply saying FastAPI is "easier."

There are still things I miss about Go

Python is much more concise for a lot of application code.

There are places in my Go projects where the equivalent Python implementation would probably be much smaller.

There is less repetitive error handling, less manual DTO plumbing and generally less code needed to express common web application behavior.

But explicitness has advantages too.

In Go, this is boring:

if err != nil {
    return err
}

especially after seeing it for the thousandth time.

But I always know where the failure is being handled.

In FastAPI, a small route can depend on several other functions, and part of its behavior may live somewhere deeper in that dependency graph.

SQLAlchemy can express a query very compactly, but I still need to care about the SQL it eventually generates.

Neither approach feels universally better to me.

They just put complexity in different places.

The part I liked most was realizing I did not start from zero

This was probably the nicest part of switching stacks.

My Python fluency was much lower than my familiarity with the backend problems I was trying to solve.

Sometimes I had to look up a Python construction while already knowing exactly what I wanted the application to do.

I could forget the syntax for an async generator dependency while still understanding why the database session needed cleanup.

I could be unfamiliar with a particular SQLAlchemy API while already understanding the transaction boundary I wanted.

I could still be getting used to asyncio while knowing that running Argon2 directly on an event loop was a bad idea once I understood how the loop worked.

The language changed.

The framework changed.

The libraries changed.

PostgreSQL did not.

Redis did not.

HTTP did not.

Transactions did not.

Most of the problems simply appeared in a different form.

I was not learning backend development again.

I was translating what I already knew into another ecosystem.

Do I prefer FastAPI now?

I still do not know.

I am not sure I need to choose either.

I like Go because the system often feels very visible. There is relatively little framework machinery between me and the application.

I like FastAPI because I can describe a lot of application behavior with surprisingly little code.

What changed after moving from one to the other is that I now look at convenient abstractions a little differently.

Instead of assuming the complexity disappeared, I try to understand where it moved.

FastAPI can make backend code feel very simple.

Backend development itself is still the same problem.

SSH died. Spent 3 hours fixing the wrong thing.

Alexey — Wed, 20 May 2026 15:38:14 +0000

Three or four days ago I deployed a Gitea runner, tested it, closed the terminal and moved on. Today, for some reason I don't even remember, I tried to SSH into the server. Nothing.

Opened the browser to check if the server was alive — course project running, Gitea up, Nextcloud up, everything in the k3s cluster fine. Went into VMmanager, rebooted. Still couldn't get in. Closed The Boys and went to fix it through VNC.

nmap first

nmap -p 22 2.27.42.100

PORT     STATE    SERVICE
22/tcp   filtered ssh

filtered — not closed, not refused. Packets reaching the server and getting silently dropped. Firewall somewhere.

Checked the INPUT chain:

Chain INPUT (policy DROP)
1    KUBE-ROUTER-INPUT
2    ACCEPT     tcp dpt:22
3    KUBE-PROXY-FIREWALL
...
8    ufw-before-input

ACCEPT rule for port 22 at position 2. Looked fine.

Wasn't fine.

Where it actually breaks

nft list ruleset | grep -E "22|drop|DROP"

type filter hook input priority filter; policy drop;
...
tcp dport 22 counter packets 0 bytes 0 accept

Zero packets on the accept rule. Connections weren't reaching it at all.

The server has both iptables-nft (kube-router, UFW) and native nftables running at the same time. They fight. kube-router constantly reconciles and overwrites stuff. The output was already warning about it:

# Warning: table ip filter is managed by iptables-nft, do not touch!

The Gitea runner deploy from a few days ago triggered a reconciliation that messed up the rule ordering. I just hadn't tried SSH since then.

The fix

Native nftables table at priority -150. Runs before any filter chains (priority 0), before kube-router, before everything. kube-router only manages the iptables-nft layer — doesn't touch native nftables tables.

nft add table inet ssh_rescue
nft 'add chain inet ssh_rescue input { type filter hook input priority -150; }'
nft add rule inet ssh_rescue input tcp dport 22 accept

SSH came back. Wrote to my friend: "fixed". He said "eee". Made it a systemd service. Two minutes later wrote to him: "SSH died again".

Round two

Back in VNC. Table still there, rules still there, zero packets on the accept rule again.

systemctl start nftables had loaded the saved /etc/nftables.conf which had kube-router's warning comments in it and corrupted the nftables state. Re-added the table manually. SSHed from the server to itself — worked. sshd on 0.0.0.0:22 — fine. Still couldn't connect from outside.

Then checked fail2ban, which had installed itself as a dependency somewhere during all this. It had banned two IPs — bots hammering port 22 the whole time. My repeated failed attempts got caught in the same drop. Ran fail2ban-client unban --all, connected immediately.

Then SSH died again. Disabled fail2ban completely — still nothing.

The actual reason

I was in the middle of debugging nftables chains when I remembered — my VPN is self-hosted on a server in Finland. The day before I'd been complaining to my friend that the VPN was slow. The provider had sent an email:

Due to an increase in brute-force complaints via SSH (port 22), outgoing traffic on port 22 has been blocked for VPS servers located in Finland.

I had read it. I knew about it. And I spent three hours in VNC chasing nftables, fail2ban and kube-router while the answer was sitting in my inbox the whole time.

Turned off the VPN. SSH connected immediately.

What to do from day one

Add the ssh_rescue table before you need it:

nft add table inet ssh_rescue
nft 'add chain inet ssh_rescue input { type filter hook input priority -150; }'
nft add rule inet ssh_rescue input tcp dport 22 accept

Make it a service so it survives reboots and kube-router reconciliation:

cat > /usr/local/bin/ssh-rescue.sh << 'EOF2'
#!/bin/bash
nft list table inet ssh_rescue 2>/dev/null || {
    nft add table inet ssh_rescue
    nft add chain inet ssh_rescue input { type filter hook input priority -150; }
    nft add rule inet ssh_rescue input tcp dport 22 accept
}
EOF2
chmod +x /usr/local/bin/ssh-rescue.sh

After any deploy that touches networking — run nmap -p 22 <ip> immediately.

And read your damn emails.

I got tired of setting up Go projects from scratch, so I built a scaffolding CLI

Alexey — Sun, 17 May 2026 13:37:39 +0000

Before I wrote any actual business logic in my last project, I spent days just setting up the infrastructure around it. Parsing configs, writing Dockerfiles, wiring up Compose, connecting Postgres and Redis, getting the server to actually start. At some point I just gave up and went to play Dota instead.

I'd done this before in .NET — dotnet new generates a working project from a template in seconds. Go has nothing like that out of the box. So I built it.

It's called gofro.

What it does

One command:

gofro new myapi --postgres --redis --grafana --github johndoe --git

You get:

cmd/main.go with graceful shutdown wired up
Config loading via koanf (TOML + env vars, env wins)
Docker Compose with only the services you asked for
Multi-stage Dockerfile
pgxpool connection + goose migrations runner (if --postgres)
go-redis client (if --redis)
Prometheus scrape config + Grafana in Compose (if --grafana)
OpenAPI spec + oapi-codegen setup so you can define your API and generate typed handlers
Generic storage layer — GetAll, GetOne, Create, Update, Delete over pgx
Module path set to github.com/johndoe/myapi automatically
git init if you pass --git

Then go mod tidy and docker compose up -d and you're writing actual code.

Why I built this instead of using something existing

Coming from .NET, dotnet new was just there. Pick a template, get a project, start working. In Go I kept doing the same dance every time: copy the config parser from the last project, rewrite the Compose file, remember how pgxpool initialization works, set up graceful shutdown again.

The last time I did it was for a college logistics platform. I had a tight deadline, hadn't touched Go in a while after a stressful hackathon, and spent way too long just building the scaffolding before writing a single handler. That's when I decided to just solve it once.

gofro is opinionated. It picks the stack for you — chi for routing, koanf for config, pgx for Postgres, goose for migrations, oapi-codegen for API generation. If you want something different, it's probably not for you. But if you're fine with that stack, you skip a day of setup.

The flags

--postgres    pgxpool + goose migrations + generic storage layer
--redis       go-redis/v9 client
--prometheus  Prometheus scrape config
--grafana     Grafana in Compose (enables --prometheus automatically)
--github      sets module path to github.com/<nick>/<project>
--module      full custom module path
--git         runs git init

You can mix and match. Minimal API with no databases:

gofro new myapi --github johndoe

Full stack with observability:

gofro new myapi --postgres --redis --prometheus --grafana --github johndoe --git

What the generated project looks like

myapi/
├── cmd/
│   └── main.go              # graceful shutdown, signal handling, dependency wiring
├── configs/
│   ├── config.toml          # base config
│   └── prometheus.yml       # only with --prometheus
├── internal/
│   ├── api/
│   │   ├── api.swagger.yaml # define your endpoints here
│   │   ├── gen.go           # go:generate directive for oapi-codegen
│   │   └── oapi-codegen.yaml
│   ├── config/
│   │   └── config.go        # struct + koanf loader + DSN helpers
│   ├── database/
│   │   ├── postgres.go      # only with --postgres
│   │   └── redis.go         # only with --redis
│   └── handler/
│       └── server_impl.go   # Server struct, JSON(), Run()
├── pkg/
│   └── storage/
│       └── storage.go       # only with --postgres
├── migrations/              # only with --postgres
├── docker-compose.yml
├── Dockerfile
├── .env
└── Makefile

The workflow after generation:

Define your API in internal/api/api.swagger.yaml
Run make generate — oapi-codegen produces typed models and the server interface
Implement the interface methods on the Server struct
Run docker compose up -d and go run ./cmd/

The generic storage layer

This is the part I use in every project. It's built on go-sqlbuilder and pgx, works with any struct via generics:

// SELECT with optional filter
users, err := storage.GetAll[User](ctx, "users", db,
    func(sb *sqlbuilder.SelectBuilder) {
        sb.Where(sb.Equal("active", true))
    },
)

// SELECT one
user, err := storage.GetOne[User](ctx, db, "users",
    func(sb *sqlbuilder.SelectBuilder) {
        sb.Where(sb.Equal("id", id))
    },
)
if errors.Is(err, storage.ErrNotFound) { ... }

// INSERT
err = storage.Create(ctx, "users", newUser, db)

// UPDATE — skips fields tagged `immutable` (like created_at)
err = storage.Update(ctx, "users", user, db,
    func(sb *sqlbuilder.UpdateBuilder) {
        sb.Where(sb.Equal("id", user.ID))
    },
)

Fields tagged db:"-" are skipped on insert. Fields tagged immutable are skipped on update. No magic, just generics and struct tags.

Install

go install github.com/anxi0uz/gofro@latest

Make sure $(go env GOPATH)/bin is in your $PATH. Then:

gofro new myproject --postgres --redis --github yourname --git
cd myproject

Source: github.com/anxi0uz/gofro

My friend wanted GitLab. He got Gitea and Nextcloud for Obsidian instead.

Alexey — Thu, 14 May 2026 14:22:41 +0000

A friend sent me an article about GitHub potentially getting blocked in Russia and asked me to spin up GitLab. I suggested Gitea — I'd used it at a college hackathon, knew it was lightweight and wouldn't eat half the server. He agreed.

While the deploy was running, I asked him how he syncs Obsidian. He said — plain WebDAV, nothing fancy. Well, server's already open anyway, so I threw in Nextcloud too. Hour and a half later I had both a git host and a cloud storage.

Why not GitLab

My friend originally wanted GitLab. I opened the docs, looked at the requirements — 4 GB RAM just to start — and said no. We don't have a dedicated git server, there's already a project running on it. GitLab would've eaten everything.

Gitea idles at ~150 MB. Actions are compatible with GitHub Actions syntax, so existing workflows move over without rewriting. I'd already used it at a hackathon, knew it worked fine. Suggested it, got the green light.

On Helm

First time I touched Kubernetes was when I had to deploy a college project for a grade. Wrote manifests by hand — Deployment, Service, Ingress, PVC, repeat. I knew Helm existed but never had a reason to dig into it.

Turns out it's like pacman, but for Kubernetes. One values.yaml instead of five hundred lines of YAML, one command — everything's up. Would've lost my mind doing it the old way.

First though, Helm couldn't see the cluster at all:

Kubernetes cluster unreachable: Get "http://localhost:8080/version"

k3s puts the kubeconfig somewhere Helm doesn't look by default. Fix:

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
echo 'export KUBECONFIG=/etc/rancher/k3s/k3s.yaml' >> ~/.bashrc

Gitea

The server already had ingress-nginx and cert-manager with a letsencrypt-prod ClusterIssuer. Set the DNS beforehand — A record git.logiflowadvanced.online → 2.27.42.100.

gitea-values.yaml:

ingress:
  enabled: true
  ingressClassName: nginx
  hosts:
    - host: git.logiflowadvanced.online
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: gitea-tls
      hosts:
        - git.logiflowadvanced.online
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod

gitea:
  admin:
    username: admin
    password: yourpassword
    email: your@email.com

persistence:
  size: 10Gi

postgresql-ha:
  enabled: false

postgresql:
  enabled: true

helm repo add gitea-charts https://dl.gitea.com/charts/
helm repo update
helm install gitea gitea-charts/gitea \
  --namespace gitea --create-namespace \
  -f gitea-values.yaml

Pods came up. Opened the browser — invalid certificate, browser complaining. Checked the Ingress:

NAME    CLASS    HOSTS                          ADDRESS   PORTS
gitea   <none>   git.logiflowadvanced.online              80, 443

CLASS: <none> — the nginx controller just ignored this Ingress entirely. cert-manager didn't issue anything either, so nginx was serving its default self-signed cert.

kubectl patch ingress gitea -n gitea \
  --type=json \
  -p='[{"op":"add","path":"/spec/ingressClassName","value":"nginx"}]'

After the patch, cert-manager issued the certificate and the site opened fine.

SSH

gitea-ssh is created as a headless ClusterIP by default — not reachable from outside. Port 22 is taken by the system SSH, so I needed a NodePort. You can't patch a headless service into NodePort — have to delete and recreate:

kubectl delete svc gitea-ssh -n gitea

kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
  name: gitea-ssh
  namespace: gitea
spec:
  type: NodePort
  selector:
    app.kubernetes.io/name: gitea
  ports:
    - port: 22
      targetPort: 2222
      nodePort: 30022
EOF

Remote looks like this:

git remote add gitea ssh://git@git.logiflowadvanced.online:30022/username/repo.git

Nextcloud

Set up the DNS for cloud.logiflowadvanced.online while Gitea was still deploying.

nextcloud-values.yaml:

ingress:
  enabled: true
  ingressClassName: nginx
  hosts:
    - host: cloud.logiflowadvanced.online
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: nextcloud-tls
      hosts:
        - cloud.logiflowadvanced.online
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/proxy-body-size: "0"

nextcloud:
  host: cloud.logiflowadvanced.online
  username: admin
  password: yourpassword

mariadb:
  enabled: true
  auth:
    password: yourdbpassword
    database: nextcloud
    username: nextcloud

postgresql:
  enabled: false

persistence:
  enabled: true
  size: 10Gi

helm install nextcloud nextcloud/nextcloud \
  --namespace nextcloud --create-namespace \
  -f nextcloud-values.yaml

Same ingressClassName issue — same patch, same result.

After the first login it kept redirecting to /login/cleary?=1. Nextcloud didn't know its own external address:

kubectl exec -n nextcloud deploy/nextcloud -- \
  php occ config:system:set overwriteprotocol --value="https"
kubectl exec -n nextcloud deploy/nextcloud -- \
  php occ config:system:set overwrite.cli.url \
  --value="https://cloud.logiflowadvanced.online"

Obsidian

Created separate users for me and my friend. WebDAV URL per user:

https://cloud.logiflowadvanced.online/remote.php/dav/files/username/

In the RemotelySave plugin: type — WebDAV, URL, login, password. Works.

The thing that ate most of my time wasn't Gitea or Nextcloud — it was ingressClassName. Neither chart sets it automatically, and without it nginx just ignores the Ingress completely. cert-manager doesn't issue anything. Browser shows self-signed, you stare at the logs, pods are all Running, no errors anywhere.

Run kubectl get ingress -n <namespace> right after deploy. If CLASS says <none> — that's your problem.

The other non-obvious one: gitea-ssh is headless and you can't patch it into a NodePort — you have to delete and recreate it. Spent a few minutes trying to patch it before actually reading the error.

With Nextcloud and MariaDB — if MariaDB didn't come up on the first deploy or you uninstalled and reinstalled, helm will complain about credential mismatch on upgrade. Just helm uninstall + kubectl delete pvc --all -n nextcloud and start fresh, it's faster than untangling the creds.