2024 In Review

I have never written an annual report on my work before, but since I also spend time on open source projects, I decided to write one (maybe it will become a tradition).

Programming Languages and Tools

Rust

The main change in my work was the active use of Rust in the second half of the past year. I mainly write in Rust where I used to write in Go. Overall, the language is more complex than Go and sometimes confusing with the borrow checker, but in general, it is quite convenient to write – I can't say that the speed of writing code significantly decreases compared to other languages – in general, the approaches that need to be used are roughly the same for all languages.

What I like about Rust:

• crates ecosystem – many libraries available and can be installed in one command
• single binary file and low memory consumption
• ability to write applications of various directions (web-app, tui, gui, parsers, etc.)

What I don't like about Rust:

• slow Docker image builds (my solution)
• a large number of sub-dependencies for web projects (any project on axum will contain 200+ dependencies)
• use of C-binding in "core" libraries like rust-tls (ring) – complicates cross-compilation because you have to maintain a C toolchain as well (fixed through Zig build, but it looks monstrous)
• lack of finished solutions for serde (default values, partial initialization)

It's nice to write various parsers in Rust, and then they can be coolly parallelized (with rayon or tokio), in Go I had a problems with this due to immutable strings and lack of GC control.

Main languages

The main languages remained the same: Python, NodeJS/TypeScript. I also actively used NextJS on several projects, overall I was satisfied – it's faster and easier to write frontends, but it's quite difficult to find problem's solutions sometimes (many Github issues, often they are open, and the comments contain answers that no longer work).

The main fail of the year in the NodeJS ecosystem is the update of ESLint from 8 -> 9 – completely incompatible API with all existing plugins, the configuration file became larger and more complex – it's not clear why the authors made this diversion. I also tried Binome as an alternative to ESLint / Prettier, but it's completely different and you need to redo a lot, and not everything you need is available at the moment, I continue to monitor this project.

I also tried Zig, but didn't write anything serious. Personally, I didn't understand why this language is needed. At the time I tried it, there was no even package manager yet. Obviously Zig is a competitor to C (not Rust), but C is now much richer in its toolkit and support in other languages. The only thing I can point out is zig build which can build C projects too and does it with less problems.

About LLM

I want to separately note about use of GPTs in work. I have an opportunity to do business tasks in the format of parsing and analyzing texts. What a couple of years ago was done with a bunch of regex, etc., can now be done through GPT and it works great! I tried using local Llama2, but it would have taken weeks to process my data on M2 chip. I redo the solution to use ChatGPT API and processed the data in a couple of days. It's good that all LLMs support openai-client API in Python, so the code almost didn't need to be rewritten. Among the disadvantages of LLama-like aka local running, I also want to note the lack of support for Function calling (to guarantee to bring the LLM response to the desired structure, in my case). The last time I did something serious was more than six months ago, maybe a lot has changed. I also tested gemma2, the inference speed was impressive, and the quality of the answers was not much worse.

As local runners, I used llama.cpp and https://github.com/ml-explore/mlx – the second one uses Mac resources better (checked through macmon), but new models come out a bit slower on it. Some people use ollama, but I didn't understand why, considering that it has an incompatible client with openai-client API and slower performance.

Blog

Initially, I ran my blog only on Medium, but it's not comfortable to write text there – it's much easier to write and edit in the same editor where I write code (plus my editor has Copilot). Therefore, I decided to move my blog to a static page generator. As an engine, I chose Zola (Rust-based, simple, and fast); I also tried Astro, but it's too big and complex for such a simple task. Overall, Zola is good, but it lacks some filters in templating, not enough flexibility in file structure, and not all popular languages are supported in code highlighting.

I left Medium as a fallback for old readers. I also made a cross-post on dev.to, but there are very few views. Medium is disappointed with the lack of the ability to automate cross-posting (they have a deprecated API, but it does not support updating articles and code highlights). Most likely, it's possible to reverse engineer how they save articles through their editor (changes are sent as diffs). Does anyone else need this besides me?

New projects

Last year I launched several new projects (in chronological order):

• timewiz.cc (react, webapp) – a tool to find common working hours for distributed teams (an alternative to worldtimebuddy.com, I don't remember exactly why I made my own version). Score: fail.

• ecloop (c, cli) – secp256k1 elliptic curve implementation for Apple chips. Mainly an educational project to understand cryptography, SIMD optimizations, etc. Added several standard use cases for finding Bitcoin keys by brute force, but mathematically this is pointless, so as I said earlier, this is more of an interesting educational project (and I wanted to publish a series of articles on cryptography, but didn't have time last year, I hope to do it this year). Some people use this project in hopes of finding something. I also had ideas to add Metal API support – but that takes a lot of time, so I switched to other projects. Score: okay.

• macmon (rust, tui) – TUI program for monitoring MacOS performance (CPU/GPU/ANE usage, RAM, temps). My first public Rust project, a lot of reverse engineering of how powermetrics in MacOS works, C-bindings. The project is used both by users in TUI mode and by other projects in headless mode (JSON metrics). Score: success.

• ghstats (rust, axum, htmx, webapp) – a small self-hosted service that collects and saves GitHub repo statistics, has several built-in views and an API to retrieve all statistics. The project started well, but then kind of fizzled out in terms of issues and feature requests, most likely users are satisfied with the current functionality. I would like to add more visualizations, but I haven't had time for that yet. The main thing for now is that the statistics are collected and saved, so something can be done with this in the future. Technically, I like that the service consumes only 10 MB of RAM. Score: okay.

• ogp (rust, axum, api-service) – a small project for generating OGP articles preview as a service. Used in this blog (you can check by sending this article to your friends in a messenger or social network 😀). Worked with the SVG standard (I thought it was more adequate) and generating PNG from SVG. Score: more of fail.

• compose-updater (python, api-service) – a project for updating the version of a docker image from CI in a closed infrastructure. I didn't find anything ready to solve this task. There is watchtower, but it's more about keeping up-to-date docker images, rather than updating a specific one, and also I couldn't run it with ECR repos – the author made a super minimalist image in which I couldn't enter through sh to debug. The second alternative is umputun/updater, but it's more about running different shell commands through a webhook. I chose Python because I didn't find a Rust crate for Docker API (most likely it exists, I just didn't understand which one is really good). Score: – (I didn't promote the project anywhere, I use it for my own purposes for now).

Existing projects

• twscrape (python, lib) – a library for parsing data from X/Twitter. Mainly the project is in maintenance mode, I check its functionality every few months, and there have been no requests for new features.

• apigen-ts (typescript, lib) – a library for generating TypeScript client OpenAPI spec. I continue to use it in my projects; I also see that there are a small number of other users. My relevance has slightly decreased due to the NextJS/TRPC stack, but in other configurations (e.g. FastApi + React), it is still relevant, so use it. Periodically there are requests to fix something or add a feature.

Abandoned projects

I also have several projects that I didn't finish and abandoned. I'll write about them here, maybe someone needs something from this, then I can return to on it.

• anypost – cross-posting between static blog, Medium, Dev.to, Substack, and collecting joined article statistics. I abandoned the project due to the lack of an API at Medium – the only solution I found was through protocol reverse and session cookie.

• bcloop – a program to walk Bitcoin-like blockchain and compile the final UTXO list. I abandoned it because I couldn't figure out how to effectively create a HashMap-like structure with external memory support to run on computers with small RAM (less than 32GB, LOL).

• tg-watcher – a program that listens to all incoming messages in Telegram (chats & channels) and saves their history in a separate database with the ability to search through these messages later. I have the first version that just saves messages in json files, but I didn't develop the idea further.

Devices

I'm not a big tech geek, but I want to say about a few devices which made my life easier:

• MeLE Fanless Quieter 4C – a small fanless PC for homelab, I use it as a native Linux via SSH for testing various programs and keep some cron jobs on it for data collection. It's cool that it's silent because it has no fan, but it heats up a lot under load. The performance of the N100 chip surprised me – only 2x slower than M2 chip (tested with ecloop bench, single core).
• Promate HDMI-PD100 – nice cable format to connect a monitor and my Mac. One cable is needed to charge and transmit the video stream. Before that, I used hubs, but it's a big block. I'm glad I see fewer wires.

At Glance

Over the year, 8 articles were written, totaling 5203 views. Top-3 articles:

• Table sorting and pagination with HTMX – 1714
• macmon – Mac Usage Monitor in Terminal – 953
• Setting up WireGuard VPN at AWS with Terraform – 642

Data aggregated from this site (with umami), Medium, and Dev.to. Script.

Public GitHub projects – totaling 312 commits, 49 issues closed, 43 releases created, and 1394 stars gained. Top-3 repos:

• twscrape – +802 stars / +52515 views
• macmon – +420 stars / +6899 views
• ghstats – +128 stars / +2757 views

Data aggregated with ghstats.

I hope I will see a different top with new projects this year 😀

Donations

In twscrape repository I was asked to make it possible to send coffee as donations. Over the year I received 38 cups of coffee from 10 people. I spend these coffees on an annual Parallels subscription, which allows me to test some apps on Windows. Also, I partially paid from this fund for MeLE and VPS, which I used to test apps as well. Many thanks for supporting my work!