Krokiet and Czkawka have long ceased to be simple duplicate-finding apps. Over time, I needed more and more functionality, so I added features without much planning, which caused the project to grow in a rather chaotic way. I needed a break to sort some things out in my head and in the code. I managed to do that partially, so in this version I started adding new tools. As a result, the current release is the largest so far and at the same time the most interesting.
Usually, after releasing a new version, many comments appear saying I broke some modes, the app stopped launching or asking why basic features are still missing (like filtering results after a scan or a new button to select files created after 11 PM on the 23rd Sunday of an odd numbered year).
It was no different after the previous release. Amid a sea of regressions, like results not showing in the CLI, my attention was caught by freezes and a microscopic file preview, happening in parts of the code I hadn't touched recently. After a long investigation it turned out the bug wasn't caused by me, but was linked to one of the libraries I use. And that library is…
GTK - bugs and stability
Over five years ago, when I started developing this program, one of the main challenges was choosing the frontend Czkawka would use. At that time, Rust GUI libraries were still in their infancy, trying to find their niche. Choosing between QT or GTK seemed like a reasonable solution. Ultimately, I went with GTK 3, mainly because I believed it would be the most stable on Linux, which was my primary programming platform and initially the only supported system.
Apart from my own mistakes and some binding issues, the application ran fairly stable on Linux. In the meantime, GTK 4 was released, which required rewriting a significant part of the application. After many struggles, I also managed to compile the application for Windows and macOS. That's when the problems started in earnest. Compilation issues, random crashes that didn't occur on Linux and problems with icons and rendering made me start questioning whether choosing this library had been the right decision.
However, with the merge request - https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/8627 - the previous idyll ended. The change affected GtkImage, which I had been using since GTK 3, and caused the image preview in every version of the program using GTK 4, up to the previous release 10.0, to shrink to the size to a few pixels. According to the description of the change, this was a breaking change and the authors expected some projects to encounter issues when updating. Unfortunately, Czkawka was affected. GTK 4.20 is now present on most of recent systems, meaning their users can hardly use the application.
The current version (11.0) includes a fix that replaces GtkImage with GtkPicture, straightforwardly solving the problem. However, this change, aside from some minor inconvenience, introduced a bigger issue - a problem with my confidence in GTK stability. Unlike Krokiet, where the Slint library is embedded in the binary and handles the entire interface, Czkawka relies on dynamically linked GTK, which is prone to changing behavior randomly due to a bug or a bug fix (as happened in my case), leaving both users and developers confused. I assume this change affected a relatively small percentage of applications, but for me and those using Czkawka, it was painful.
EXIF File Cleaning - New Tool
Somewhat spontaneously, I decided to add a new EXIF data removal tool to the program. I had personally used this feature only occasionally, and until now relied entirely on https://github.com/szTheory/exifcleaner, which is no longer maintained and also uses Electron (terrible, right?). It also bothered me that the tool didn't allow previewing changes before applying them.
For several versions, the number of tools in the program stayed at 11, and I focused mainly on improving them and simplifying the logic of adding and managing tools later. I finally decided this was a good moment to expand the set.
I knew Rust had crates for handling EXIF data, but they still lacked many features compared to the popular exiftool. After a quick review, I considered only three libraries that had been updated recently:
- nom_exif
- little_exif
- guf_exif
I rejected guf_exif because it didn't have a general interface for managing all tags. I already use nom_exif to read image rotation so images display correctly. In the past, I reported several bugs in it using a fuzzer, which made it rock-solid and crash-free (at least in my cases). The problem was that it couldn't modify data, and the related issue hasn't been updated since 2024.
That left little_exif as the only option. While learning how EXIF works, I also learned how to use little_exif. Getting it to work wasn't hard. I only needed to extract the tags from a file - their name, ID and group, and allow them to be removed. I make sure all libraries working with external data are as crash resistant as possible, because in the past such operations often caused problems. Additionally, every risky operation is wrapped in a panic::catch_unwind block to safely catch any panics.
As usual, I approached testing quite thoroughly, which resulted in several issues being created, dozens of crashes, and the realization that the world doesn't end with EXIF, other formats like XMP or iTXt also exist. Without deep knowledge of the EXIF format, I was able to fix crashes only up to the point where the fuzzer stopped detecting them after testing around 1 million inputs, and I modified a few APIs that were inconvenient for me to use.
In its current form, the tool allows removing EXIF data from several popular image formats. It supports fewer file formats and recognized tags than exiftool, but it is enough for my needs so far. I would be happy to see support for other metadata formats, so I am keeping an eye out for libraries that could expand the tool's capabilities in the future.
This tool, like most of the features described below, is only available in Krokiet and the CLI. Czkawka GUI, which is now in maintenance mode, will not receive this mode or any other major functionality (unless someone implements it themselves, which I honestly encourage).
Gtk4-rs and mysterious bugs
From the very first versions, Czkawka has used GTK to build its GUI. As is well known, GTK is written in C, and using it in Rust requires either writing the bridging code yourself or relying on an existing library. As a beginner, trying to build C <-> Rust interoperability from scratch would have been asking for trouble, so the only reasonable choice was a new and experimental gtk-rs library. This library provided fairly good capabilities for calling C functions from Rust, but the combination of my skills at the time and the early version of this crate, meant I had to deal with many strange bugs.
Over time, the code, despite being written in a not-so-great style, became "rock-solid" and increasingly difficult to break on a typical Linux system. Due to problems on other platforms and difficulties in further development, I shifted most of my efforts to Krokiet, and marked the GTK application as deprecated, meaning I did not plan to implement new features, tools, or modes in it. However, most(I think) Czkawka users still rely on the GTK frontend, so I couldn't leave them on their own and decided to fix emerging bugs as much as possible.
I started cleaning up the code a bit to make it easier to fix issues, and everything worked fine until I created a single test that, like other tests, checked the function for removing items from a list that weren't linked to other items. The logic was simple:
- Iterate over the model
- If the iterator points to a regular item, save it to a vector
- If the iterator points to a header or the model ends, then if the vector has only one item, remove both the previous header and that item (for example, if there are two similar images, and one is deleted from disk, the other must be removed from the list as well, because keeping an item in the GUI without a linked counterpart is pointless)
This test would simply hang. I started analyzing why almost identical tests passed correctly while this one did not. The most troublesome part was that the test usually worked fine when compiled on its own, but failed when compiled together with others. In the meantime, on an older manually compiled version of Czkawka, I began noticing occasional freezes after deleting files, though at the time I assumed these were regressions caused by other changes in the code.
My first hypothesis was incorrect API usage, for example operating on freed memory or an invalid iterator. However, neither AddressSanitizer nor Valgrind reported any issues. While preparing a minimal example to report a bug in gtk4-rs, I noticed that the problem occurred mainly in release mode and when running a larger number of tests.
After reporting the issue in the gtk4-rs repository, I also opened a report in the Rust repository, since the problem started appearing with version 1.86.0. User yvt, followed by ds84182, determined that the code in gtk4-rs was unsound. Specifically, in the iter_next function, the iter parameter was an immutable reference without interior mutability, even though its state was in fact being modified internally.
#[doc(alias = "gtk_tree_model_iter_next")]
fn iter_next(&self, iter: &TreeIter) -> bool {
unsafe {
from_glib(ffi::gtk_tree_model_iter_next(
self.as_ref().to_glib_none().0,
mut_override(iter.to_glib_none().0),
))
}
}
The fix, which changes iter to the type &mut TreeIter, was included in the new release, gtk4-rs 0.11.0. Its publication was one of the blockers for releasing the new version of the application.
Video optimizer - new tool
Codec Change
Under constant pressure to keep enough free disk space to store random memes and videos, I decided to focus on reducing the size of the data I already had. I had several large MJPEG video files, and converting them to a more efficient format seemed like the simplest solution. This allowed me to shrink their size several times over and save more than a dozen gigabytes of space. I could have used ffmpeg manually for each file, but searching for such videos and converting them one by one would have been tedious. That is how the idea for a new tool was born.
Like the other tools, the logic is simple. Files that look like videos based on their extension are collected. Then, ffprobe is used to retrieve their metadata. The list of files is displayed in the GUI. The user can select files and "optimize" them by changing the codec and optionally limiting the video resolution using ffmpeg and its specific options.
A simple Python script would probably work just as well, but I prefer having everything in one place, with the ability to manually review and select the results.
Cropping Black Bars and Static Content
A slightly more advanced mode in this tool is video cropping.
While going through my collection of random video clips, I noticed that many of them have random black bars on the sides or colored, static borders. Such frames not only waste valuable kilobytes, but also make the video less comfortable to watch.
One thing I truly dislike is watching a horizontal video that has black bars at the top and bottom. In full screen, it ends up using only a fraction of the available space. It just looks very very bad.
There are programs where you can simply select the area you want to crop to, but they usually add their own logo to the video and or only work on one file at a time, which makes the process time consuming. So I decided to start with a simple mode that automatically removes black bars without adding anything to the video. To keep things straightforward, the logic works as follows:
- Videos are first detected and their metadata is retrieved using ffprobe, such as resolution, bitrate, codec, and so on.
- Next, ffmpeg is used to extract the first frame of the video, which is then used to override the dimensions from the metadata. ffprobe does not account for rotation and sometimes reports dimensions that differ from the actual frame.
- For each video, up to 60 frames are extracted at intervals of at least 0.25 seconds. For a 15 second video, this means a frame every 0.25 seconds. For a 60 minute video, roughly one frame per minute.
- On each frame, a bounding box of the actual content is calculated, so that everything outside the rectangle is treated as black bars. The process runs from four directions, top, bottom, left, and right. Starting from the outer edge, lines are checked where at least a given percentage of pixels, by default 90 percent, have all three RGB components below a defined threshold, by default 20. If the condition is met, the next line is checked. If not, the analysis from that direction stops and the edge of the rectangle is determined.
- The next steps repeat the same procedure for subsequent frames, with one important rule, the content rectangle can only expand, meaning the cropped area can only shrink. This follows a simple principle: if an element appears in a given area on even one frame, it cannot be removed, even if that area is dark on other frames.
- If the final rectangle matches the full frame size, it means cropping is not needed because there is nothing to remove. Otherwise, ffmpeg performs the actual crop operation to the calculated dimensions.
Some videos are more problematic. They contain white subtitles or backgrounds in other colors. At first, I considered extending the algorithm to detect colors other than black, for example white. The issue was that the number of analyzed colors and edge cases would quickly grow, making the solution harder to implement and still not covering every situation users might expect.
I also considered a more complex approach: extracting 60 frames and comparing each row and column against the previous frame. In practice, this made the comparisons significantly more complicated.
Instead, I decided to compare all extracted frames to the first frame using a simple function that calculates pixel differences. For each RGB component of a pixel, the absolute difference is computed relative to the pixel at the same coordinates in the first frame. For example: RGB(50, 0, 50) and RGB(75, 50, 25) results in RGB(25, 50, 25). This produces a new image where pixel values represent how much the frame differs from the first one.
Since areas that change very little compared to the first frame become very dark in this difference image, I could reuse the previous cropping logic. Unchanged regions, regardless of their original color, effectively turn black in the difference image and can be cropped using the same simple threshold-based method.
Initially, I was not fully convinced this would work well. In practice, the results exceeded my expectations and it performs very well on my collection of random videos.
Of course, this tool isn't perfect and has some flaws and limitations:
- Sometimes it leaves thin black lines, probably because quality loss causes the image to blur, making these lines not completely black anymore so algorithm doesn't remove them.
- Static parts mode can also crop parts of the video that are usefull. The algorithm doesn't know whether a static section is important or not. For example, in a moving beach scene, the upper part of the frame often shows almost static sky, which is frequently cropped out, leaving just the moving area of the beach where the algorithm detects motion.
- Transcoding is necessary, which can reduce quality and sometimes even increase file size. You can use H.265 conversion to reduce the size, since the video has to be transcoded anyway.
- If changes in certain parts of the image happen very rarely, sampling 60 frames at regular intervals might miss them. The algorithm could mistakenly consider that spot safe to crop. This is a trade-off between processing speed and reasonably good results. Checking every single frame would almost guarantee nothing important is cropped, but for longer videos it would increase processing time linearly. For instance, a 1-hour video might take around 5 hours to process, instead of just a few seconds or minutes as it does now.
Bad names - new mode
Another new mode that nobody asked for and that probably few people will use is searching for potentially undesirable filenames.
This includes filenames with uppercase extensions, emojis, spaces at the beginning or end, non-ASCII characters, character limits, or repeated sequences. All of these can be searched for.
To keep names reasonably sensible, the steps are applied in a strict, unchangeable order. This means a file like " 🚀 świstak .txt" won't suddenly turn into "rocket swistak.txt" when emojis, non-ASCII characters, and extra spaces are removed, but it will become "swistak.txt".
These modes came to mind first when I looked at the rather strange filenames on my computer, so I might expand the list of searchable/replaceable elements or tweak the existing rules in future versions.
Preview of video parameters and frames
A more interesting feature in this version is the added ability to display parameters of the found video files. In addition to the file path, size, and duration, the GUI (added both to the Krokiet interface and the GTK version, which, as a reminder, is in maintenance mode) now also shows information such as video length, bitrate, codec, framerate, and pixel dimensions.
Using an external crate, all this information is read through ffprobe, so ffprobe needs to be installed alongside ffmpeg to access it. The earliest issues requesting this feature date back three years, but no one addressed them, and I didn't feel particularly motivated to tackle it either.
The need arose when my collection of random videos grew and I couldn't keep up with removing duplicates. Without having these parameters visible directly with each video, I had to manually check each one by playing them to identify which were lower quality or smaller in size.
Another improvement in the video view was adding preview generation for the results. Although it sounds simple, it wasn't. To create the previews, I used ffmpeg, similar to how it's currently used for hashing and comparing video files. This is a time-consuming operation (even hundreds of milliseconds), so generating previews on the fly would cause noticeable delays when clicking. Additionally, this would either have to discard these images and regenerate them on each click or store them in some kind of cache, which would complicate the logic.
It would also be problematic that, when clicking, either the entire GUI would freeze until the preview finished processing (as currently happens for normal images, excluding ffmpeg operations), or it would have to be loaded in a separate thread, adding complexity.
I decided that handling this entirely on the GUI side was unnecessarily difficult, so I moved it to the core. Now there is a new scanning step that, if the user has opted to generate previews, creates a preview for each file that is similar to others. The filename, modification time, file size, and the frame location are hashed so that the same parameters always produce the same path, avoiding regeneration in subsequent scans. Alternatively, this could be stored in a cache, but then you would lose the ability to choose which frame to extract.
There is also an option to generate a video preview as a single image made up of 9 frames taken from different points in the video. Since this requires generating 9 frames instead of 1, it is much slower.
Image previews are available only as an option in Krokiet. The CLI version has preview generation hardcoded as disabled (because, honestly, it doesn't make sense there), and the GTK version, well, you know the situation. In crop mode, on thumbnails are also visible rectangles, which indicates the area where the video will be cropped
To avoid excessive memory and CPU usage, extracting previews from a file with ffmpeg is limited to a single thread per video. By default, ffmpeg would create as many threads as are available on the system, which could lead to out-of-memory errors. During testing, the program crashed several times before I implemented this limit.
Other changes
GTK Czkawka dialog about Krokiet
I was surprised that so few people know that Krokiet is a new and improved version of Czkawka, even though I tried to communicate it in various ways. So, following the "because I know better" principle, I decided to add a new dialog window that explicitly tells users this. Its display depends on whether a special file exists indicating the window has already been shown (it should appear only once) or whether the user has set the magic variable CZKAWKA_DONT_ANNOY_ME.
New Krokiet Logo
The previous logo was a quick, hand-drawn unicorn in a simple raster image. That worked when the app was experimental. Today, since Krokiet is widely seen as an almost perfect application, it needed an almost perfect logo. The new logo depicts a unicorn with a shield, helmet, and a banner featuring a Krokiet. Why this design? I'd answer: why not (though it probably has something to do with the Hussite Trilogy audiobook I was listening to at the time). It's also in SVG format now, so it scales nicely.
Delayed cleaning of outdated cache records
The usual application flow is:
- Collect file list
- Load cache
- Process files
- Save cache
- Display results
The longest operations are usually processing files and collecting the file list, since both access the disk and check many small files. Loading and saving the cache operates on individual files, so it should be fast, right? Not entirely - there's a default option to remove outdated records to keep the cache size under control, minimizing load and save times. During each load operation, all records are checked for existence, modification time, and size. This is usually quick, but on external drives connected via Samba or on HDDs, operations on many small files can take unnecessarily long.
To address this, I decided to delay these operations so they run as rarely as possible. Now, during cache loading, outdated records are checked only once per week. Users also have the option to manually clear the cache at any time if needed.
Ability to scan single files
Since the beginning, the only way to scan was to select a folder first. This makes sense because it covers all use cases and allows scanning any combination of files. In some situations, this required creating new folders and moving files into them to scan. Some users wanted to check just a single image/video/file for duplicates without creating a folder. Previously, this required manually creating a folder, adding the file, and setting it as a reference folder. Now, it's possible to add individual files directly, set them as reference paths, and check them without moving them. Personally, this didn't bother me much, and over the years I simply got used to testing files by moving them around on the disk.
Sounds
A subtle new feature in this version is adding sounds to Krokiet when scanning finishes. During long scans, it was easy to get distracted and miss the end. For now, I added only one test sound to see if it's useful, and it may be expanded in the future. The scan-complete sound is a noise I made with my jaw, which probably isn't very pleasant, so users can set any custom sound via an environment variable.
AI translations
Almost all Rust code in the project, build systems, and scripts are developed mainly by me, since I wrote most of it and understand it fairly well. The only exception is translations, where I'm responsible for the majority of texts, but only for English and Polish. Other languages, like Italian or French, were actively maintained in the past; some are currently being updated, such as Brazilian Portuguese, while others were created from scratch using online translators in response to requests for that language. Currently, dozens of languages are supported, amounting to tens of thousands of words, and every release requires updating dozens or hundreds of new phrases. The scale may not be huge for some projects, but for me it is, and faced with so many changes, I had to choose between:
- Leaving translations partially incomplete
- Removing/hiding incomplete translations
- Translating missing texts manually
To prevent repeated requests for adding an existing language and to avoid exposing users to incomplete texts, I decided - without a few hundred or thousand dollars to hire translators or a dedicated team - to first manually translate certain phrases using publicly available translation websites, then use a script leveraging the zongwei/gemma3-translator:4b model. It's not fast (a few words per second on my machine), but it seems to produce better results than other methods.
Refactoring Czkawka GTK code
Saying the code in Czkawka GTK isn't very tidy would be an understatement. I wrote it mainly when my Rust knowledge was fairly primitive. Over time, the project grew, and because of its size, I never fully rewrote the GTK code in a cleaner style, only patching what I could. In this version, I've done a bit more of the same, extracting operations on operators into shared functions and grouping related elements (e.g., scrollview, treeview, and gestureclick), making the code somewhat more approachable.
I had considered preparing the code for potential GTK 5 support, but due to the extensive changes required (for example, replacing TreeView, which is the base for all models), I have no plans to pursue it. However, anyone brave enough could take it on - I have no objection to using AI extensively for this task, as long as the code is reasonably good, stylistically consistent with the rest of the program, and passes a series of my reviews.
The End
Repository - https://github.com/qarmin/czkawka
Download files with the changelog - https://github.com/qarmin/czkawka/releases
License - MIT/GPL depending on the program (in short, it's free)
Top comments (0)