GSOC 2025 Report: AprilTag, FAST, and ORB Detectors

This project adds three detection/feature modules to kornia-rs ecosystem: a full AprilTag Detector, FAST Corner Detector, and an ORB Detector. The implementations are CPU-focused, modular, well-documented, and include examples, tests, and benchmarks.

Goals

1. Implement AprilTag detection and decoding pipeline (configurable, mult-family)
2. Implement newer complete FAST corner detection
3. Implement ORB Detector
4. Provide examples, tests, and benchmarks; compare to common libraries when possible

AprilTag Detectors

An AprilTag is a fiducial marker: a flat, square pattern that encodes a small binary code inside a black/white border. Once detected and decoded in an image, a tag lets you compute camera pose relative to the tag if you know the tag's physical size and the camera intrinsics. They are widely used for camera calibration, robot localization, AR markers and many tasks where 2D to 3D reference is useful.

Why AprilTags are useful:

• Robust detection under many viewing angles and lighting conditions.
• Provide direct geometric constraints for pose estimation
• Small payload, easy to print and deploy

During the first half of the coding period, I primarily worked on creating the kornia-apriltag crate that focuses on AprilTag Detection. It breaks down the entire decoding pipeline in the following steps:

1. Adaptive Threshold
2. Connected Components (aka Union Find)
3. Gradient Clusters
4. Quad Fitting
5. Tag Decoding

The API of kornia-apriltag is a well documented crate which is easy to use while being super customizable. The following code snippet is the easiest way to use the crate:

use kornia_apriltag::*;

let config = DecodeTagsConfig::all();
let mut decoder = AprilTagDecoder::new(config, image_size)?;

let detections = decoder.decode(&img)?;
// Do whatever you want with the detections, or just plot them

But you can go much further by choosing which Tag Families to consider detection and their individual configs and even replace/customize any step of the pipeline as all of the functions are well-documented and exposed publicly.

You can find a complete example in examples/apriltag

PRs associated: #380, #384, #406, #410, #419, #420, #429, #433, #435, #436, #438, #440, #444, #458

Issues associated: #408, #450

Resources

• AprilRobotics/apriltag
• powei-lin/aprilgrid-rs
• https://april.eecs.umich.edu/papers/details.php?name=olson2011tags
• https://april.eecs.umich.edu/papers/details.php?name=wang2016iros
• https://april.eecs.umich.edu/papers/details.php?name=krogius2019iros

ORB Detectors

ORB (Oriented FAST and Rotated BRIEF) is a feature detection and description algorithm optimized for speed and robustness. It combines FAST for keypoint detection, orientation assignment for each keypoint, and BRIEF-like binary descriptors that are rotation-aware. ORB is widely used for feature matching, visual odometry, and as input to SLAM pipelines.

The second half of the coding period, I implemented an ORB Detector in the kornia-imgproc crate.

use kornia_imgproc::features::*;

let mut orb_detector = OrbDetector::new(OrbDetectorConfig::default(), image.size())?;

orb_detector.detect(&image)?;
let (descriptors, _) = orb_detector.extract(&image)?;
let Detection { keypoints, .. } = orb_detector.get_detection();

The matching of the descriptors is also as simple as the detection:

// Your detection logic

let matches = match_descriptors(&first_descriptors, &second_descriptors, None, true, None);

// let's plot them
let mut coords = Vec::new();
for &(i1, i2) in matches.iter() {
    let kp1 = &keypoints1[i1];
    let kp2 = &keypoints2[i2];

    coords.push([kp1, kp2]);
}

// Your plotting logic

PRs associated: #457 (UNDER REVIEW)

Resources

• scikit-image
• OpenCV
• https://ieeexplore.ieee.org/document/6126544

FAST Corner Detector

FAST (Features from Accelerated Segment Test) is a high-speed corner detector. For each candidate pixel, FAST compares the intensities of a circle of surrounding pixels to the center pixel. If a contiguous arc of pixels is all brighter (or all darker) than the center by a threshold, then the pixel is classified as a corner.

During the second half of the coding period while implementing ORB Detector I needed complete implementation of FAST corner detector. So, I implemented a newer complete version of FAST Corner detector in kornia-imgproc crate.

use kornia_imgproc::features::FastDetector;

let mut fast_detector = FastDetector::new(image_size, threshold, arc_length, min_distance)?;
fast_detector.compute_corner_response(&image)?;

let keypoints = fast_detector.extract_keypoints()?;
// Go ahead plot the keypoints

You can find a complete example using FAST Corner Detector in examples/fast_detector.

PRs associated: #460 (UNDER REVIEW)

Resources

• scikit-image
• https://www.sciencedirect.com/science/article/abs/pii/S0262885697000565

Timeline

The following is the list of all the PR/Issue(s) created by me during the GSoC Coding Period.

Goal Classification

• AprilTag
• ORB Detector
• FAST Detector
• Other

	Date	Title	Type	Goal	Link
1.	Jun 2	Add kornia-apriltag crate	PR	AprilTag	#380
2.	Jun 2	Use std::hint::black_box instead of criterion::black_box because it’s deprecated	PR	Other	#381
3.	Jun 2	Update ONNX example to use ort 2.0.0-rc.10	PR	Other	#382
4.	Jun 4	Add Adaptive Threshold Binary functions	PR	AprilTag	#384
5.	Jun 8	Fix kornia-io docs.rs not building	PR	Other	#388
6.	Jun 20	Add Connected Components for AprilTag detection	PR	AprilTag	#406
7.	Jun 21	Safe Implementation of Image TileIterator yielding borrowed slices	Issue	AprilTag	#408
8.	Jun 21	Implement Zero-copy Image conversion in kornia-py	Issue	Other	#409
9.	Jun 23	Add gradient cluster detection	PR	AprilTag	#410
10.	Jun 26	Replace clippy-action with direct cargo clippy command	PR	Other	#412
11.	Jun 29	Add Quad Fitting	PR	AprilTag	#419
12.	Jul 1	Add AprilTag Decoding	PR	AprilTag	#420
13.	Jul 5	Remove license-file from smol_vlm example	PR	Other	#424
14.	Jul 10	Add support for multiple AprilTag Families	PR	AprilTag	#429
15.	Jul 16	Add TagFamilyKind::all method and use it in DecodeTagsConfig	PR	AprilTag	#433
16.	Jul 18	Reset sharpening buffer for every Quad	PR	AprilTag	#435
17.	Jul 18	Add AprilTag detection example	PR	AprilTag	#436
18.	Jul 19	Add AprilTag Benchmark	PR	AprilTag	#438
19.	Jul 22	Update apriltag-imgs submodule URL	PR	AprilTag	#440
20.	Aug 2	Add Image downscaling support in AprilTag Detection	PR	AprilTag	#444
21.	Aug 27	Implement ORB Detector	PR	ORB Detector	#457
22.	Aug 28	Add benchmark aprilgrid-rs	PR	AprilTag	#458
23.	Aug 29	Implement newer FAST Corner Detector	PR	FAST Detector	#460
24.	Aug 30	Add Gaussian Blur support for u8 images	Issue	Other	#461
25.	Aug 30	Add HarrisResponse support for u8 images	Issue	Other	#462

Challenges faced

• My university end-term exams overlapped with the first half of the coding period, and I initially expected this to limit my time. In practice, I became more interested in AprilTags and allocated increasing effort to the project.

• Since kornia-rs is a low-level Computer Vision library, I need to understand the algorithms/detectors that I implemented well. But the issue was that I was in my 2nd semester and core computer science stuff starts from 3rd semester in my university. Luckily I understood the papers well and understood the code of existing implementations which made it slightly easier.

Learning Outcomes

I learned a lot during these 3 months and had a lot of new experiences. Prior to this, I had not worked in a team; my experience was primarily solo projects and occasional open-source contributions. But during this time, we had weekly sync and regular conversation of what and how to do the next thing. It was truly a wonderful experience.