DEV Community: Muhammad Ikhwan Fathulloh

Vector Similarity Search with DuckDB: A Practical Guide to the VSS Extension

Muhammad Ikhwan Fathulloh — Sat, 15 Aug 2026 14:21:20 +0000

Most people reach for a dedicated vector database — Pinecone, Qdrant, Milvus, pgvector — the moment a project needs embedding search. But if you are already using DuckDB for analytics or building a lightweight RAG pipeline, there is a good chance you do not need another moving part. DuckDB ships an official vss extension that adds HNSW-based approximate nearest neighbor search directly on top of its native ARRAY type.

This article walks through what the extension does, how to use it, and where its limits are.

What the VSS Extension Actually Is

vss is an experimental core extension that adds indexing support to accelerate similarity search over DuckDB's fixed-size ARRAY columns. It implements HNSW (Hierarchical Navigable Small Worlds), the same graph-based ANN algorithm used by most production vector search engines. In practice, this means you can store embeddings as a normal column, build an index on it, and run ORDER BY ... LIMIT queries that DuckDB will automatically route through the index instead of a full scan.

Because it is embedded, there is no separate service to run, no network hop, and no extra infrastructure to operate — the vector index lives in the same process as the rest of your SQL engine.

Setting It Up

Installation follows DuckDB's usual extension pattern:

INSTALL vss;
LOAD vss;

Then create a table with a fixed-size ARRAY column and build an HNSW index on it:

CREATE TABLE my_vector_table (vec FLOAT[3]);

INSERT INTO my_vector_table
    SELECT array_value(a, b, c)
    FROM range(1, 10) ra(a), range(1, 10) rb(b), range(1, 10) rc(c);

CREATE INDEX my_hnsw_index ON my_vector_table USING HNSW (vec);

Note the dimensionality (FLOAT[3] here) must be fixed at table-creation time — DuckDB's ARRAY type is size-constrained, unlike the variable-length LIST type.

Querying with the Index

Once the index exists, DuckDB will use it automatically whenever a query orders by a supported distance function against a constant vector and limits the result set:

SELECT *
FROM my_vector_table
ORDER BY array_distance(vec, [1, 2, 3]::FLOAT[3])
LIMIT 3;

You can confirm the index is actually being used by checking the query plan for an HNSW_INDEX_SCAN node:

EXPLAIN
SELECT *
FROM my_vector_table
ORDER BY array_distance(vec, [1, 2, 3]::FLOAT[3])
LIMIT 3;

For one-shot nearest-neighbor lookups, the overloaded min_by(col, arg, n) aggregate is also index-accelerated when arg matches a supported distance function, and it conveniently returns the full matched row as a struct:

SELECT min_by(my_vector_table, array_distance(vec, [1, 2, 3]::FLOAT[3]), 3 ORDER BY vec) AS result
FROM my_vector_table;

Distance Metrics

By default, HNSW indexes use l2sq (squared Euclidean distance), matching array_distance. You can choose a different metric at index-creation time:

CREATE INDEX my_hnsw_cosine_index
ON my_vector_table
USING HNSW (vec)
WITH (metric = 'cosine');

Metric	Function	Description
`l2sq`	`array_distance`	Euclidean distance
`cosine`	`array_cosine_distance`	Cosine similarity distance
`ip`	`array_negative_inner_product`	Negative inner product

For most embedding models used in RAG or semantic search (OpenAI, Sentence-Transformers, etc.), cosine similarity is the natural choice. You can also build multiple indexes on the same column with different metrics, or index multiple columns independently — each HNSW index applies to exactly one column.

Tuning the Index

Index quality and search speed are controlled by a handful of hyperparameters, all familiar to anyone who has tuned HNSW before:

Option	Default	Effect
`ef_construction`	128	Candidate vertices considered while building the index. Higher = more accurate, slower build.
`ef_search`	64	Candidate vertices considered per query. Higher = more accurate, slower search.
`M`	16	Max neighbors per graph vertex. Higher = more accurate, slower build.
`M0`	2 × `M`	Base connectivity at the zero-th graph level.

ef_search can also be overridden per connection at runtime without rebuilding the index:

SET hnsw_ef_search = 128;
-- ...run queries...
RESET hnsw_ef_search;

This is useful when you want to dial accuracy up or down depending on the query, without paying the cost of a full reindex.

Persistence: The Part You Need to Read Carefully

This is the biggest practical caveat. By default, HNSW indexes can only be created on in-memory databases. If you want the index to persist in a disk-backed .duckdb file, you must explicitly opt in:

SET hnsw_enable_experimental_persistence = true;

It is locked behind this flag because WAL (write-ahead log) recovery is not yet fully implemented for custom extension indexes. If DuckDB crashes or is killed while there are uncommitted changes to an HNSW-indexed table, the index can end up corrupted or lose data. The docs are explicit that this is not recommended for production use.

If you do enable it and hit an unexpected shutdown, recovery is possible by starting DuckDB separately, loading vss, and then ATTACHing the database file before letting WAL replay run — this makes the HNSW functionality available during recovery.

When persistence is enabled, the entire index is serialized to disk on every checkpoint (no incremental updates) and deserialized back into memory on the next access after restart — which is still generally faster than dropping and rebuilding it from scratch.

For a local RAG prototype or an in-memory analytical session, this is a non-issue. For anything that needs durable, crash-safe vector storage in production, treat this as a hard constraint and plan accordingly — or keep the source-of-truth embeddings elsewhere and rebuild the index on startup.

Inserts, Updates, Deletes

The index supports mutation after creation, with two practical notes:

It is faster to build the index after bulk-loading data, since the initial build parallelizes better than incremental inserts.
Deletes are lazy: rows are marked deleted rather than removed from the graph immediately, which causes gradual quality and performance degradation.

To reclaim this, run:

PRAGMA hnsw_compact_index('my_hnsw_index');

or periodically drop and recreate the index if the table sees heavy churn.

Bonus: Fuzzy Joins with `vss_join` and `vss_match`

Beyond single-query nearest-neighbor search, the extension ships two table macros for matching two sets of vectors against each other — useful for deduplication, entity resolution, or batch retrieval:

CREATE TABLE haystack (id INT, vec FLOAT[3]);
CREATE TABLE needle (search_vec FLOAT[3]);

INSERT INTO haystack
    SELECT row_number() OVER (), array_value(a, b, c)
    FROM range(1, 10) ra(a), range(1, 10) rb(b), range(1, 10) rc(c);

INSERT INTO needle VALUES ([5, 5, 5]), ([1, 1, 1]);

SELECT * FROM vss_join(needle, haystack, search_vec, vec, 3) res;

vss_match offers the same brute-force k-NN matching but as a lateral join, grouping results per left-table row:

SELECT * FROM needle, vss_match(haystack, search_vec, vec, 3) res;

Important: neither macro uses the HNSW index — they perform brute-force search. They are convenience utilities for correctness, not performance, though the docs note they may become index-accelerated in the future.

Limitations to Keep in Mind

Only 32-bit FLOAT vectors are supported today — no DOUBLE, no quantized/int8 vectors.
The index is not buffer-managed and must fit entirely in RAM.
Index memory does not count against DuckDB's memory_limit setting, so it is easy to overshoot available memory without warning.
Persistent indexes require the experimental flag discussed above.
vss_join / vss_match never use the index, regardless of whether one exists on the underlying columns.

When This Makes Sense

vss is a strong fit when:

You are building a local-first or embedded RAG system (for example, a FastAPI service backed by DuckDB and a local GGUF model) and do not want to run a separate vector database.
Your embedding volume is modest enough to fit in memory, and you are comfortable with in-memory or experimental-persistence trade-offs.
You want vector search to live in the same SQL surface as your relational and analytical queries — joining structured metadata filters with ORDER BY array_cosine_distance(...) in a single statement, without shipping data to another system.

It is a weaker fit if you need durable, crash-safe indexes at scale in a multi-writer production environment — for that, a dedicated vector database or an extension like pgvector on a durable RDBMS is currently the safer choice.

Summary

The vss extension turns DuckDB into a capable, embedded ANN engine: INSTALL/LOAD the extension, store embeddings in a fixed-size ARRAY column, CREATE INDEX ... USING HNSW, and query with ORDER BY array_distance(...) LIMIT k. It supports L2, cosine, and inner-product metrics, exposes the usual HNSW tuning knobs, and even offers brute-force join macros for batch matching. The one thing to plan around carefully is persistence — it is opt-in and explicitly experimental, so treat durability as something you design for rather than assume.

Secure File and Photo Upload Validation in PHP: Beyond Checking the Extension

Muhammad Ikhwan Fathulloh — Mon, 10 Aug 2026 12:57:45 +0000

File upload is a feature present in almost every web application, from document uploads and profile photos to report attachments. Unfortunately, it's also one of the most common entry points for attacks: web shells disguised as images, HTML files containing malicious scripts, or files deliberately oversized to overload the server.

This article covers a more thorough approach to validating file and photo uploads in PHP, drawn from experience building an upload helper in a Laravel project. The code examples here are simplified to avoid depending on any specific storage service, so they can be adapted to local disk, S3, or any other storage backend.

Why Extension Validation Alone Isn't Enough

A common pattern looks like this:

$extension = $file->getClientOriginalExtension();
if (!in_array($extension, ['jpg', 'png', 'pdf'])) {
    // reject
}

The problem is that getClientOriginalExtension() reads the filename sent by the user's browser. This filename can be manipulated entirely. A file named shell.jpg could actually contain PHP code. The extension is just a label, not proof of the file's actual content.

Layers of Validation to Combine

Secure upload validation shouldn't rely on a single check. It works best as several complementary layers.

1. Validate MIME Type from File Content, Not from the Client

Use a function that reads the file's opening bytes (magic numbers), not the header sent by the browser.

function getRealMimeType(string $path): string
{
    $finfo = finfo_open(FILEINFO_MIME_TYPE);
    $mime = finfo_file($finfo, $path);
    finfo_close($finfo);

    return $mime;
}

$allowedMimes = [
    'image/jpeg',
    'image/png',
    'application/pdf',
];

if (!in_array(getRealMimeType($file->getPathname()), $allowedMimes)) {
    // reject
}

With finfo, file type detection is based on actual content, so a file with a swapped extension alone won't pass.

2. Limit File Size Up Front

Besides saving resources, a size limit also prevents denial-of-service attacks through oversized files.

$maxFileSize = 2 * 1024 * 1024; // 2 MB

if ($file->getSize() > $maxFileSize) {
    // reject, and state the maximum limit in the error message
}

Ideally this limit should also be enforced at the web server level (e.g. client_max_body_size in Nginx or upload_max_filesize in php.ini), not just in application code.

3. Scan Content for Dangerous Patterns

For files like images that shouldn't contain any executable code, a simple check for suspicious patterns can serve as an additional layer.

$content = file_get_contents($file->getPathname());
$dangerousPatterns = ['<?php', '<script', '<html'];

foreach ($dangerousPatterns as $pattern) {
    if (stripos($content, $pattern) !== false) {
        // reject
    }
}

Note that this is not a substitute for antivirus or malware scanning, only a cheap additional filter for the most common cases. For production needs with higher stakes, integrating ClamAV or a third-party scanning service is recommended.

4. Never Trust the Original Filename

A filename from the user can contain path traversal characters like ../../etc/passwd or characters that cause problems on certain filesystems. Always generate a new filename on the server under controlled conditions.

function generateSafeFilename(string $originalName): string
{
    $extension = pathinfo($originalName, PATHINFO_EXTENSION);
    $safeName = preg_replace('/[^a-zA-Z0-9_-]/', '_', pathinfo($originalName, PATHINFO_FILENAME));

    return time() . '_' . strtolower($safeName) . '.' . strtolower($extension);
}

The final filename should be built from a combination of a timestamp, a random identifier, or a user ID, never copied raw from user input.

5. Store Outside the Web Root, Serve Through Application Code

Uploaded files shouldn't be stored directly in a publicly accessible directory. Store them outside public/, then create a dedicated endpoint that verifies access rights before serving the file.

Route::get('/files/{id}', function ($id) {
    $file = File::findOrFail($id);

    abort_unless(auth()->user()->can('view', $file), 403);

    return response()->file(storage_path('app/private/' . $file->path));
});

This approach prevents a file that passed validation from still being directly executable by the web server, since the server never serves it as a static path.

6. Use a Whitelist, Not a Blacklist

Instead of rejecting a list of known-dangerous file types (which can easily be incomplete), explicitly define which types are allowed.

$allowedExtensions = ['jpg', 'jpeg', 'png', 'pdf'];
$allowedMimes = ['image/jpeg', 'image/png', 'application/pdf'];

A whitelist approach is far safer because it defaults to rejecting anything unrecognized, rather than only rejecting what's already known to be harmful.

Putting the Layers Together

Here's a validation function that combines all the points above:

function validateUpload($file, array $config): array
{
    $extension = strtolower($file->getClientOriginalExtension());
    if (!in_array($extension, $config['allowed_extensions'])) {
        return ['valid' => false, 'message' => 'File extension is not allowed.'];
    }

    $realMime = getRealMimeType($file->getPathname());
    if (!in_array($realMime, $config['allowed_mimes'])) {
        return ['valid' => false, 'message' => 'File type does not match its actual content.'];
    }

    if ($file->getSize() > $config['max_size']) {
        return ['valid' => false, 'message' => 'File size exceeds the maximum limit.'];
    }

    $content = file_get_contents($file->getPathname());
    foreach ($config['dangerous_patterns'] as $pattern) {
        if (stripos($content, $pattern) !== false) {
            return ['valid' => false, 'message' => 'File contains disallowed content.'];
        }
    }

    return ['valid' => true, 'filename' => generateSafeFilename($file->getClientOriginalName())];
}

Closing Thoughts

Secure upload validation isn't about adding a single, "smartest" check. It's about layering several simple safeguards that cover each other's blind spots: MIME type from actual content, size limits, content scanning, controlled filename generation, and storage outside public access. No single layer is perfect on its own, but the combination makes the attack surface far harder to exploit.

For applications with higher security requirements, also consider adding external malware scanning and audit logging for every upload activity, so every incoming file can be traced back to its origin and history.

T-Guard: An Integrated Open-Source SOC Solution for Your Organization

Muhammad Ikhwan Fathulloh — Fri, 12 Jun 2026 15:44:45 +0000

Introduction

In an increasingly complex digital era, cyber threats continue to evolve at a rapid pace. Organizations of all sizes need security systems that are both robust and efficient. Enter T-Guard an innovative Security Operations Center (SOC) solution that harnesses the power of leading open-source tools into a single, cohesive platform.

T-Guard is not just a collection of security tools. It is a complete ecosystem designed to provide comprehensive protection for your digital assets. Developed with support from Universitas Indonesia - Japan International Cooperation Agency (UI-JICA Project) , this project demonstrates how collaboration between academia and security practitioners can produce truly impactful solutions.

Why T-Guard?

T-Guard addresses a major challenge organizations face when building a SOC: integration. Instead of hiring a large team to manage multiple disparate platforms, T-Guard unifies four key pillars of cybersecurity under one roof:

Component	Function	License
Wazuh	Real-time monitoring, threat detection, and compliance	GPL v2.0
DFIR-IRIS	Incident management and digital forensics	LGPL v3.0
Shuffle	Security workflow automation (SOAR)	AGPL v3.0
MISP	Open-source threat intelligence platform	AGPL v3.0

By combining these four tools, T-Guard delivers a complete defense lifecycle: detection (Wazuh), intelligence (MISP), automated response (Shuffle), and incident management (IRIS).

Simple Installation

One of T-Guard's greatest strengths is its ease of installation. Within minutes, you can have a fully functional SOC up and running.

Prerequisites

OS: Ubuntu 24.04 LTS (fresh machine recommended)
Connection: Broadband internet

Minimum requirements (trial deployments):

CPU: 8 cores
RAM: 16 GB (swap memory required)
Storage: 100 GB

Standard requirements (production environments):

CPU: 8 cores
RAM: 32 GB (no swap needed)
Storage: 250 GB

Installation Steps

Installation is performed through an interactive command-line script. Here's the overview:

Clone the repository:

git clone https://github.com/sguresearcher/nusantara.git
cd nusantara
chmod +x setup.sh
./setup.sh

Main menu options:
- 1 → Update system & install dependencies (Docker, etc.)
- 2 → Install T-Guard SOC package (choose environment: local VM or cloud)
- 3 → Module integration (API keys, webhooks)
- 4 → Use case simulation menu

The installation process runs automatically. Upon completion, you will receive a table with dashboard access information for each module:

Service	URL	Username	Password
Wazuh	`https://<ip>`	`admin`	`SecretPassword`
IRIS	`https://<ip>:8443`	`administrator`	`MySuperAdminPassword!`
Shuffle	`http://<ip>:3001`	Create your own	Create your own
MISP	`https://<ip>:1443`	`admin@admin.test`	`admin`

Note: SSL warnings in your browser are normal due to self-signed certificates. For production, installing official certificates is recommended.

Use Cases: Proving T-Guard's Capabilities

The T-Guard documentation provides three simulated attack scenarios to demonstrate how all components work together automatically.

1. Brute-Force Detection

Scenario: Simulated repeated login attempts against SSH.

Results:

Wazuh detects the brute-force attempts
Alerts are sent to Shuffle
Shuffle automatically creates a new incident ticket in IRIS
The SOC team can immediately review attack details and begin investigation

2. Malware Detection with Auto-Response

Scenario: A malicious file (malware) is uploaded to the system.

Results:

Wazuh detects the file
Integration with VirusTotal (API key required) confirms the file as malware
Wazuh automatically executes a response (e.g., deleting the file)
Alerts and automated responses are recorded in the Wazuh dashboard
A new incident ticket is created in IRIS

3. Web Defacement Detection

Scenario: A sample website page is defaced.

Results:

Wazuh's Integrity Monitoring module detects file content changes
Alert with rule ID 550 (Integrity Checksum Changed) is triggered
The SOC team is immediately notified of unauthorized changes to web assets

Threat Intelligence Integration with MISP

One of the most powerful features is the integration with MISP (Malware Information Sharing Platform). After installation, administrators can import over 100 threat feeds from MISP's defaults.json.

Quick steps:

Navigate to https://github.com/MISP/MISP/blob/2.4/app/files/feed-metadata/defaults.json
Copy the entire JSON content
In MISP Dashboard: Sync Actions → Feeds → Import Feeds from JSON
Select all feeds → Enable Selected → Fetch and Store All Feed Data

The threat intelligence from MISP is then used by Wazuh to enrich detection and perform IOC (Indicators of Compromise) matching.

Automation Workflow: Connecting the Pieces

A key highlight of T-Guard is how Shuffle orchestrates actions between components. For example, when Wazuh detects an alert:

A webhook trigger in Shuffle receives the alert
Shuffle extracts relevant data (source IP, rule description, log details)
The workflow calls the IRIS module to create a new case with all alert information
The SOC team sees a complete, auto-populated ticket ready for investigation

This automation eliminates manual hand-offs and accelerates response times dramatically.

Conclusion

T-Guard is concrete proof that enterprise-grade SOC solutions can be built from well-integrated open-source components. This project is ideally suited for:

Organizations looking to build an internal SOC with limited budgets
Security teams wanting workflow automation without building from scratch
Educational institutions and researchers studying modern SOC architecture

With clear installation guides, available testing scenarios, and community support, T-Guard offers a fast path toward maturing your organization's cybersecurity capabilities.

Resources & Links

Resource	Link
Official T-Guard Website	https://tguard.org/
Installation Documentation	https://docs.tguard.org/installation
GitHub Repository	https://github.com/sguresearcher/nusantara
License Information	Available on GitHub (mix of GPL, Apache, AGPL, LGPL)

Closing Thought: Cybersecurity is not about the most expensive tools it's about the most integrated systems that are ready to respond. T-Guard proves that open source can be a solid foundation for building the digital fortresses of tomorrow.

Secure Your Microservices: Meet Halimun, the High-Performance Encrypted Proxy

Muhammad Ikhwan Fathulloh — Fri, 29 May 2026 15:44:21 +0000

Meet Halimun Proxy a high-performance, ultra-low latency proxy tunnel system built from the ground up in Rust.

Why Rust?

By leveraging Rust, Halimun achieves extreme efficiency. Using the Axum web framework and Tokio for non-blocking asynchronous I/O, it manages to maintain a tiny footprint—running on as little as ~15MB of RAM. It’s designed to be fast, memory-safe, and incredibly stable under load.

Core Security Features

Halimun isn't just a proxy; it’s a security layer. It enforces strict request validation to ensure your internal services are never exposed to malicious actors:

AES-256-CBC Encryption: End-to-end payload masking. Even if your traffic is intercepted, the actual API endpoint and data remain indecipherable.
HMAC-SHA256 Integrity: Validates that data hasn't been tampered with in transit.
Replay Attack Prevention: Uses Nonce and timestamp verification in-memory (via DashMap) to reject duplicate spoofed requests.
SSRF Protection: Built-in mechanisms to prevent attackers from targeting your internal network infrastructure (e.g., 127.0.0.1).
Camouflage Routing: It hides your actual API structure behind random, dummy URL segments, making traffic profiling by WAFs or human analysts nearly impossible.

Quick Start (Docker)

Halimun is "Docker-ready," making it easy to drop into any existing infrastructure.

1. Configuration

First, generate your encryption keys using the built-in generator:

# Generate keys and save to .env
docker build -t halimun-proxy .
docker run --rm halimun-proxy ./halimun-proxy --keygen --format=env > .env

2. Deployment

Configure your config.yaml to map your backend services, then launch your cluster:

docker-compose up -d

Your production proxy is now live, listening securely on port 80 while your backend services remain completely secluded within a private Docker network.

Under the Hood: Request Lifecycle

Halimun uses an encrypted tunnel approach. A typical request follows this structure:
POST /proxy/1/SEGMENT1/SEGMENT2/SEGMENT3/SEGMENT4/SEGMENT5

The actual data is sent via x-www-form-urlencoded with the body encrypted as a Base32 string. Once received, Halimun decrypts the payload, validates the HMAC, checks the Nonce for replays, and only then forwards the request to your microservice.

Monitoring & Management

Security doesn't have to be a "black box." Halimun includes a sleek, Glassmorphism-styled Admin UI accessible via your dashboard. It provides:

Live Traffic Logs: See exactly what’s happening in real-time.
Registry Hub: Overview of all your active backend mappings.
Key Management: Rotate credentials remotely to maintain high security.

Join the Community

Whether you are building a microservices mesh or simply want to add a hardened security layer to your existing API, Halimun is a fantastic, open-source choice.

GitHub Repository: Muhammad-Ikhwan-Fathulloh/Halimun-Proxy
Docker Hub: ikhwan17/halimun-proxy

Have you experimented with Rust-based proxies yet? Let us know how Halimun fits into your stack in the comments below!

Happy coding!

Professional Inverse Kinematics on Arduino: Deep Dive into NocKinematics

Muhammad Ikhwan Fathulloh — Tue, 21 Apr 2026 08:04:28 +0000

While professional robotics often use the Jacobian Inverse or Cyclic Coordinate Descent (CCD), these methods are either too "heavy" for an Arduino or produce jerky, unnatural movements.

Today, we are exploring NocKinematics, a lightweight C++ library that brings the industry-standard FABRIK algorithm to the world of microcontrollers.

🔬 The Science: Why FABRIK?

NocKinematics is built upon the foundational research of Andreas Aristidou and Joan Lasenby. If you are looking for the academic rigor behind this library, you should refer to the original paper:

> Aristidou, A., & Lasenby, J. (2011). "FABRIK: A fast, iterative solver for the Inverse Kinematics problem." Graphical Models, 73(5), 243-260.

Why this matters for Embedded Systems:

Unlike traditional methods that rely on expensive trigonometric functions or matrix inversions, FABRIK (Forward And Backward Reaching Inverse Kinematics) uses an iterative approach based on finding points on a line.

Low Computational Cost: Perfect for 8-bit AVR (Arduino Uno) and 32-bit ESP32 alike.
Fast Convergence: It usually finds a solution in just a few iterations.
Handle Constraints: It gracefully handles "unreachable" targets by stretching the arm to its maximum toward the point.

🛠️ Key Features of NocKinematics

Developed by Muhammad Ikhwan Fathulloh and the Nocturnailed Community, this library bridges the gap between game engine math and hardware.

N-Joint Support: Solve kinematics for 2, 3, or even 20+ joints (ideal for snake robots or tentacles).
Memory Optimized: It avoids std::vector to prevent heap fragmentation. It allocates memory exactly once during initialization.
Platform Agnostic: Runs on Arduino Uno, Nano, Mega, ESP8266, and ESP32.

🚀 Step-by-Step Implementation

1. Define Your Robot's Anatomy

Think of your robot as a collection of "joints" connected by "bones." If you have 4 joints, you have 3 bones.

#include <NocKinematics.h>

const int NUM_JOINTS = 4;
// Length of each bone segment in your chosen unit (cm, mm, etc.)
float boneLengths[NUM_JOINTS - 1] = { 10.0, 8.0, 5.0 }; 

// Initialize the solver on the Heap
NocKinematics::FABRIK* armSolver = new NocKinematics::FABRIK(boneLengths, NUM_JOINTS);

2. Set the Anchor

The "Base" is the stationary part of your robot (e.g., the shoulder).

armSolver->setBasePosition(NocCore::Vector3(0, 0, 0));

3. Reach the Target

Instruct the arm to calculate the positions needed to touch a point in space.

NocCore::Vector3 target(12.0, 5.0, 3.0);
bool success = armSolver->solve(target);

4. Mapping to Hardware (Servos)

The library gives you Vector3 coordinates for each joint. To move a real servo, you simply use atan2 to find the angle between these points.

for (int i = 0; i < NUM_JOINTS; i++) {
    NocCore::Vector3 pos = armSolver->getJointPosition(i);
    Serial.print("Joint "); Serial.print(i);
    Serial.print(": "); Serial.print(pos.x_val);
    Serial.print(", "); Serial.println(pos.y_val);
}

📂 Real-World Examples Included

The library ships with several built-in examples (File > Examples > NocKinematics):

BasicArm.ino: The "Hello World" of kinematics.
DynamicTarget.ino: Watch the arm follow a moving coordinate in real-time.
MultiJointSnake.ino: A stress test showing 10+ joints moving smoothly on an Arduino.
ServoArm4DOF/5DOF: The most practical scripts. They show how to translate $X, Y, Z$ math into actual servo.write(angles) commands.

📦 Installation

Get started today by downloading the library through your preferred channel:

Arduino Library Manager: Search for NocKinematics.
GitHub: Nocturnailed-Community/NocKinematics
Documentation: Arduino Libraries Info

Conclusion

Inverse Kinematics doesn't have to be a "math wall" that stops your project. With NocKinematics, you can focus on building the hardware and the behavior, while the FABRIK algorithm handles the heavy lifting of spatial geometry.

License: Released under the MIT License.
Maintained by: Nocturnailed Community.

#Robotics #Arduino #InverseKinematics #FABRIK #ESP32 #OpenSource #NocLab #Mathematics

Bringing Generative AI to Microcontrollers: Introducing NocLLM

Muhammad Ikhwan Fathulloh — Tue, 21 Apr 2026 05:30:05 +0000

The barrier between resource-constrained hardware and Large Language Models (LLMs) has finally been broken. While microcontrollers lack the VRAM to run a 70B parameter model locally, they can now act as intelligent gateways to the world's most powerful AI engines.

Enter NocLLM, an optimized integration and inference library designed specifically for Arduino and embedded systems.

What is NocLLM?

NocLLM is a high-performance C++ library that allows microcontrollers to communicate with LLM providers (OpenAI, Gemini, Groq, DeepSeek) or local LLM servers (Ollama, LMStudio) using a non-blocking, stream-oriented architecture.

Unlike traditional HTTP clients that hang while waiting for a full JSON response, NocLLM parses incoming data chunks in real-time. This means your Arduino can keep reading sensors or driving motors while the AI is "typing" its response.

Core Strengths:

Zero-Overhead Streaming: Uses background TCP polling to prevent CPU stalls.
Multi-Provider Support: One unified syntax for various AI infrastructures.
Smart Parsing: Automatically adapts its internal configuration based on the target URL (e.g., switching between Gemini and OpenAI protocols).
Edge-First Design: Optimized for memory efficiency, preventing "Out of Memory" crashes during long AI conversations.

5 Ways to Power Your Hardware with LLMs

NocLLM ships with five comprehensive examples (found in File -> Examples -> NocLLM) to get you started:

01_Sumopod: DeepSeek-V3 integration via Sumopod Cloud.
02_OpenAI: The industry standard—perfect for GPT-4o or GPT-3.5 Turbo.
03_Gemini_Native: Harness Google’s gemini-3-flash. NocLLM handles the specific Google GenAI headers and parsing logic automatically.
04_Groq: Experience ultra-low latency with llama3-70b. Ideal for voice assistants or real-time robotics.
05_Local_LMStudio: The privacy-focused choice. Connect to Ollama or LMStudio on your local network. It uses bare TCP streams with 0 SSL overhead, providing blazing-fast speeds for local AI setups.

Technical Spotlight: Non-Blocking Execution

The most powerful feature of NocLLM is its ability to multitask. Here is how simple it is to implement a streaming AI response without freezing your microcontroller:

#include "NocLLM.h"

// Initialize with your Key, Endpoint, and Model
NocAI ai("YOUR_API_KEY", "https://api.openai.com/v1", "gpt-3.5-turbo");

// Callback function triggered as each word/chunk arrives
void onStream(String chunk) {
    Serial.print(chunk);
}

void setup() {
    Serial.begin(115200);
    // ... [Insert your WiFi connection logic here] ...

    // Attach the listener and trigger a prompt
    ai.onMessage(onStream);
    ai.beginStream("Write a 1-sentence poem about a robot."); 
}

void loop() {
    // This gently pulls the data from the network in the background
    ai.loop();

    // Your main logic stays alive!
    // Example: Blink an LED or read a DHT22 sensor here.
}

Why Use NocLLM for Your Next Project?

Interactive Robotics: Give your robot a "brain" that can understand complex natural language commands.
Smart Home Hubs: Build a private voice assistant that processes logic via a local Ollama server.
Intelligent Data Analysis: Send sensor logs to an LLM to receive a human-readable summary of system health.

Installation & Resources

Ready to build the next generation of smart hardware?

Arduino Library Manager: Search for "NocLLM" and click install.
GitHub Repository: Nocturnailed-Community/NocLLM
Registry Details: NocLLM on Arduino Libraries

NocLLM is part of the Noc Lab ecosystem, dedicated to pushing the boundaries of what is possible on the edge.

#Arduino #LLM #GenerativeAI #IoT #EdgeAI #NocLab #OpenSource #Programming

Bringing Intelligence to the Edge: Introduction to NocML for Arduino

Muhammad Ikhwan Fathulloh — Tue, 21 Apr 2026 05:22:10 +0000

Edge AI is often seen as a field reserved for powerful single-board computers, but what if you could run machine learning logic directly on a standard Arduino?

In this article, we will explore NocML, an efficient machine learning library specifically designed for resource-constrained microcontrollers.

What is NocML?

NocML is a lightweight C++ library built to bridge the gap between complex ML logic and the limited processing power of microcontrollers like the ESP32, Arduino Uno, or Nano. Inspired by the Scikit-Learn API, it offers a familiar workflow for developers coming from a Python background.

Key Features:

Low Memory Footprint: Optimized to run within the tight SRAM limits of common microcontrollers.
Data Preprocessing: Includes tools like MinMaxScaler for data normalization on-device.
Versatile Algorithms: Supports Classification (KNN, Naive Bayes), Clustering (K-Means), and Regression (Linear Regression).
Zero Latency: Perform inference locally on the device, ensuring privacy and real-time response without cloud dependency.

Use Case: K-Nearest Neighbors (KNN) Classification

One of the strongest features of NocML is its ability to perform sensor classification directly on the "edge." Imagine building a device that classifies activity types based on accelerometer data.

Here is a practical example of how to implement a KNN algorithm using NocML:

#include <NocML.h>

// Define training data (Features)
float X_train = {
  {1.0, 2.0}, // Category A
  {1.5, 1.8}, // Category A
  {5.0, 8.0}, // Category B
  {6.0, 7.0}  // Category B
};

// Labels for training data
int y_train = {0, 0, 1, 1};

// Initialize KNN with k=3
KNN knn(3);

void setup() {
  Serial.begin(115200);

  // Local training (Fit)
  knn.fit((float*)X_train, y_train, 4, 2);
  Serial.println("KNN Model Ready!");
}

void loop() {
  // New sensor data to classify
  float input = {1.2, 2.1};

  // Perform prediction
  int prediction = knn.predict(input);

  Serial.print("Classification Result: ");
  Serial.println(prediction == 0 ? "Category A" : "Category B");

  delay(2000);
}

Why NocML?

Smart IoT: Transform passive sensors into intelligent nodes that make decisions without a server.
Predictive Maintenance: Detect anomalous vibration patterns in industrial motors before failure occurs.
Human-Machine Interaction: Recognize gestures or simple audio patterns in real-time.

Getting Started

You can easily integrate NocML into your project via the Arduino Library Manager or by visiting the official repositories.

Installation

Open your Arduino IDE.
Go to Sketch -> Include Library -> Manage Libraries...
Search for "NocML" and click install.

Alternatively, explore the source code and documentation:

GitHub Repository: Nocturnailed-Community/NocML
Arduino Library Registry: NocML on Arduino Libraries

Conclusion

NocML is part of the TinyML movement, making artificial intelligence accessible on devices costing only a few dollars. By bringing logic closer to the data source, we create faster, more reliable, and smarter IoT systems.

Are you working on an Edge AI project? Give NocML a try and share your results!

#Arduino #MachineLearning #TinyML #IoT #OpenSource #NocLab #ArtificialIntelligence

DEV Community: Muhammad Ikhwan Fathulloh

Vector Similarity Search with DuckDB: A Practical Guide to the VSS Extension

What the VSS Extension Actually Is

Setting It Up

Querying with the Index

Distance Metrics

Tuning the Index

Persistence: The Part You Need to Read Carefully

Inserts, Updates, Deletes

Bonus: Fuzzy Joins with vss_join and vss_match

Limitations to Keep in Mind

When This Makes Sense

Summary

Secure File and Photo Upload Validation in PHP: Beyond Checking the Extension

Why Extension Validation Alone Isn't Enough

Layers of Validation to Combine

1. Validate MIME Type from File Content, Not from the Client

2. Limit File Size Up Front

3. Scan Content for Dangerous Patterns

4. Never Trust the Original Filename

5. Store Outside the Web Root, Serve Through Application Code

6. Use a Whitelist, Not a Blacklist

Putting the Layers Together

Closing Thoughts

T-Guard: An Integrated Open-Source SOC Solution for Your Organization

Introduction

Why T-Guard?

Simple Installation

Prerequisites

Installation Steps

Use Cases: Proving T-Guard's Capabilities

1. Brute-Force Detection

2. Malware Detection with Auto-Response

3. Web Defacement Detection

Threat Intelligence Integration with MISP

Automation Workflow: Connecting the Pieces

Conclusion

Resources & Links

Secure Your Microservices: Meet Halimun, the High-Performance Encrypted Proxy

Why Rust?

Core Security Features

Quick Start (Docker)

1. Configuration

2. Deployment

Under the Hood: Request Lifecycle

Monitoring & Management

Join the Community

Professional Inverse Kinematics on Arduino: Deep Dive into NocKinematics

🔬 The Science: Why FABRIK?

Why this matters for Embedded Systems:

🛠️ Key Features of NocKinematics

🚀 Step-by-Step Implementation

1. Define Your Robot's Anatomy

2. Set the Anchor

3. Reach the Target

4. Mapping to Hardware (Servos)

📂 Real-World Examples Included

📦 Installation

Conclusion

Bringing Generative AI to Microcontrollers: Introducing NocLLM

What is NocLLM?

Core Strengths:

5 Ways to Power Your Hardware with LLMs

Technical Spotlight: Non-Blocking Execution

Why Use NocLLM for Your Next Project?

Installation & Resources

Bringing Intelligence to the Edge: Introduction to NocML for Arduino

What is NocML?

Key Features:

Use Case: K-Nearest Neighbors (KNN) Classification

Why NocML?

Getting Started

Installation

Conclusion

Bonus: Fuzzy Joins with `vss_join` and `vss_match`