What 18 surviving mutations in a MAVLink C parser say about fuzz harnesses

b6bs62fhys-jpg — Sun, 11 Oct 2026 15:40:10 +0000

I planted 40 small changes (mutations) into the MAVLink C parser (c_library_v2, commit 56f6435e, file mavlink_helpers.h) and checked whether my test setup notices them. The setup is a libFuzzer harness with AddressSanitizer in a local container. With the seed corpus it caught 22 and missed 18, no timeouts.

The 18 misses are the interesting part.

1. Most misses are not equivalent mutants. For 15 of the 18 a targeted input shows an observable difference between original and mutant. For 3 I found none after three attempts. I treat those as equivalent candidates, not as proven equivalent.

2. A crash only harness sees a small part. I sorted the 15 observable misses by the first point where the difference becomes visible (my own classification, not a measurement): 2 runtime or memory integrity, 6 protocol acceptance and parser state, 5 semantic or API behaviour, 2 outbound representation. 13 of the 15 change output without crashing. They need assertions on acceptance, decoded fields or sent bytes.

3. The detection rate depends on the corpus. Earlier runs gave 21 and 22 of 40. In the forum discussion I found that the difference comes from the directory replayed after fuzzing: the run's own corpus in the runs with 21, the 219 seed files in the runs with 22. So 22 of 40 is the figure for this corpus, not a stable property of the parser. This explanation is not yet part of the signed record.

What I am not claiming: all differences come from mutants I planted myself. They show gaps in my test setup, not defects in the released library.

Only source code and local builds, no running systems. Runs, inputs and results are hashed into a hash chain.

Where would you cut the categories differently, and which observation points do you assert in your own parser tests?

I ran a contract check against the Swagger Petstore. Here is what came back.

b6bs62fhys-jpg — Mon, 21 Sep 2026 14:43:58 +0000

Most API bugs I care about are not crashes. They are small lies between the spec and the live API. A field that went missing. A status code nobody documented. A body that is not what the spec says it is.

I built SpecSentinel to catch those. It is a small command line tool. You give it an OpenAPI spec and the address of a running API. It sends GET requests, compares every answer with the spec and ends with an exit code: 0 for a match, 1 for drift, 2 if the check could not be done. That makes it easy to use as a step in CI.

To see what it does on something real, I pointed it at the public Swagger Petstore, the demo API from the OpenAPI world.

specsentinel https://petstore3.swagger.io/api/v3/openapi.json --url https://petstore3.swagger.io/api/v3

This is the real output, shortened:

GET /pet/findByStatus       200  OK
GET /pet/findByTags         500  OK
    warning SERVER_ERROR             status
            server error 500 is only covered by the default response
GET /user/login             200  DRIFT
    error   INVALID_JSON             body
            response is declared as JSON but the body is not valid JSON
GET /user/{username}        200  OK

8 checked, 1 with drift, 0 skipped, 0 failed
Result: DRIFT (exit code 1)

Two things came out of it.

GET /user/login is documented as JSON and answers with Content-Type: application/json, but the body is plain text. A client that trusts the spec and parses the answer as JSON will fail. That is exactly the kind of drift I built the tool for.
Three endpoints answered with a 500. The spec only covers server errors through its default response, so SpecSentinel reports them as warnings, not as errors.

A fair note: the Petstore is a shared demo server, so results can change from day to day. This is not a complaint about the Petstore. It is a real spec and a real server, which makes it a good test.

What the tool does not do yet: it checks GET requests and JSON responses only. I also ran the schema checks against the public specs of the Petstore, GitHub and Stripe (920 GET operations). For every response I generated a conforming example from its schema and fed it through the checker. It produced no findings and no crashes. That shows the checker does not raise false alarms on large real specs. It does not prove that every kind of drift is caught, and it is no replacement for running it against your own API.

Version 0.2.0 adds warnings for string formats, lengths, number ranges, patterns and required response headers, JSON output for other tools, and a one line GitHub Action.

Try it:

pip install specsentinel

The code is on GitHub: https://github.com/b6bs62fhys-jpg/specsentinel

If you run it on your own API and it says something wrong, or misses something it should have caught, I want to hear about it. That is the feedback that makes the tool better.

DEV Community: b6bs62fhys-jpg

What 18 surviving mutations in a MAVLink C parser say about fuzz harnesses

I ran a contract check against the Swagger Petstore. Here is what came back.