DEV Community

Suliman Mokhtar
Suliman Mokhtar

Posted on Originally published at xroot.dev

Solana P-256 Syscalls: Agave and Firedancer, Edge Case by Edge Case

Originally published on xroot.dev.

On 27 October both of Solana’s independent runtimes merged the same new feature, written separately. Agave added P-256 curve operations to its syscalls at 02:38 UTC; Firedancer added its own sixteen hours later. Different authors, different languages, different arithmetic, and consensus needs them to return the same bytes for every input a program can send, including every input the specification rejects.

SIMD-0565 extends Solana’s curve syscalls to P-256, also called secp256r1: the curve behind passkeys, WebAuthn and the secure chips in phones. We wrote our own reference implementation from the curve’s published constants, generated 626 test vectors aimed at the edges of the specification, ran them through the code Agave’s syscalls call, and checked Firedancer’s tests and source for the same cases. We found no case where the two differ. We also built a small program that verifies a P-256 signature with three of the new syscalls and no modular inverse. It costs 22,760 compute units.


What SIMD-0565 adds

No new syscall names. The proposal adds two curve ids, 7 for little-endian and 0x87 for big-endian, to the four curve syscalls programs already use for curve25519 and BLS12-381: point validation, decompression, group operations (add, subtract, multiply by a scalar) and multiscalar multiplication. Points are 64 bytes, x then y, and the point at infinity is 64 zero bytes. Compressed points are 33 bytes: a 02 or 03 parity prefix, then x. Every call has a fixed price, charged before its inputs are read, whether it succeeds or not:

Validate a point — 20 CU

Decompress — 610 CU

Add or subtract — 520 CU

Multiply by a scalar — 9,000 CU

Multiscalar, n points — 9,000 + 3,800 × (n − 1)

Most points within 1.4M CU — 367

A multiscalar call with more than 512 points fails outright, before anything is charged. The SIMD’s authors set the prices on a private ten-validator cluster with 200 ms slots, so that a block packed with any one of these operations still replays within its slot.


Two implementations of one rulebook

Agave’s version, #13918, opened in July, calls solana-secp256r1 0.1.0, a pure-Rust P-256 crate Anza published in September. Firedancer’s, #12498, opened on 22 October, is C built on AWS’s s2n-bignum routines where the build has them. What the two share is everything consensus can see: the same feature-gate key, the same seven compute prices, and five test vectors. Firedancer’s syscall tests reuse the expected outputs from Agave’s, in both byte orders.

Agreement matters more than either implementation being right by its own tests. If one client returns 0 where the other returns 1, a program that branches on the result writes different account data on each, the two compute different bank hashes for the same block, and the network splits along client lines. The dangerous inputs are the ones nobody thought to try.


The edge cases, and what both clients do

These are the inputs where a reasonable implementation could choose differently. In every row the two give the same answer: Agave on our vectors through the crate its syscalls call, Firedancer in its own tests or, where its tests don’t reach, in its source.

64 zero bytes — identity, valid

x = 0, y = √b, a real point — valid

(0, 1) — rejected

x or y written as value + p — rejected

Prefix other than 02 or 03 — rejected

33 zero bytes, SEC1’s identity — rejected

P + P, or P − (−P) — 2P

P − P, or k·P + (n − k)·P — identity

0·P, or k times the identity — identity

Scalar n, n + 1 or 2²⁵⁶ − 1 — rejected

Multiscalar, 0 points — identity

Multiscalar, bad last scalar or point — rejected

Multiscalar, over 512 points — InvalidLength

Any P-256 call before activation — InvalidAttribute

“Rejected” means the call returns 1 and writes nothing; the last two rows are errors that abort the transaction. Firedancer’s tests cover the identity, off-curve and out-of-range points, the 04 prefix, both byte orders, P + P, P − P, P + (−P), adding and subtracting the identity, 0·P, k times the identity, n·P, multiscalar calls of 0, 1, 2 and 70 points including a bad scalar or point, running out of compute and the gate being off. The x = 0 point, the value-plus-p encodings, the other prefixes, n + 1, doubling through subtraction and cancellation inside a multiscalar call we confirmed in its source.


626 vectors, and three things to know before you call them

Our reference is about 160 lines of Python written from the curve constants in NIST SP 800-186 and the SIMD text, without reading either client’s arithmetic. It reproduces all five shared vectors. From it we generated 313 vectors per byte order: 73 hand-picked cases and 240 random operations. The hand-picked ones include everything in the table above, 64-point multiscalar calls whose two 32-point halves are equal or cancel (Firedancer works in batches of 32 points, so these land on its batch boundary), and calls of 367 and 512 points. Run through solana-secp256r1 0.1.0, called exactly as Agave’s syscalls call it at the merge commit, all 626 match, including all 127 rejections.

  • x = 0 is on the curve. The identity’s encoding has x = 0, and so does a real point, (0, √b). Both clients treat only the exact all-zero encoding as the identity, so (0, √b) is an ordinary point, decompression of x = 0 succeeds, and (0, 1) is rejected.
  • Scalars are never reduced. A scalar of n, n + 1 or 2²⁵⁶ − 1 makes the call return 1, even when the point is the identity. A SHA-256 hash is a valid scalar only if it is below n, so reduce it first: any 256-bit value is below 2n, so one comparison and one subtraction are enough.
  • Little-endian is per coordinate. With curve id 7, x and y are each byte-reversed in place, and a compressed point keeps its prefix byte first with only x reversed, which is not SEC1. The generator written in the other byte order fails validation.

The one place the code differs

Read side by side, the clients take different paths in one place: an unknown curve id or group operation in validation, group operations or multiscalar multiplication. Firedancer’s syscalls always abort with InvalidAttribute. Agave aborts only when the feature from SIMD-0137 is active, and otherwise returns 1. That feature has been active on mainnet since slot 311,904,000, the first slot of epoch 722, so the result is the same there. Nothing on the P-256 path itself branches this way.


A signature check from three syscalls

The obvious use is checking passkey signatures inside a program, which today goes through the secp256r1 precompile. ECDSA verification normally needs s⁻¹ mod n, and no syscall computes it. It can be avoided. The signature is valid when s·R equals z·G + r·Q up to sign, where R is the point whose x-coordinate is r, z the message hash and Q the public key. Decompression gives R, one multiplication gives s·R, one two-point multiscalar call gives z·G + r·Q, and comparing x-coordinates ignores the sign:

use solana_define_syscall::{
    curve_constants::{GROUP_OP_MUL, SECP256R1_BE},
    definitions::{sol_curve_decompress, sol_curve_group_op, sol_curve_multiscalar_mul},
};

/// ECDSA check: s·R == ±(z·G + r·Q), where R is the point with x = r.
/// Same answer as the textbook check except for x(R) = r + n, which has
/// probability about 2^-128 and is not tried here.
pub fn verify_p256(q: &[u8; 64], z: &[u8; 32], r: &[u8; 32], s: &[u8; 32]) -> bool {
    if is_zero(r) || is_zero(s) || *r >= N || *s >= N {
        return false;
    }
    // The syscalls reject scalars >= n instead of reducing them.
    let z = if *z >= N { sub(z, &N) } else { *z };

    let mut c = [0u8; 33];
    c[0] = 0x02; // either parity: s·(−R) = −(s·R) has the same x
    c[1..].copy_from_slice(r);
    let (mut rp, mut u, mut t) = ([0u8; 64], [0u8; 64], [0u8; 64]);
    let scalars = [z, *r];
    let points = [G, *q];
    unsafe {
        // R: 610 CU
        if sol_curve_decompress(SECP256R1_BE, c.as_ptr(), rp.as_mut_ptr()) != 0 {
            return false;
        }
        // U = s·R: 9,000 CU
        if sol_curve_group_op(SECP256R1_BE, GROUP_OP_MUL, s.as_ptr(), rp.as_ptr(), u.as_mut_ptr()) != 0 {
            return false;
        }
        // T = z·G + r·Q: 9,000 + 3,800 CU; returns 1 if Q is not a valid point
        if sol_curve_multiscalar_mul(
            SECP256R1_BE,
            scalars.as_ptr() as *const u8,
            points.as_ptr() as *const u8,
            2,
            t.as_mut_ptr(),
        ) != 0
        {
            return false;
        }
    }
    t != [0u8; 64] && u[..32] == t[..32]
}
// N and G are the group order and generator, big-endian; is_zero and sub are byte helpers.
Enter fullscreen mode Exit fullscreen mode

We built it with platform-tools v1.57 and ran it in solana-sbpf 0.26.0, the VM version Agave pinned at the merge, set up as in our CPI measurement, with the three curve syscalls implemented the way Agave implements them on top of the same crate. OpenSSL signed 20 messages, and the program got each signature, the same signature with the other half of S, a hash with one bit flipped, and a different public key. All 80 verdicts match textbook ECDSA:

Curve syscalls: 610 + 9,000 + 12,800 — 22,410 CU

Memory syscalls — 30 CU

Program instructions — 320 CU

One signature check — 22,760 CU

Precompile, cost-model charge — 4,800 CU

Two differences from the precompile. Cost: a check through the syscalls takes about 4.7 times the 4,800 compute units the cost model charges per precompile signature, before the message is hashed. Policy: this check accepts both halves of S, where the precompile rejects the high one, the reason half of raw WebAuthn signatures fail it. A program that wants the precompile’s rule adds s ≤ n/2; one that doesn’t can skip normalising S altogether. The SIMD gives that freedom as the reason for curve operations instead of a single verify syscall, which would have had to fix one malleability rule for everyone.


Where it stands

Both pull requests introduce the gate secp256r1_syscall_enabled, key SeRjtX7Abiq5hQYbzdCny266oveDseHZh9tnrs2cdVC, so the syscalls were off on every cluster when they merged, and a program calling them gets InvalidAttribute until the feature is activated. The SIMD itself was still marked Draft, with its compute prices added a week earlier. The program side is ready: solana-define-syscall 5.2.0, published in July, already declares the curve ids and all four syscalls, and our program imports them from there.


What this does not show

We did not build or run Firedancer. Its side rests on reading its source and on its own CI run, which passed test_secp256r1 and test_vm_syscall_curve on the pull request’s final commit, whose P-256 files are identical to the merged ones. On Agave’s side we ran the crate its syscalls call, not the syscalls: memory translation, compute charging and the gate we read rather than executed. Vectors find disagreements; they cannot prove there are none. The compute prices are the SIMD’s calibration, not our measurement, and 22,760 is one program built with one toolchain.


P-256 Syscalls: Questions People Actually Ask

What does SIMD-0565 add to Solana?

It extends the four existing curve syscalls (point validation, decompression, group operations and multiscalar multiplication) to P-256, also called secp256r1, with curve id 7 for little-endian and 0x87 for big-endian encoding. Points are 64 bytes, x then y, with 64 zero bytes for the point at infinity. Agave and Firedancer both merged an implementation on 27 October 2026, behind a new feature gate.

How much does it cost to verify a P-256 signature with the new syscalls?

Our test program verified an ECDSA P-256 signature for 22,760 compute units: 22,410 for one decompression, one scalar multiplication and one two-point multiscalar multiplication, 30 for memory syscalls and about 320 program instructions. Hashing the message is extra. The secp256r1 precompile is charged 4,800 compute units per signature in the cost model.

Do the secp256r1 syscalls reduce scalars modulo the group order?

No. A scalar of n or more makes the call return 1 in both Agave and Firedancer, including n + 1 and 2^256 − 1. A program that uses a 256-bit hash as a scalar must reduce it first; since any 256-bit value is below 2n, one comparison and one subtraction are enough.

How is the point at infinity encoded for Solana’s P-256 syscalls?

As 64 zero bytes. Only that exact encoding is the identity: (0, √b), which also has x = 0, is an ordinary point on the curve, and (0, 1) is rejected. Decompression cannot produce the identity, and 33 zero bytes are rejected.

Are the P-256 syscalls active on mainnet?

Not when they were merged. Both pull requests introduced the feature gate secp256r1_syscall_enabled, so it was inactive on every cluster, and every P-256 call fails with InvalidAttribute until it is activated.


The one-line version

Agave and Firedancer implemented SIMD-0565’s P-256 syscalls independently and agree on every edge case we could find, from the x = 0 point to unreduced scalars; with them, a program verifies a P-256 signature for 22,760 compute units and sets its own rule for S.

Sources: SIMD-0565 at commit f13f5d65; Agave #13918, merged as cf58f070; Firedancer #12498, merged as 8e7388db; solana-secp256r1 0.1.0, solana-define-syscall 5.2.0 and solana-sbpf 0.26.0. The reference, vectors, harnesses and test program are our own.

Top comments (0)