DEV Community: Jordi Mon Companys

SIMDVec Supercharges Vector Hunt

Jordi Mon Companys — Thu, 23 Apr 2026 19:54:13 +0000

Elasticsearch simdvec is the engine behind every vector distance computation in Elasticsearch. It provides hand-tuned AVX-512 and NEON kernels for every vector type Elasticsearch supports.

In bulk scoring (where production actually lives), the hand-tuned SIMD kernel library powering all vector distance computations in Elasticsearch, processes float32 vectors at 95 ns/vector. FAISS is at 165 ns. jvector at 412 ns on x86. That's 4x.

The gap widens with scale. Once you exceed CPU cache capacity, simdvec's prefetching drops L1 cache misses from 139K to 19K operations, doubling instructions per cycle. On ARM (Graviton), interleaved loading keeps the pipeline busy during memory fetches — backend stalls down 40%, and the gains hold regardless of dataset size.

This matters more than a benchmark. AI agents in 2026 are retrieval-bottlenecked, not model-bottlenecked. We ran this to ground with BrowseComp-Plus: same model, same prompts — accuracy went from 15% to 93% when retrieval got better. simdvec is what makes that retrieval fast enough to actually ship.

Chris Hegarty, Lorenzo Dematté, and Simon Cooper wrote the full breakdown, benchmarks and methodology included, not just the headline number.

→

What retrieval bottleneck are you actually hitting?

[Boost]

Jordi Mon Companys — Wed, 04 Jun 2025 12:08:16 +0000

fmerian

Jun 1 '25

Who's hiring — June 2025

#career #devrel #writing #opensource

Comments 13

2 min read

Open sourcing Octo STS

Jordi Mon Companys — Thu, 02 May 2024 17:40:57 +0000

Credential leaks are one of the most common ways systems are compromised, the 2024 Verizon Data Breach Report found that 31% of all breaches over the past 10 years have involved the use of stolen credentials, and long-lived credentials are at the heart of that. To eliminate our need for long-lived credentials, we created and published a GitHub Application Octo STS.

‍

Octo STS is a “Security Token Service” (STS) for GitHub credentials. The idea of an STS is largely inspired by the cloud providers like AWS and GCP, but other services have them too, including Chainguard. An STS exchanges a short-lived third-party token for a short-lived first-party token, after checking that the caller has permission to make the exchange.

‍

Our previous post where we talk about our solution to solving this problem seems to have struck a chord, and the outreach and interest we have received has been nothing short of incredible.

‍

We have heard a lot of: “We absolutely need something like this,” or, “We built something like this ourselves.”

‍

Also very understandably, we have had folks express skepticism about the level of permissions that the app needs, as we called out in our previous post:

‍

To address all of these, today we are happy to announce that we are open sourcing Octo STS. This repository contains all of the source code for Octo STS, as well as the infrastructure as code we use to deploy and monitor it.

‍

For folks building something similar: let’s collaborate and build something better together. We have already gotten some fantastic ideas from folks in some of the discussions spurred by our previous post. If you are interested in collaborating, reach out to us.

‍

For folks that want to adopt this, but are unsure about the permissions: this new repository will let you see what we are doing, as well as allow your team to host and manage its own instance.

We are currently building a Chainguard Image for Octo-STS and will share more about that once it is available. If you are interested in receiving updates on the availability of that image, reach out.

Original post