<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Bernard Abuto</title>
    <description>The latest articles on DEV Community by Bernard Abuto (@bernard_abuto_9970170b9c4).</description>
    <link>https://dev.to/bernard_abuto_9970170b9c4</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4211640%2F07b3c29d-f6f5-4797-83f7-3f2f9385f8d2.jpg</url>
      <title>DEV Community: Bernard Abuto</title>
      <link>https://dev.to/bernard_abuto_9970170b9c4</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/bernard_abuto_9970170b9c4"/>
    <language>en</language>
    <item>
      <title>Why I Started Sauti Labs</title>
      <dc:creator>Bernard Abuto</dc:creator>
      <pubDate>Tue, 27 Oct 2026 21:10:47 +0000</pubDate>
      <link>https://dev.to/bernard_abuto_9970170b9c4/why-i-started-sauti-labs-5b23</link>
      <guid>https://dev.to/bernard_abuto_9970170b9c4/why-i-started-sauti-labs-5b23</guid>
      <description>&lt;p&gt;It began with a speech model that didn’t understand the people I was building for. What I found when I went looking for alternatives, and why I think the distance between African-language research and usable products is worth building a company around.&lt;/p&gt;

&lt;p&gt;Sauti Labs started with a practical problem, not a plan to start a company.&lt;/p&gt;

&lt;p&gt;In July 2026, I was building Scribe AI, an offline system that listens to a church service and puts the scripture being referenced on screen. Everything in it depends on speech recognition, and for that I used OpenAI’s Whisper.&lt;/p&gt;

&lt;p&gt;On the African languages and accents the system needed to handle, it disappointed me.&lt;/p&gt;

&lt;p&gt;I went looking for alternatives. As I explain in Coverage Is Not Capability: What Whisper Taught Me About African Speech, Whisper’s training data is heavily concentrated in English. Swahili appears with roughly five hours of transcribed speech, while Dholuo—the language that first drew me into this work—is absent from its supported-language list.&lt;/p&gt;

&lt;p&gt;I wanted a model I could access, integrate, and rely on. That search took longer than I expected, and it changed what I was looking for.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Arriving in a Field That Already Existed&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The first thing I learned was that I was arriving late to serious, sustained work.&lt;/p&gt;

&lt;p&gt;Masakhane, a grassroots community focused on strengthening natural language processing research in African languages, has spent years building open datasets, models, and benchmarks. Its MasakhaNER project, for example, released a large, publicly available named-entity recognition dataset covering ten African languages, alongside code and models.&lt;/p&gt;

&lt;p&gt;Closer to home, the Maseno Centre for Applied AI is collecting voice data in five Kenyan languages, including Dholuo, as part of African Next Voices—a multi-country effort to build speech datasets for African languages, funded mainly by the Gates Foundation.&lt;/p&gt;

&lt;p&gt;In South Africa, Lelapa AI builds products directly. Its Vulavula platform transcribes and translates contact-centre conversations in languages such as isiZulu and Sesotho. Its researchers have also released InkubaLM, a small language model for five African languages.&lt;/p&gt;

&lt;p&gt;Global research labs have expanded their efforts too. Meta’s speech models now claim coverage of more than 1,600 languages.&lt;/p&gt;

&lt;p&gt;This is not a field waiting to be discovered. It has years of difficult, often under-resourced work behind it, and whatever I build stands on that work.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Distance I Kept Running Into&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;And yet, as a developer with a concrete problem, I kept running into the same distance.&lt;/p&gt;

&lt;p&gt;A dataset existed, but not a model trained for my use case.&lt;/p&gt;

&lt;p&gt;A model listed my language, but its error rates were far from what a product needed.&lt;/p&gt;

&lt;p&gt;A model worked, but its licence ruled out commercial use, or it required hardware my users did not have.&lt;/p&gt;

&lt;p&gt;The products that did exist had been built, sensibly, around particular languages and markets. That is how products should start, but it still left the speech I needed uncovered.&lt;/p&gt;

&lt;p&gt;Over time, that experience hardened into a conviction. I sometimes put it bluntly:&lt;/p&gt;

&lt;p&gt;Africa doesn’t need another conference paper. It needs products that are accessible to the people and work for them in the most efficient and natural way.&lt;/p&gt;

&lt;p&gt;I should be clear about what I mean, because it is easy to read that statement as a criticism of research. It is not.&lt;/p&gt;

&lt;p&gt;Papers, datasets, and benchmarks are how a field learns what works. Without them, there would be nothing for products to stand on. Research and product development are not opposites.&lt;/p&gt;

&lt;p&gt;My point is about where the work tends to stop.&lt;/p&gt;

&lt;p&gt;Knowledge has its greatest effect when someone carries it the rest of the way: into a model trained on real speech, served behind an interface a developer can call, packaged so a business can deploy it, and made good enough that the person on the other end stops noticing the technology.&lt;/p&gt;

&lt;p&gt;That last stretch is often nobody’s job.&lt;/p&gt;

&lt;p&gt;It is the stretch I want to work on.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why Speech?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I chose speech and voice because I believe they will become some of the main ways people interact with AI and digital services generally. Yet many African languages remain underrepresented in precisely the systems that make those interactions possible.&lt;/p&gt;

&lt;p&gt;That underrepresentation is more than a technical inconvenience. It limits what people can build.&lt;/p&gt;

&lt;p&gt;A developer who wants to create a voice service for their community—a clinic line, an advisory service for farmers, or a support agent for a local business—cannot build it reliably if there is no capable model underneath.&lt;/p&gt;

&lt;p&gt;It also shapes how people access services and how naturally they can speak to the technology around them.&lt;/p&gt;

&lt;p&gt;When I listen to AI-generated speech in African languages today, I often hear the gap: pronunciation that is slightly off, an accent that belongs to someone else, or a delivery without the rhythm of a real speaker.&lt;/p&gt;

&lt;p&gt;Something I watched recently made the stakes concrete. It was a demonstration of a system that allowed a person who had lost the ability to speak to communicate again in a synthetic version of their own voice.&lt;/p&gt;

&lt;p&gt;Programmes such as ElevenLabs’ Impact Program offer voice-restoration support to people living with conditions such as ALS.&lt;/p&gt;

&lt;p&gt;What stayed with me was not only the technical achievement, impressive as it was, but what had been returned to that person.&lt;/p&gt;

&lt;p&gt;I found myself asking what the equivalent could mean for someone whose language such a system does not yet speak.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I’m Building Towards&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;ElevenLabs is one of the clearest examples I know of carrying research the rest of the way into useful products.&lt;/p&gt;

&lt;p&gt;Its work spans speech generation, voice cloning, dubbing, and conversational agents. What inspires me is less any single model than the range of things its products allow people to do.&lt;/p&gt;

&lt;p&gt;I want Sauti Labs to make that same connection between serious technical work and useful products—for languages, accents, and settings that existing systems were not designed around, and with the linguistic, social, and technical challenges those contexts bring.&lt;/p&gt;

&lt;p&gt;Sauti Labs is an AI research and products company. Its research areas span speech, language, language resources, model intelligence, and responsible AI.&lt;/p&gt;

&lt;p&gt;The ambition extends beyond any single capability:&lt;/p&gt;

&lt;p&gt;Speech recognition&lt;br&gt;
Natural speech synthesis&lt;br&gt;
Voice cloning and dubbing&lt;br&gt;
Conversational systems and voice agents&lt;br&gt;
Datasets and models&lt;br&gt;
The infrastructure needed to develop and deploy these technologies&lt;/p&gt;

&lt;p&gt;It also extends beyond any one language.&lt;/p&gt;

&lt;p&gt;Dholuo shaped how I came to this work, but the work is meant for African languages generally.&lt;/p&gt;

&lt;p&gt;These capabilities represent a direction, not a claim that every product or system is already complete. There is still a great deal to build, test, and learn.&lt;/p&gt;

&lt;p&gt;I am learning from the communities and companies that came before me, and from the work itself.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why Take the Risk?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Speech and language AI is expensive and research-intensive. It requires data, compute, funding, expertise, and time—precisely the resources many young Africans who recognise these problems clearly do not have.&lt;/p&gt;

&lt;p&gt;I decided to take the risk anyway.&lt;/p&gt;

&lt;p&gt;I believe these problems are important enough to justify starting early, even with limited resources. I also want people like me to be among those who build these systems, not only among those who receive technologies already designed around other languages and contexts.&lt;/p&gt;

&lt;p&gt;This is not a claim that the work will be easy, or that conviction alone is enough. Building capable speech systems requires rigorous research, engineering, evaluation, and sustained investment.&lt;/p&gt;

&lt;p&gt;But the difficulty is not a reason to ignore the problem.&lt;/p&gt;

&lt;p&gt;I want African languages to take a full part in whatever AI becomes.&lt;/p&gt;

&lt;p&gt;That is why Sauti Labs exists.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Sources and Further Reading&lt;/strong&gt;&lt;br&gt;
Masakhane — Research and community work in African-language NLP.&lt;br&gt;
MasakhaNER: Named Entity Recognition for African Languages — Adelani et al. (2021), Transactions of the Association for Computational Linguistics.&lt;br&gt;
African languages for AI: The project that’s gathering a huge new dataset — The Conversation.&lt;br&gt;
Lelapa AI — African-language AI research and products.&lt;br&gt;
InkubaLM: A Small Language Model for Low-Resource African Languages — arXiv.&lt;br&gt;
Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages — arXiv.&lt;br&gt;
Robust Speech Recognition via Large-Scale Weak Supervision — Radford et al., the Whisper paper.&lt;br&gt;
ElevenLabs Impact Program Announcement — ElevenLabs.&lt;/p&gt;

&lt;p&gt;About Sauti Labs: Sauti Labs is working towards making African-language speech and language technology more useful, accessible, and deployable. Its long-term ambition spans research, models, datasets, and products across African languages.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>machinelearning</category>
      <category>startup</category>
    </item>
    <item>
      <title>Coverage Is Not Capability: What Whisper Taught Me About African Speech</title>
      <dc:creator>Bernard Abuto</dc:creator>
      <pubDate>Tue, 27 Oct 2026 21:01:38 +0000</pubDate>
      <link>https://dev.to/bernard_abuto_9970170b9c4/coverage-is-not-capability-what-whisper-taught-me-about-african-speech-39l2</link>
      <guid>https://dev.to/bernard_abuto_9970170b9c4/coverage-is-not-capability-what-whisper-taught-me-about-african-speech-39l2</guid>
      <description>&lt;p&gt;A language appearing in a model’s supported list says little about whether the model will work for your speakers. Why Whisper struggled with the speech I needed, and what I now check before building on any speech model.*&lt;/p&gt;

&lt;p&gt;When I built the first version of Scribe AI in July 2026, speech recognition was the part I expected to be easy.&lt;/p&gt;

&lt;p&gt;OpenAI’s Whisper was open, widely used, and described as multilingual. I ran it locally through Faster-Whisper, wired it into the live pipeline, and moved on to the parts I thought would be hard.&lt;/p&gt;

&lt;p&gt;That expectation did not survive the speech I actually needed to handle. On the African languages and accents I cared about, the results were disappointing—not reliable enough to build on.&lt;/p&gt;

&lt;p&gt;The problem was not a bug in my pipeline. When I went back to the Whisper paper, I found that the results were consistent with what the model’s own training data would lead us to expect.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What “Multilingual” Meant in Practice&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Whisper was trained on 680,000 hours of labelled audio. The paper’s dataset statistics show how that total is divided:&lt;/p&gt;

&lt;p&gt;438,218 hours of English speech recognition&lt;br&gt;
125,739 hours of speech translation into English&lt;br&gt;
117,113 hours of speech recognition across 96 other languages&lt;/p&gt;

&lt;p&gt;Within that last group, the distribution is extremely uneven.&lt;/p&gt;

&lt;p&gt;Swahili, one of Africa’s most widely spoken languages, is listed with approximately 5.4 hours of transcribed speech. Dholuo does not appear in Whisper’s language list.&lt;/p&gt;

&lt;p&gt;“The amount of pre-training speech recognition data for a given language is very predictive of zero-shot performance on that language.” — Radford et al., the Whisper paper&lt;/p&gt;

&lt;p&gt;Whisper was behaving in a way its training distribution helps explain.&lt;/p&gt;

&lt;p&gt;The mistake was mine: I had treated multilingual as a uniform property of the model, when it actually described a highly uneven distribution of language coverage and capability.&lt;/p&gt;

&lt;p&gt;The speech I needed sat in its thin tail.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Wider Coverage, the Same Question&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Whisper is not the only option, and the field has moved forward.&lt;/p&gt;

&lt;p&gt;In 2023, Meta released Massively Multilingual Speech (MMS), extending speech recognition to more than 1,100 languages. Much of its training data came from recorded readings of the New Testament, with roughly 32 hours per language.&lt;/p&gt;

&lt;p&gt;In late 2025, Meta released Omnilingual ASR, which it describes as supporting more than 1,600 languages under the Apache 2.0 licence.&lt;/p&gt;

&lt;p&gt;These are significant achievements. They expand what is possible for languages that previously had little or no speech recognition support.&lt;/p&gt;

&lt;p&gt;But they sharpen the lesson rather than eliminate it.&lt;/p&gt;

&lt;p&gt;Meta reports a character error rate below 10 for approximately 78% of the languages evaluated with Omnilingual ASR. Among low-resource languages, that figure falls to approximately 36%.&lt;/p&gt;

&lt;p&gt;Coverage is a list. Capability is a distribution of error rates—usually measured on someone else’s test set.&lt;/p&gt;

&lt;p&gt;When evaluating a speech model now, three questions matter more to me than whether a language appears on its supported-language list.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Does It Work on My Speakers?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Training and benchmark audio is often relatively clean and may consist largely of read speech.&lt;/p&gt;

&lt;p&gt;Real-world audio can be very different.&lt;/p&gt;

&lt;p&gt;A sermon, a phone call, or a customer-support conversation may contain spontaneous speech, unfamiliar accents, background noise, overlapping speakers, and frequent language switching.&lt;/p&gt;

&lt;p&gt;A model that performs well on a benchmark may struggle under those conditions.&lt;/p&gt;

&lt;p&gt;The most useful next step is to evaluate it on a small set of recordings representative of the intended environment, transcribed by hand to establish reference transcripts.&lt;/p&gt;

&lt;p&gt;Then measure word error rate (WER) or character error rate (CER), as appropriate for the language and task.&lt;/p&gt;

&lt;p&gt;The goal is not to find a model with an impressive headline metric. It is to find out how it performs on the speech your application actually needs to handle.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Can I Use It for What I Am Building?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Model quality is only one part of the decision. Licensing matters, too.&lt;/p&gt;

&lt;p&gt;The original MMS models were released for non-commercial use only. That may be acceptable for some research projects, but it can prevent their use in a commercial product under those terms.&lt;/p&gt;

&lt;p&gt;A model can perform well technically and still be unsuitable for a particular deployment.&lt;/p&gt;

&lt;p&gt;Before building around one, I now consider the licence alongside performance, intended use, redistribution conditions, and deployment requirements.&lt;/p&gt;

&lt;p&gt;An evaluation that ignores licensing can lead to an expensive architectural dead end.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Can It Run Where My Users Are?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Scribe AI is designed to run on the kind of laptop available at a church production desk, without depending on an internet connection.&lt;/p&gt;

&lt;p&gt;That rules out models that are too large or too slow for the target hardware, regardless of how accurate they might be in a data centre.&lt;/p&gt;

&lt;p&gt;Inference speed, memory requirements, quantisation support, and CPU performance all matter.&lt;/p&gt;

&lt;p&gt;A model that achieves excellent accuracy on powerful hardware but cannot keep up with live speech on the intended machine is not a practical solution for that setting.&lt;/p&gt;

&lt;p&gt;This is why model evaluation cannot end with a benchmark table. The deployment environment is part of the problem being solved.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Where That Left Scribe—and Me&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;For Scribe, the practical response was architectural.&lt;/p&gt;

&lt;p&gt;Speech recognition sits in its own module, separate from detection and display, so a better model can be introduced later. A formal interface for swapping models is still on the roadmap.&lt;/p&gt;

&lt;p&gt;Every detection also passes through a human operator before reaching the screen.&lt;/p&gt;

&lt;p&gt;Neither decision improves the underlying recognition quality. Together, however, they help keep the system usable while recognition remains imperfect.&lt;/p&gt;

&lt;p&gt;For me, the experience raised a larger question.&lt;/p&gt;

&lt;p&gt;If a developer in Kenya wants to build a voice product for people who speak Dholuo, Kalenjin, or even Kenyan-accented English, what should they build it on?&lt;/p&gt;

&lt;p&gt;Searching for an answer led me deeper into African speech and language AI as a field—and eventually to starting Sauti Labs.&lt;/p&gt;

&lt;p&gt;I wrote separately about that journey: Why I Started Sauti Labs.&lt;/p&gt;

&lt;p&gt;The broader motivation is straightforward: building useful speech technology requires more than making a language appear on a list. It requires understanding the data, the speakers, the deployment environment, and the limitations that remain.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Sources&lt;/strong&gt;&lt;br&gt;
Radford, A., et al. (2022). Robust Speech Recognition via Large-Scale Weak Supervision. arXiv.&lt;br&gt;
OpenAI. Whisper language list (tokenizer.py). GitHub.&lt;br&gt;
Pratap, V., et al. (2023). Scaling Speech Technology to 1,000+ Languages. arXiv.&lt;br&gt;
The Decoder. Meta’s new open-source models speak more than 1,100 languages.&lt;br&gt;
Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages. arXiv.&lt;br&gt;
The Decoder. Meta’s Omnilingual ASR brings speech recognition to 1,600 languages.&lt;/p&gt;

&lt;p&gt;Project: Scribe AI — an offline-first system for detecting scripture references during church services, with human approval before display.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>machinelearning</category>
      <category>openai</category>
    </item>
    <item>
      <title>The API Chose My Sampling Design</title>
      <dc:creator>Bernard Abuto</dc:creator>
      <pubDate>Tue, 27 Oct 2026 20:55:57 +0000</pubDate>
      <link>https://dev.to/bernard_abuto_9970170b9c4/the-api-chose-my-sampling-design-2g8c</link>
      <guid>https://dev.to/bernard_abuto_9970170b9c4/the-api-chose-my-sampling-design-2g8c</guid>
      <description>&lt;p&gt;How an undocumented sort order in a public biodiversity API put 97% of my background points in January or February—and what I changed so that kind of bias gets caught before a model ever sees it.&lt;/p&gt;

&lt;p&gt;Exploratory analysis is supposed to be the calm part of a project.&lt;/p&gt;

&lt;p&gt;For QueleaGuard, it was where I discovered that 122 of my 126 background records—96.8%—fell in January or February. Among the real red-billed quelea observations, only 20.3% did.&lt;/p&gt;

&lt;p&gt;Nothing had crashed. Every validation check up to that point had passed.&lt;/p&gt;

&lt;p&gt;If I had gone straight to training, the model might have produced impressive-looking results for entirely the wrong reason. That is exactly what made the problem dangerous.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why the Background Points Exist at All&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;QueleaGuard investigates where red-billed quelea occur around Ahero Rice Irrigation Scheme in Kenya.&lt;/p&gt;

&lt;p&gt;The occurrence data comes from GBIF, which aggregates biodiversity observations, including citizen-science sightings from platforms such as eBird. This is presence-only data: it records where birds were observed, but not necessarily where someone looked and found none.&lt;/p&gt;

&lt;p&gt;To model occurrence, I needed something to contrast the confirmed presences against.&lt;/p&gt;

&lt;p&gt;I used an approximate target-group background: observations of other bird species recorded in the same area serve as a proxy for places where birders were active but did not record quelea.&lt;/p&gt;

&lt;p&gt;The idea is to account, imperfectly, for observation effort. Bird sightings cluster around places people actually visit, so treating the landscape as though observation effort were evenly distributed would be misleading.&lt;/p&gt;

&lt;p&gt;This approach has limitations. Individual bird records are a noisier proxy for observation effort than complete checklists, and I documented that limitation when adopting the method.&lt;/p&gt;

&lt;p&gt;But the approach only helps if the background sample is drawn fairly across space and time.&lt;/p&gt;

&lt;p&gt;Mine was not.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why a Calendar Bias Is So Harmful&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Almost every environmental feature in the dataset changes with the seasons:&lt;/p&gt;

&lt;p&gt;Rainfall&lt;br&gt;
Temperature&lt;br&gt;
Dewpoint&lt;br&gt;
Vegetation greenness&lt;/p&gt;

&lt;p&gt;If nearly all background points fall in January and February while confirmed presences are distributed throughout the year, a model can distinguish the classes by learning the calendar rather than the ecology.&lt;/p&gt;

&lt;p&gt;It might appear to predict quelea occurrence accurately, when it is actually exploiting a sampling artefact.&lt;/p&gt;

&lt;p&gt;A confound like this does not produce an error. It produces a convincing result.&lt;/p&gt;

&lt;p&gt;This is one of the more dangerous forms of data-quality failure: the pipeline can run successfully, the model can train without complaint, and the evaluation can look respectable.&lt;/p&gt;

&lt;p&gt;The underlying research question, however, has been compromised before training even begins.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Finding the Cause&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The effort-proxy pool came from a script that queried GBIF's occurrence/search endpoint one year at a time, with a cap of 150 records per year through pagination.&lt;/p&gt;

&lt;p&gt;When I inspected the raw pool, several years—including 2000, 2011, and 2012—contained 150 January records out of 150.&lt;/p&gt;

&lt;p&gt;GBIF does not guarantee a default sort order that would make this sampling strategy safe. Empirically, results from a year-only query were heavily front-loaded with January observations. Whenever a year contained more records than the cap, pagination stopped before the sample reached the rest of the year.&lt;/p&gt;

&lt;p&gt;In effect, the API's response order was influencing my research sampling design.&lt;/p&gt;

&lt;p&gt;The uncomfortable part was that I had already fixed a year-level version of the same problem when I first built the pool.&lt;/p&gt;

&lt;p&gt;This was the same class of failure, one level deeper. It survived because I had not checked the monthly distribution of the background records.&lt;/p&gt;

&lt;p&gt;The API had returned data. My code had accepted it. My validation had passed.&lt;/p&gt;

&lt;p&gt;But I had not asked the question that mattered: What does the distribution of the returned records actually look like?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Fix: Stop Depending on the Order&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The endpoint accepts a month parameter alongside year.&lt;/p&gt;

&lt;p&gt;Rather than trying to work around an undocumented ordering, I removed the dependency on it.&lt;/p&gt;

&lt;p&gt;I changed the collection strategy to query individual year–month combinations:&lt;/p&gt;

&lt;p&gt;16 years × 12 months = 192 queries, each targeting approximately 13 records.&lt;/p&gt;

&lt;p&gt;The resulting pool looked substantially different.&lt;/p&gt;

&lt;p&gt;Measure Before  After&lt;br&gt;
Effort-proxy records    Year-capped, January-heavy  2,371 records, stratified by month&lt;br&gt;
Largest single-month share  90.7%   8.8%&lt;br&gt;
Records per month   Extremely uneven    182–208&lt;/p&gt;

&lt;p&gt;The key improvement was not simply collecting more records. It was making the sampling procedure explicit and reproducible rather than allowing the server's response order to determine which observations entered the dataset.&lt;/p&gt;

&lt;p&gt;Another problem surfaced during the rebuild&lt;/p&gt;

&lt;p&gt;While rebuilding the pool, I also closed a gap found during an earlier validation pass.&lt;/p&gt;

&lt;p&gt;Three background records appeared to come from the same eBird checklist as a confirmed quelea sighting. That directly contradicted the purpose of using those records as background.&lt;/p&gt;

&lt;p&gt;The rebuilt pipeline now excludes any effort-proxy record that shares both a grid cell and a date with a confirmed presence before sampling, rather than relying on later validation to catch the contradiction.&lt;/p&gt;

&lt;p&gt;That rule removed 94 records that could otherwise have reintroduced the same problem.&lt;/p&gt;

&lt;p&gt;The lesson here is important: fixing the monthly distribution alone would not have been enough. The sampling procedure also needed to enforce the assumptions behind the background data.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Checking Before Moving On&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I deliberately held back the expensive part of the workflow—re-extracting rainfall, meteorological, and vegetation features for the new records—until the corrected sample passed its own validation checks.&lt;/p&gt;

&lt;p&gt;The results were encouraging:&lt;/p&gt;

&lt;p&gt;Monthly distribution: The largest single-month share was 18.8% of presences in November and 16.8% of background records in May. Neither class was concentrated in a single season.&lt;br&gt;
Cross-class contradictions: There were zero cell-and-date pairs shared between presences and background.&lt;br&gt;
Year coverage: Both classes spanned 2000–2026, with greater representation from 2021–2024 in both. This was consistent with the growth of eBird observations rather than an obvious artefact of the query strategy.&lt;br&gt;
Spatial coverage: Background points now occupied 81 grid cells, up from 49.&lt;/p&gt;

&lt;p&gt;I also kept the month-biased files, renamed and documented as superseded rather than deleting them.&lt;/p&gt;

&lt;p&gt;That matters for reproducibility. If someone later wants to understand the original problem or investigate how much the sampling bias affected the analysis, the evidence is still available.&lt;/p&gt;

&lt;p&gt;A corrected dataset is useful. A traceable record of how it became corrected is even better.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I Took From It&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Four lessons came out of this failure.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Whenever a data request has a cap, find out what determines which records are returned.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If the answer is effectively "whatever order the server uses," the sampling process is not fully under your control.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Stratify on the dimensions the model will be sensitive to.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Season influences almost every environmental feature in this project. Month therefore had to be part of the sampling design, not something I hoped would average out.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Check every dimension the model can see, separately for each class.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A pooled distribution can hide differences between presences and background records. Looking at the two classes separately exposed the problem.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Document the failure instead of quietly patching it.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The decision log now explains why the dataset looks the way it does. That record is part of the research, not administrative overhead.&lt;/p&gt;

&lt;p&gt;These practices are useful beyond biodiversity modelling. Any pipeline that retrieves, filters, samples, or combines data can introduce a bias that remains invisible until someone examines the distribution of what actually came through.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Research Models Have Not Been Trained Yet&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;QueleaGuard has not trained its research models yet, so I cannot tell you how much correcting this bias will change the eventual results.&lt;/p&gt;

&lt;p&gt;I do not want to claim an improvement in predictive performance before the experiments have been run.&lt;/p&gt;

&lt;p&gt;What I can say is that the dataset now has a more defensible sampling procedure, clearer validation rules, and a documented record of the problem that motivated the changes.&lt;/p&gt;

&lt;p&gt;That is meaningful progress even before the first model is trained.&lt;/p&gt;

&lt;p&gt;Whatever the results turn out to be, they should not simply be a measurement of what month it is.&lt;/p&gt;

&lt;p&gt;Project: QueleaGuard — a research project investigating environmental, spatial, temporal, and agricultural drivers of red-billed quelea occurrence around Ahero Rice Irrigation Scheme in Kenya.&lt;/p&gt;

</description>
      <category>api</category>
      <category>data</category>
      <category>datascience</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>AI as a Detector, Not a Decision-Maker</title>
      <dc:creator>Bernard Abuto</dc:creator>
      <pubDate>Tue, 27 Oct 2026 20:46:23 +0000</pubDate>
      <link>https://dev.to/bernard_abuto_9970170b9c4/ai-as-a-detector-not-a-decision-maker-4bn2</link>
      <guid>https://dev.to/bernard_abuto_9970170b9c4/ai-as-a-detector-not-a-decision-maker-4bn2</guid>
      <description>&lt;p&gt;Designing Scribe AI’s live path: offline speech recognition, rule-based scripture detection, and a human approval step that keeps a wrong guess off a church screen.&lt;/p&gt;

&lt;p&gt;Scribe AI listens to a church service and tries to catch scripture references as they are spoken — “turn with me to John chapter three, verse sixteen” — so the verse can be put on screen without a volunteer scrambling to type it.&lt;/p&gt;

&lt;p&gt;It is a small system, but it operates in an unforgiving environment: whatever it gets wrong is projected in front of the whole room.&lt;/p&gt;

&lt;p&gt;Two constraints shaped nearly every design decision: the system must keep working without the internet, and nothing it detects should reach the screen without human approval.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Offline Is a Requirement, Not a Feature&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;In the settings Scribe is designed for, a dropped connection in the middle of a service is ordinary, not an edge case. So everything in the live path runs locally against local data: microphone capture, speech recognition, detection, Bible lookup, and display. There is no network call anywhere in this path.&lt;/p&gt;

&lt;p&gt;Speech recognition uses Faster-Whisper’s small model on the CPU, quantised to int8, with greedy decoding (beam_size=1) and voice-activity filtering to skip silence. Audio is captured at 16 kHz and transcribed in five-second chunks.&lt;/p&gt;

&lt;p&gt;These are deliberate trade-offs for the kind of laptop a church production desk actually has: a smaller, faster model designed to keep up with speech, rather than a potentially more accurate one that falls behind it.&lt;/p&gt;

&lt;p&gt;Offline operation is not simply a convenience. If the internet connection fails during a service, the system should still be able to do its job.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Rules, Where Rules Are Enough&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;It would have been easy to reach for a learned model to find scripture references in the transcript. I did not, because the detection problem does not require one.&lt;/p&gt;

&lt;p&gt;Spoken references tend to follow a limited number of patterns. The detector covers four common forms:&lt;/p&gt;

&lt;p&gt;John 3:16&lt;br&gt;
John chapter 3 verse 16&lt;br&gt;
John 3 16&lt;br&gt;
John 316&lt;/p&gt;

&lt;p&gt;The last example matters because speech recognition can compress a spoken reference into a run of digits.&lt;/p&gt;

&lt;p&gt;The patterns match against all 66 Bible book names and common abbreviations. Each match is then checked against the actual number of chapters in that book. For example, Genesis 60 is rejected because Genesis has only 50 chapters.&lt;/p&gt;

&lt;p&gt;That validation is particularly important for compressed references, where the detector must decide where the chapter number ends and the verse number begins.&lt;/p&gt;

&lt;p&gt;The benefit is not accuracy for its own sake. It is inspectability. When the detector fires—or fails to fire—the logic behind the decision can be examined. There is no confidence score, and I have not pretended there is one. The detector checks whether a reference is valid according to its rules.&lt;/p&gt;

&lt;p&gt;This is a useful reminder that not every component in an AI system needs to be another learned model. Sometimes a small, explicit rule set is easier to understand, test, and maintain.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;References Don't Respect Chunk Boundaries&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A preacher does not pause neatly every five seconds.&lt;/p&gt;

&lt;p&gt;“John chapter three” might land at the end of one audio chunk, while “verse sixteen” arrives at the beginning of the next. If each chunk were processed independently, the detector could miss the complete reference.&lt;/p&gt;

&lt;p&gt;To address this, when a chunk produces no detection, the system carries the last 60 characters of its transcript forward and prepends them to the next chunk.&lt;/p&gt;

&lt;p&gt;When a chunk does produce a detection, the carried text is cleared to reduce the chance of detecting the same reference twice.&lt;/p&gt;

&lt;p&gt;It is a small implementation detail, but it reflects a broader engineering problem: real-time systems have to account for the difference between how data is processed and how events actually occur.&lt;/p&gt;

&lt;p&gt;The audio arrives in chunks. Human speech does not follow those boundaries.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Approval Step&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A detected reference never goes straight to the screen.&lt;/p&gt;

&lt;p&gt;Instead, it appears in an operator console: a separate, always-on-top window where a person can approve the detection, dismiss it, or type a reference manually. Only an operator's action—approving a detection or entering a reference themselves—puts a verse on the display.&lt;/p&gt;

&lt;p&gt;This is the system's answer to a simple question:&lt;/p&gt;

&lt;p&gt;What happens when the AI is wrong?&lt;/p&gt;

&lt;p&gt;A misheard reference costs the operator a click. Without the approval gate, it could mean displaying the wrong verse in front of an entire congregation.&lt;/p&gt;

&lt;p&gt;Manual entry is equally important. The operator is never dependent on the detector. If the system misses a reference, the service can continue without waiting for the AI to recover.&lt;/p&gt;

&lt;p&gt;The implementation also separates audio processing from the user interface. Audio capture and transcription run on a background thread, while the console and display belong to the Tkinter UI thread.&lt;/p&gt;

&lt;p&gt;The audio thread never manipulates the interface directly. Instead, it schedules work on the UI thread with root.after, helping prevent the two threads from interfering with the same widgets.&lt;/p&gt;

&lt;p&gt;The design principle is broader than this particular application: automating detection does not require automating the final decision.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Being Clear About What Is Not Connected&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Scribe also contains a service-state machine—keyword rules that track whether the service is in worship, sermon, prayer, or announcements—and a lyric detector that fuzzy-matches transcript text against a song library.&lt;/p&gt;

&lt;p&gt;Both are implemented and tested. Neither is connected to the live microphone path yet. They run together in a separate pipeline driven by simulated text.&lt;/p&gt;

&lt;p&gt;That distinction matters.&lt;/p&gt;

&lt;p&gt;It would be easy to describe the modules the system contains and leave readers with the impression that everything operates live. But having a working module is not the same as integrating it into the complete application.&lt;/p&gt;

&lt;p&gt;There are other limitations, too. The lyric matcher's threshold has only been tested on clean text, and the song library contains placeholder data.&lt;/p&gt;

&lt;p&gt;These limitations belong in the description of the system, not in a footnote. A clear account of what works, what has been tested, and what remains disconnected is more useful than a broader claim that the implementation cannot yet support.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What Comes Next&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;There are three main priorities for the next stage of Scribe AI:&lt;/p&gt;

&lt;p&gt;Build a real-world evaluation set. Use recordings from actual services, including accents, crosstalk, and music, to measure detection performance under the conditions the system will encounter.&lt;br&gt;
Integrate the separate pipelines. Create one live entry point that includes service-state tracking and lyric detection, replacing the split between the live path and the simulated pipeline.&lt;br&gt;
Make speech recognition interchangeable. Introduce an interface that allows other models—including models better suited to African languages—to be swapped in and compared fairly.&lt;/p&gt;

&lt;p&gt;These steps move the project beyond demonstrating that its components work in isolation and towards evaluating how well the complete system performs in practice.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Model Is Not the Whole System&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The general lesson I keep returning to is this: the most important design decision in an applied AI system is often not the model.&lt;/p&gt;

&lt;p&gt;It is where the model's output is allowed to go, what happens when that output is wrong, and who gets to say no.&lt;/p&gt;

&lt;p&gt;In Scribe AI, speech recognition proposes what it heard. Rules check whether a scripture reference is valid. A human operator decides whether the verse reaches the screen.&lt;/p&gt;

&lt;p&gt;That separation is intentional. In a live environment where an incorrect output is immediately visible to everyone in the room, a useful AI system is not simply one that produces answers quickly. It is one designed to handle uncertainty without taking control away from the person responsible for the outcome.&lt;/p&gt;

&lt;p&gt;Project: Scribe AI — an offline-first system for detecting scripture references during church services, with human approval before display.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>architecture</category>
      <category>systemdesign</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>Modelling the Landlord Who Isn’t There</title>
      <dc:creator>Bernard Abuto</dc:creator>
      <pubDate>Tue, 27 Oct 2026 20:40:04 +0000</pubDate>
      <link>https://dev.to/bernard_abuto_9970170b9c4/modelling-the-landlord-who-isnt-there-5flb</link>
      <guid>https://dev.to/bernard_abuto_9970170b9c4/modelling-the-landlord-who-isnt-there-5flb</guid>
      <description>&lt;p&gt;Modelling the Landlord Who Isn’t There&lt;/p&gt;

&lt;p&gt;Most rental software assumes both sides of a tenancy are on the platform. That assumption fails for many tenants, so Rentra had to represent the relationship before the landlord arrives.&lt;/p&gt;

&lt;p&gt;The first version of most rental-management software makes a quiet assumption: the landlord signs up, adds a property, and invites tenants. Every record begins on the landlord’s side.&lt;/p&gt;

&lt;p&gt;When I started building Rentra in June 2026, I began from that assumption too. The first entities in the repository were users and the passkeys a landlord issues to tenants.&lt;/p&gt;

&lt;p&gt;It took about two weeks for the assumption to break.&lt;/p&gt;

&lt;p&gt;If Rentra only works once a landlord has joined, a tenant whose landlord never joins gets nothing: no rent history, no reminders, and no record of what they have paid. For a product meant to make rental life more orderly, the tenant is often the person who most needs that record.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two Kinds of Tenant&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Rentra now distinguishes two kinds of tenant.&lt;/p&gt;

&lt;p&gt;A managed tenant is linked to a real tenancy: a unit, in a building, in a landlord’s portfolio.&lt;/p&gt;

&lt;p&gt;An independent tenant is someone whose landlord is not on Rentra. They declare their own rental—a name for the property, the rent, how often it is due, and where it is paid—and Rentra keeps the record on their behalf.&lt;/p&gt;

&lt;p&gt;In the code, that self-declared rental is its own entity, ShadowRentalRelationship, rather than a tenancy with some fields left empty.&lt;/p&gt;

&lt;p&gt;There were three reasons for keeping it separate.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The ownership relationship is different.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A managed tenancy has an owner on the other side; a self-declared rental has only the tenant’s account of the arrangement. Mixing them in one table would mean every query about rent, arrears, or occupancy has to ask which kind of record it is looking at. Every authorisation rule would also have to account for a landlord who might not exist on the platform.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The information is genuinely different.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A self-declared rental records where rent is actually paid—an M-Pesa Paybill, a Till number, or a bank—and allows daily, weekly, or monthly billing cycles, because those are the facts a tenant knows.&lt;/p&gt;

&lt;p&gt;A managed tenancy gets its structure from the landlord’s records instead.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Uncertainty remains visible.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Each self-declared rental carries a status: unverified, converted, or ended.&lt;/p&gt;

&lt;p&gt;That distinction matters. A tenant’s declaration is useful information, but it is not the same as a tenancy verified by both sides. The data model should not imply otherwise.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;When the Landlord Arrives&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The interesting part is what happens when the absent landlord eventually joins.&lt;/p&gt;

&lt;p&gt;Rentra’s approach is a passkey: a code the landlord generates for a specific unit. A new tenant can redeem it to register directly as a managed tenant. An independent tenant who already has an account can redeem it to link that account to the real unit.&lt;/p&gt;

&lt;p&gt;But the transition is not complete yet.&lt;/p&gt;

&lt;p&gt;The step after redemption—carrying the self-declared rental’s history into the managed tenancy—has not been implemented. The converted status exists in the schema, but no code path sets it, and payments are not yet linked to self-declared rentals at all.&lt;/p&gt;

&lt;p&gt;I would rather leave that gap visible than paper over it, because merging two records about the same rental is where the hard questions live.&lt;/p&gt;

&lt;p&gt;Which rent amount wins if the two disagree?&lt;/p&gt;

&lt;p&gt;Does a payment recorded by the tenant become confirmed once the landlord can see it, or only once the landlord agrees?&lt;/p&gt;

&lt;p&gt;What happens when the tenant’s record and the landlord’s record disagree about the history of the tenancy?&lt;/p&gt;

&lt;p&gt;These are not merely database questions. They affect what the system claims to know, what each party can trust, and how disputes might be handled.&lt;/p&gt;

&lt;p&gt;A conversion mechanism that simply merges records would conceal those decisions rather than resolve them.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Roles Are Not Ownership&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The second modelling lesson came later, when I added role-based authorisation.&lt;/p&gt;

&lt;p&gt;Rentra has three roles: tenant, landlord, and caretaker. The backend enforces them with guards on its controllers, so a modified mobile client cannot grant itself permissions simply by changing its local state.&lt;/p&gt;

&lt;p&gt;But role checks answer only one question:&lt;/p&gt;

&lt;p&gt;Is this kind of user allowed to do this kind of thing?&lt;/p&gt;

&lt;p&gt;They do not answer the question that matters most for a tenant’s data:&lt;/p&gt;

&lt;p&gt;Is this your record?&lt;/p&gt;

&lt;p&gt;The tenant role might permit a user to view a tenancy, but it must not permit them to view someone else’s tenancy merely because they are a tenant.&lt;/p&gt;

&lt;p&gt;That second check, ownership scoping, is still being built. It requires a different mechanism because it depends on the relationship between a specific user and a specific record, rather than on a label attached to the user.&lt;/p&gt;

&lt;p&gt;Caretakers expose the same gap from the other side.&lt;/p&gt;

&lt;p&gt;A caretaker acts on a landlord’s behalf—but for which buildings?&lt;/p&gt;

&lt;p&gt;There is not yet an entity that scopes a caretaker to particular properties. The role exists, but the relationship that would make it safe to grant access across specific properties is not yet modelled.&lt;/p&gt;

&lt;p&gt;The lesson is that role-based access control and resource-level authorisation solve different problems. Having the right role should not automatically grant access to every resource of that type.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Two Vocabularies&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;One smaller decision has held up well.&lt;/p&gt;

&lt;p&gt;Internally, the backend talks about organisations and rooms. People using the app see a property portfolio and units.&lt;/p&gt;

&lt;p&gt;The internal names are precise and stable, while the product names are the ones people actually use.&lt;/p&gt;

&lt;p&gt;Keeping them separate lets the data model be refactored without rewriting the interface. It also lets the interface speak plainly without exposing database concepts to users.&lt;/p&gt;

&lt;p&gt;The distinction is small, but it reflects a broader principle: the language used to implement a domain does not have to be the language used to explain it to customers.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I Took From It&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Domain modelling is mostly the work of noticing which real situations your first schema cannot represent.&lt;/p&gt;

&lt;p&gt;The absent landlord is not an edge case in the markets Rentra is designed for. I suspect it is closer to the norm.&lt;/p&gt;

&lt;p&gt;Giving that relationship its own entity, with an explicit status and an explicit path towards becoming something else, made the system more honest about what it actually knows.&lt;/p&gt;

&lt;p&gt;It also made the unfinished work easier to identify. The conversion process still needs to reconcile conflicting records. Payments need to be linked to self-declared rentals. Ownership scoping needs to be completed. Caretaker access needs a property-level relationship.&lt;/p&gt;

&lt;p&gt;Those gaps are easier to reason about when they are visible instead of being hidden behind a schema that pretends every tenancy starts with a landlord.&lt;/p&gt;

&lt;p&gt;The repository’s README now lists, line by line, which pieces exist in code and which do not.&lt;/p&gt;

&lt;p&gt;That list has been almost as useful to me as the code itself.&lt;/p&gt;

&lt;p&gt;Project: Rentra — rental-management software designed to help tenants and landlords maintain clearer rental records, including situations where a landlord has not yet joined the platform.&lt;/p&gt;

</description>
      <category>architecture</category>
      <category>product</category>
      <category>saas</category>
      <category>startup</category>
    </item>
    <item>
      <title>Spatial Cross-Validation When the Data Won’t Spread Out</title>
      <dc:creator>Bernard Abuto</dc:creator>
      <pubDate>Tue, 27 Oct 2026 20:04:11 +0000</pubDate>
      <link>https://dev.to/bernard_abuto_9970170b9c4/spatial-cross-validation-when-the-data-wont-spread-out-5dja</link>
      <guid>https://dev.to/bernard_abuto_9970170b9c4/spatial-cross-validation-when-the-data-wont-spread-out-5dja</guid>
      <description>&lt;p&gt;Spatial Cross-Validation When the Data Won’t Spread Out&lt;/p&gt;

&lt;p&gt;Random k-fold cross-validation can overstate model performance on clustered ecological data. But the obvious alternatives failed on my dataset too. Here is the spatial fold design I settled on for QueleaGuard, why the first two attempts failed, and what I verified before training a model.&lt;/p&gt;

&lt;p&gt;Before QueleaGuard trains a research model, it needs a reliable way to evaluate whether that model generalises. For ecological point data, this is harder than it sounds. I tried two validation designs that failed before settling on one I trust.&lt;/p&gt;

&lt;p&gt;&lt;u&gt;&lt;strong&gt;Why Random Folds Are Not an Option&lt;/strong&gt;&lt;/u&gt;&lt;/p&gt;

&lt;p&gt;Observations close together in space tend to share characteristics: similar rainfall, similar vegetation, and similar distances to water.&lt;/p&gt;

&lt;p&gt;If a random split places one observation in the training set and its neighbour a few hundred metres away in the test set, the model is partly being evaluated on data it has effectively already seen.&lt;/p&gt;

&lt;p&gt;Research by Roberts et al. (2017) and Valavi et al. (2019) explains why ignoring spatial structure can underestimate predictive error and why spatially structured validation is often necessary.&lt;/p&gt;

&lt;p&gt;The question for QueleaGuard was not whether to account for spatial structure. It was how to do so with an extremely uneven dataset.&lt;/p&gt;

&lt;p&gt;&lt;u&gt;&lt;strong&gt;Attempt 1: Use the Grid I Already Had&lt;/strong&gt;&lt;/u&gt;&lt;/p&gt;

&lt;p&gt;The analysis already divided the area around Ahero into 328 grid cells, each approximately 5.5 km across.&lt;/p&gt;

&lt;p&gt;Blocking observations by these cells seemed like the natural starting point. But only 93 of the 328 cells contained records, and one persistent site along the Winam Gulf shoreline accounted for 62 of the 266 observations.&lt;/p&gt;

&lt;p&gt;Grouping folds by grid cell would have produced folds dominated by a small number of cells, while others would have contributed little or no data.&lt;/p&gt;

&lt;p&gt;The existing grid was useful for spatial analysis, but it did not automatically provide a workable cross-validation design.&lt;/p&gt;

&lt;p&gt;&lt;u&gt;&lt;strong&gt;Attempt 2: Cluster All the Points&lt;/strong&gt;&lt;/u&gt;&lt;/p&gt;

&lt;p&gt;Next, I applied k-means clustering to the coordinates of all records, using five clusters.&lt;/p&gt;

&lt;p&gt;One resulting fold contained no presence observations at all.&lt;/p&gt;

&lt;p&gt;In hindsight, the problem was straightforward. Background points were distributed across the entire 50 km analysis extent, while presence observations were concentrated around the rice scheme and the persistent shoreline site.&lt;/p&gt;

&lt;p&gt;Clustering all points together could isolate a region containing only background observations.&lt;/p&gt;

&lt;p&gt;That creates an evaluation problem: a fold without positive cases cannot provide meaningful estimates of recall or F1, and precision may also be undefined depending on the model's predictions.&lt;/p&gt;

&lt;p&gt;A spatial partition can look reasonable geographically and still be unsuitable for the evaluation task.&lt;/p&gt;

&lt;p&gt;&lt;u&gt;&lt;strong&gt;What Worked: Anchor the Clusters on Presences&lt;/strong&gt;&lt;/u&gt;&lt;/p&gt;

&lt;p&gt;I changed the clustering strategy.&lt;/p&gt;

&lt;p&gt;Instead of fitting k-means to all observations, I fitted it to presence coordinates only, using five clusters and a fixed random seed. I then assigned every record—both presence and background—to its nearest centroid.&lt;/p&gt;

&lt;p&gt;This placed the centroids among the observed presences, ensuring that every cluster contained at least one presence observation.&lt;/p&gt;

&lt;p&gt;The resulting fold distribution was:&lt;/p&gt;

&lt;p&gt;Fold| Presence| Background| Total&lt;br&gt;
0| 8| 42| 50&lt;br&gt;
1| 9| 5| 14&lt;br&gt;
2| 11| 20| 31&lt;br&gt;
3| 36| 22| 58&lt;br&gt;
4| 69| 44| 113&lt;br&gt;
Total| 133| 133| 266&lt;/p&gt;

&lt;p&gt;The folds are uneven. I kept them that way.&lt;/p&gt;

&lt;p&gt;Fold 4 is large because it contains the persistent shoreline site, while fold 1 is small. Forcing the folds into equal sizes would mean prioritising numerical balance over the spatial structure I was trying to preserve.&lt;/p&gt;

&lt;p&gt;The objective was not to make every fold identical. It was to construct spatially meaningful partitions that remained usable for evaluation.&lt;/p&gt;

&lt;p&gt;&lt;u&gt;&lt;strong&gt;Checking That the Folds Do What They Claim&lt;/strong&gt;&lt;/u&gt;&lt;/p&gt;

&lt;p&gt;I checked the resulting assignments against several invariants.&lt;/p&gt;

&lt;p&gt;The persistent site stays together. All 62 records from the shoreline site landed in one fold, preventing that site's observations from being split between training and testing.&lt;/p&gt;

&lt;p&gt;Grid-cell splitting is limited. Only one grid cell was split across two folds. Its two records are 4.6 km apart, farther than the largest 2 km neighbourhood used by the rice-landscape features. Consequently, those records do not share the same feature window.&lt;/p&gt;

&lt;p&gt;This does not eliminate every possible source of spatial autocorrelation, and I do not claim that it does.&lt;/p&gt;

&lt;p&gt;Every record is accounted for. The fold file covers the modelling dataset exactly: 266 records, each assigned once, without duplicates, across five fold IDs. Every fold contains both classes.&lt;/p&gt;

&lt;p&gt;The assignment is reproducible. Repeating the clustering with the same data and configuration produces zero mismatches in fold assignments.&lt;/p&gt;

&lt;p&gt;These checks do not prove that the validation design eliminates all spatial dependence. They establish that the implementation satisfies its intended assignment rules and addresses the specific failure modes encountered in the earlier attempts.&lt;/p&gt;

&lt;p&gt;&lt;u&gt;&lt;strong&gt;A Related Decision: Keep Coordinates Out of the Model&lt;/strong&gt;&lt;/u&gt;&lt;/p&gt;

&lt;p&gt;The same reasoning influenced the baseline predictor set.&lt;/p&gt;

&lt;p&gt;I deliberately excluded raw latitude and longitude.&lt;/p&gt;

&lt;p&gt;With observations this geographically concentrated, a model given raw coordinates could learn where the clusters occur instead of learning which environmental conditions are associated with quelea occurrence.&lt;/p&gt;

&lt;p&gt;That would complicate interpretation: even with spatially structured folds, I would need to distinguish environmental relationships from geographical patterns that the model could exploit.&lt;/p&gt;

&lt;p&gt;The baseline instead uses ten environmental and landscape predictors, plus season, selected after checking candidate predictors for redundancy.&lt;/p&gt;

&lt;p&gt;The intention is to evaluate relationships with environmental and landscape conditions rather than hand the model explicit coordinates as shortcuts.&lt;/p&gt;

&lt;p&gt;&lt;u&gt;&lt;strong&gt;What This Design Does Not Fix&lt;/strong&gt;&lt;/u&gt;&lt;/p&gt;

&lt;p&gt;Fold 1 contains only 14 observations, nine of which are presences. Metrics computed on such a small sample will be noisy.&lt;/p&gt;

&lt;p&gt;The appropriate response is to report per-fold results alongside the aggregate results, rather than allowing one average to conceal substantial variation between folds.&lt;/p&gt;

&lt;p&gt;The dataset itself is also small: 133 presence observations. No cross-validation design can manufacture additional independent evidence from those records.&lt;/p&gt;

&lt;p&gt;There are further limitations. Assigning records to the nearest presence-derived centroid does not guarantee a minimum geographic separation between training and test observations, nor does it remove all spatial autocorrelation. The design should therefore be treated as a deliberate spatial partition, not proof of complete spatial independence.&lt;/p&gt;

&lt;p&gt;Most importantly, no model has been trained using these folds yet. This work establishes the evaluation design before performance results exist.&lt;/p&gt;

&lt;p&gt;That order matters. If the validation scheme is fixed and documented first, there is less temptation to choose whichever design produces the most favourable scores.&lt;/p&gt;

&lt;p&gt;For QueleaGuard, the immediate achievement is not a high-performing model. It is a reproducible evaluation setup whose assumptions, trade-offs, and limitations are explicit.&lt;/p&gt;




&lt;p&gt;Project: QueleaGuard — research into environmental, spatial, temporal, and agricultural drivers of Red-billed Quelea occurrence around the Ahero Rice Irrigation Scheme.&lt;/p&gt;

&lt;p&gt;&lt;u&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/u&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Roberts, D. R., et al. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography. &lt;a href="https://doi.org/10.1111/ecog.02881" rel="noopener noreferrer"&gt;https://doi.org/10.1111/ecog.02881&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Valavi, R., et al. (2019). blockCV: An R package for generating spatially or environmentally separated folds for k-fold cross-validation of species distribution models. Methods in Ecology and Evolution. &lt;a href="https://doi.org/10.1111/2041-210X.13107" rel="noopener noreferrer"&gt;https://doi.org/10.1111/2041-210X.13107&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>machinelearning</category>
      <category>datascience</category>
      <category>python</category>
    </item>
  </channel>
</rss>
