DEV Community: Nsukami

Vectors

Nsukami — Thu, 26 Feb 2026 00:00:00 +0000

Disclaimers :

Opinions : The views expressed here are my own unless otherwise specified. They do not represent any employer, organization, or affiliated group.
Code examples : All code examples are educational. Production use requires careful consideration of performance, security, and your specific data characteristics.
Model choice : The all-MiniLM-L6-v2 model is a practical default, but embedding model selection should depend on your domain. Consider fine-tuning for specialized use cases.
This article is for : Django developers curious about semantic search and vector embeddings, or anyone wanting to understand how vectors fit alongside traditional structured data.
**This article was written on a Framework laptop 16, yeah!

Hook

When I start a new Django project, the instinct is immediate and familiar: open models.py and start writing classes. I think about my entity, its attributes, its relationships, and I translate that mental model almost directly into Python. That's great. It works. Since so many years. It's readable. It's what Django was built for. Amazing. Perfect. Flawless.

If you follow my amazing and colossal work, you know that since more that two years now, I'm working on a project in the public health sector. I'm handling lots and lots and lots of data. I have to answer interesting questions like... Yet, I'm still reprensenting my data using classes and attributes. Reflexes.

There's another way to represent data that's been quietly powering search engines, recommendation systems, and AI applications for years, it's increasingly accessible to application developers like me. I should learn to use it more often. Instead of describing data as a collection of named attributes, I can describe it as a point in a high-dimensional space , a vector.

This article walks through what that actually means, when it matters, and how to implement both approaches side by side in a real Django project. To increase your chances of understanding, it is recommended that you listen to this track while reading the article. Enjoy!

The classical approach: data as structure

When you model a medical facility in a surveillance platform, you probably write something like this:

class HealthFacility(models.Model):
    name = models.CharField(max_length=255)
    region = models.CharField(max_length=100)
    facility_type = models.CharField(max_length=50) # hospital, clinic, health_post
    capacity = models.IntegerField()
    services = models.ManyToManyField('Service')

This is structured representation. Data lives in named slots. Querying it means matching those slots exactly:

# Find clinics in Dakar with pediatric services
HealthFacility.objects.filter(
    region="Dakar",
    facility_type="clinic",
    services__name="pediatrics"
)

This is powerful for what it is. You know exactly what you're looking for, the schema enforces consistency, and SQL is extraordinarily good at these kinds of exact, predicate-based lookups.

The limitation reveals itself the moment your questions become fuzzy. What if you want facilities that are similar to a given one — without being able to define exactly what "similar" means? What if a user types "district hospital with maternal care near the coast" and you need to find the best match? Predicate-based filtering breaks down because similarity isn't a predicate. It's a geometry problem.

The vector approach: data as position

A vector is simply an ordered list of numbers:

[0.82, 0.14, 0.67, 0.03, 0.91, ...]

The idea is that you can encode the meaning or characteristics of any piece of data as a point in a high-dimensional space, such that things which are semantically similar end up close together , and things that are different end up far apart.

This is not hand-crafted feature engineering (though it can be). Modern embedding models — neural networks trained on massive datasets — can take a sentence, a document, even an image, and produce a dense vector that captures its semantic content. The model has learned, from context, what concepts relate to each other. "Maternal health clinic" and "obstetric care center" will produce vectors that are very close. "Warehouse" will be far away.

The distance between two vectors is typically measured with cosine similarity:

similarity = cos(θ) = (A · B) / (||A|| × ||B||)

A value of 1 means identical direction (semantically equivalent). A value of 0 means orthogonal (unrelated). This single number replaces the entire predicate matching logic.

Why this is not just for AI projects

You might think: "I don't need semantic search. I need to store patient records." That's fair. This is exactly how I've done it on the 4S project. But I'm more and more facing these new types of scenarios:

Duplicate detection. Two users register "Hôpital Principal de Dakar" and "hopital principal dakar". Exact matching fails. Vector similarity catches them as near-duplicates.

Recommendation."Users who viewed this report also found these relevant." You don't need a collaborative filtering setup — you can just find the reports whose vectors are closest to the one currently being read.

Free-text search that actually works. Users don't search with your field names. They type natural language. Vectors let you match that intent.... This one stung me very badly recently.

Clustering without labels. You have 10,000 epidemiological case reports. You want to discover natural groupings without writing filtering rules. Find neighborhoods in vector space.... We are working in this particular topic. I look at how we are implementing this feature and I am already disgusted by the direction we are taking. Marvelous.

Anomaly detection. A new weekly report has a vector that is very far from all historical ones. Flag it for review.

Both approaches in Django: a concrete example

Let's build a very small project that models epidemiological reports using both approaches. We'll use pgvector, a PostgreSQL extension, so vectors live right next to our traditional Django models. No separate vector databases, no need for new infrastructure if you're already on the amazing Postgres.

Setup

pip install django pgvector sentence-transformers psycopg2-binary # easy peasy

Enable the extension in a Django migration:

from pgvector.django import VectorExtension

class Migration(migrations.Migration):
    operations = [
        VectorExtension(),
    ]

The traditional model

# surveillance/models.py

from django.db import models
from pgvector.django import VectorField

class EpiReport(models.Model):
    """
    Classical structured representation.
    Every piece of information lives in a named, typed field.
    """
    title = models.CharField(max_length=255)
    region = models.CharField(max_length=100)
    disease = models.CharField(max_length=100)
    week = models.IntegerField()
    year = models.IntegerField()
    case_count = models.IntegerField(default=0)
    severity = models.CharField(
        max_length=20,
        choices=[("low", "Low"), ("moderate", "Moderate"), ("high", "High")]
    )
    summary = models.TextField()

    # The vector representation lives alongside the structured one.
    # 384 dimensions is the output size of the model we'll use.
    embedding = VectorField(dimensions=384, null=True, blank=True)

    class Meta:
        indexes = [
            # Traditional index for structured queries
            models.Index(fields=["region", "disease", "year"]),
        ]

    def __str__ (self):
        return f"{self.title} ({self.region}, W{self.week}/{self.year})"

Note that both representations coexist in the same table. This is intentional — we're not replacing one with the other. We're giving ourselves two different query interfaces into the same data.

Generating the embedding

# surveillance/embeddings.py

from sentence_transformers import SentenceTransformer

# All-MiniLM-L6-v2 is small (80MB), fast, and produces 384-dim vectors.
# It runs on CPU without issue, which matters in resource-constrained environments.
# See here: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2

_model = None

def get_model():
    global _model
    if _model is None:
        _model = SentenceTransformer("all-MiniLM-L6-v2")
    return _model

def embed_report(report: "EpiReport") -> list[float]:
    """
    Convert a report's semantic content into a vector.
    The text we embed is a natural language description of the report,
    so the model can capture the meaning, not just the keywords.
    """
    text = (
        f"{report.title}. "
        f"Disease: {report.disease}. "
        f"Region: {report.region}. "
        f"Severity: {report.severity}. "
        f"{report.summary}"
    )
    model = get_model()
    return model.encode(text).tolist()

The two query styles, side by side

# surveillance/queries.py

from pgvector.django import CosineDistance
from .models import EpiReport
from .embeddings import get_model

# ── Approach 1: Structured Query ────────────────────────────────────────────

def get_high_severity_reports(region: str, disease: str, year: int):
    """
    Classical Django ORM query.
    Fast, precise, requires you to know exactly what you want.
    """
    return EpiReport.objects.filter(
        region=region,
        disease=disease,
        year=year,
        severity="high",
    ).order_by("-week")

# ── Approach 2: Semantic Vector Query ───────────────────────────────────────

def find_similar_reports(query: str, top_k: int = 5):
    """
    Vector similarity search.
    The query is natural language. No schema knowledge required.
    Returns the most semantically similar reports.
    """
    model = get_model()
    query_vector = model.encode(query).tolist()

    return (
        EpiReport.objects
        .annotate(distance=CosineDistance("embedding", query_vector))
        .order_by("distance")[:top_k]
    )

def find_similar_to_report(report: EpiReport, top_k: int = 5):
    """
    Given an existing report, find others that are semantically similar.
    Useful for "related reports" features or duplicate detection.
    """
    return (
        EpiReport.objects
        .exclude(pk=report.pk)
        .annotate(distance=CosineDistance("embedding", report.embedding))
        .order_by("distance")[:top_k]
    )

Wiring it into views

# surveillance/views.py
from django.http import JsonResponse
from .models import EpiReport
from .queries import get_high_severity_reports, find_similar_reports

def structured_search(request):
    """Traditional form-based search."""
    reports = get_high_severity_reports(
        region=request.GET.get("region", ""),
        disease=request.GET.get("disease", ""),
        year=int(request.GET.get("year", 2024)),
    )
    return JsonResponse({
        "approach": "structured",
        "results": list(reports.values("title", "region", "disease", "severity"))
    })

def semantic_search(request):
    """Free-text semantic search."""
    query = request.GET.get("q", "")
    reports = find_similar_reports(query, top_k=5)
    return JsonResponse({
        "approach": "vector",
        "query": query,
        "results": [
            {
                "title": r.title,
                "region": r.region,
                "disease": r.disease,
                "similarity": round(1 - r.distance, 3), # convert distance to similarity
            }
            for r in reports
        ]
    })

Now try these two requests:

GET /search/structured/?region=Dakar&disease=cholera&year=2024
GET /search/semantic/?q=waterborne disease outbreaks in coastal areas

The structured search finds exactly what you specified. The semantic search finds what you meant, even if those exact words don't appear anywhere in your data. The world is beautiful.

When to use which

On top of my tiny experience, and as a software engineer who has never used this technique on on production, I would say the right answer for most systems is both. And pgvector makes that even more practical without adding infrastructure complexity.

Lean on structured queries when you need exactness: filtering by date range, counting cases, producing regulatory reports, joining across tables. SQL is unbeatable here.

Lean on vector queries when you need semantic understanding: search boxes, recommendation engines, duplicate detection, anomaly flagging, clustering, or any time a user's intent can't be reduced to a dropdown value.

Even more interesting, hybrid search : use structured filters to narrow the candidate set, then rank the survivors by vector similarity:

def hybrid_search(region: str, query: str, top_k: int = 5):
    """
    Filter structurally, then rank semantically.
    This is typically the most useful approach in production.
    """
    model = get_model()
    query_vector = model.encode(query).tolist()

    return (
        EpiReport.objects
        .filter(region=region) # hard filter: only this region
        .annotate(distance=CosineDistance("embedding", query_vector))
        .order_by("distance")[:top_k]
    )

Going further

Once you've internalized the two-lens model, here's where to take it next:

1. Fine-tune your own embedding model on domain data

The all-MiniLM-L6-v2 model was trained on general English text. It has no idea that "paludisme" and "malaria" are the same thing, or that "cas suspects" and "suspected cases" are equivalent in my context. Training a domain-specific embedding model on my own epidemiological corpus — even with a small dataset using contrastive learning — would make similarity search dramatically more meaningful.

2. Retrieval-augmented generation (RAG) on top of our vector store

Once we have a vector store of reports, we're one step away from a system that can answer questions about your data in natural language. A user asks: "What were the major respiratory disease trends in Guinea-Bissau in 2023?" — our system retrieves the most relevant reports by vector similarity, then feeds them as context to an LLM that synthesizes a coherent answer.

3. Vectors for anomaly detection in surveillance data

Every week a new batch of case reports comes in. We can embed each report and compare it to the historical distribution of vectors for that region and disease. A report that lands far from the cluster is a statistical anomaly, potentially a real outbreak signal. This turns our vector store into a passive early-warning system without writing any explicit outbreak-detection rules.

4. Multimodal embeddings

Text isn't the only thing we can embed. We can embed laboratory results, genomic sequences, geographic coordinates combined with case counts, even scanned paper forms via vision models. The moment we have multiple modalities in the same vector space, we can do cross-modal search: find reports whose geographic spread pattern resembles this map image.

placeholder image for brain explosion

5. The theory underneath: why vector spaces work

Representation models: other ways to represent data

Structured attributes and vectors are two lenses. And there are other ways to represent data:

Representation	Best question it answers	Tools
Class + attributes	What does this thing have?	Django ORM, PostgreSQL
Vector	What is this thing like?	pgvector, sentence-transformers
Graph	How does this thing connect?	Neo4j, networkx
Event stream	How did this thing become what it is?	Kafka, EventStoreDB
Logic / facts	What can we conclude about this thing?	Prolog, Datalog, RDF/OWL
Probability	How certain are we about this thing?	PyMC, Stan
Spatial	Where is this thing, relative to what?	PostGIS, GeoDjango
Tensor	How does this vary across multiple dimensions?	NumPy, PyTorch, Xarray

How to practice

You don't need production data to start experimenting with vectors and embeddings:

Local development:

Chroma: Lightweight vector database that runs locally or in Docker.
pgvector in Docker: A local PostgreSQL with pgvector extension in seconds.
Sentence Transformers Models: Experiment with domain-specific models beyond all-MiniLM-L6-v2.

Learning resources:

Hugging Face Hub: 100k+ pre-trained models. Read papers, compare benchmark results.
Embedding Models Course: Free, practical introduction to embeddings.
pgvector Documentation: All indexing strategies and similarity operators explained.

While I still have your attention

The Norwegian Consumer Council is an independent, governmentally funded organisation that advocates for consumer’s rights. It should be easy for consumers to make sustainable choices every day. Consumers have the right to be protected against exploitation – both financially and digitally. To ensure this, we work to provide easy access to information, enforceable rights, and sufficient redress options when something goes wrong.

Digital products and services keep getting worse. In the new report Breaking Free: Pathways to a fair technological future, the Norwegian Consumer Council has delved into enshittification and how to resist it. The report shows how this phenomenon affects both consumers and society at large, but that it is possible to turn the tide.

Geez, I would have loved to see African leaders do the same....

Syndromic Surveillance! (Part 1)

Nsukami — Sun, 24 Aug 2025 00:00:00 +0000

Disclaimers :

Opinions expressed in this post (and in any of all my posts) are solely, unless otherwise specified, those of the authors, me. Those opinions absolutely do not reflect the views, policies, positions of any organizations, employers, affiliated groups.
I've strived for accuracy throughout this piece, but if you catch any errors, please reach out—I'd be grateful for the feedback and happy to make updates!

Hook:

From June 30 to July 2, 2025 , a coordination meeting for the entire 4S network was recently held in Lomé, Togo. During this meeting, a new implementation of the 4S platform was presented to the participants.

Your browser does not support the video tag.

Quote:

Le réseau 4S a été conçu pour répondre à un besoin crucial, celui de disposer d'un système régional de surveillance capable de détecter précocément les maladies de santé publique, de renforcer les capacités de laboratoires et de partager des données pour une réponse adéquate et adaptée aux problèmes sanitaires.

Quote :

The 4S network was designed to meet a crucial need: to have a regional surveillance system capable of early detection of public health diseases, strengthening laboratory capacities and sharing data for an adequate and appropriate response to health issues.

The striking disconnect that exists across many sectors, public health included. Officials excel at announcing initiatives and securing funding. Press releases about "implementing advanced early detection systems" generate buzz and demonstrate progress. But when it comes to the unglamorous technical work: database design, API development, system integration... Silence. These crucial implementation details, handled by anonymous engineers and developers, rarely see daylight in public discourse.

While policy makers avoid technical specifics, it's precisely these software choices that determine whether systems function during critical moments.

Definition, context, history:

A syndromic surveillance system is a public health surveillance system that monitors symptom clusters rather than specific confirmed diagnoses. It aims to detect epidemics or public health events at an early stage, by analyzing data in real time.

2007:

Before 2007, Madagascar only had a passive, monthly paper-based reporting system that was _ too slow _ for outbreak response. The chikungunya outbreak in the Indian Ocean exposed _ critical gaps _ in Madagascar's disease surveillance.

The solution was to setup a real-time sentinel surveillance system using mobile phones for daily data transmission. This implementation was _ the first nationwide real-time-like _ surveillance system ever established in Madagascar.

2012:

The Syndromic Sentinel Surveillance System in Senegal, the 4S network, is Another syndromic sentinel surveillance system based on febrile syndromes. It was implemented in Senegal in 2012 and inpired by the Madagascar model.

The criteria of individual target diseases are based on World Health Organization recommended surveillance standards. This means: for each disease or syndrome there is a description of the rationale for surveillance, case definition, types of surveillance, minimum data elements, data analyses and principal uses of data for decision-making.

Data are transmitted on a daily basis by the general practitioners of the sentinel sites. They are validated and analyzed daily within the epidemiology unit of the Institution. These data are sent to the Ministry of Health in the form of a weekly newsletter.

2015:

The existing implementation faced significant challenges. The database consisted of scattered Excel files distributed across multiple locations, making studies and analyses complicated and time-consuming. There was an urgent need to organize, structure, and centralize these disparate files.

To address these issues, a new platform was developed: Teranga. Built using Apache, PHP, Javascript and MySQL, Teranga provides improved data management, enhanced tracking capabilities, modular architecture, and streamlined deployment during outbreak situations.

The new sentinel surveillance system included 17 health centers and identified 4 confirmed outbreaks. The system has proved the feasibility of improving disease surveillance capacity through innovative technological solutions.

2020:

When COVID-19 struck, Teranga rose to meet an unprecedented challenge, becoming the backbone of Senegal's pandemic response. Seamlessly coordinating patient care, traveler monitoring, financial operations, clinical research, and treatment center management across the entire country. The pandemic trully elevated Teranga to a mission-critical status.

2022:

What began as a surveillance system for on institution, evolved to become West Africa indispensable healthcare hub: The 4S network. Teranga is now expanding from Senegal into other West African countries: Gambia, Cape-Verde, Mauritania, Niger, Mali, Togo, Guinea, Guinea Bissau, Sierra-Leone and Benin. The goal being to provide broader coverage and improve health systems in a sustainable and effective manner.

2023:

I am hired by the Institution. I'm tasked with reimplementing the current system, rebuilding it from scratch while improving on what came before. I don't have much say in the tech stack, just enough freedom to go with what works for me. The usual lineup of Debian, Python, Django, Postgresql, Nginx, Podman, Git, Gitlab, Emacs, Tmux, etc ... More on that in the following episodes.

2025:

The surveillance platform my colleagues and I have spent two years developing has now been presented to the public. But public recognition is not really the source of my pride. What matters is that our system, our algorithms, our data structures, our functions, our for loops, our SQL queries are actively making a difference in saving lives. Phew!

4S overview:

identify unusual health events earlier and provide a quicker response.

I can't say it better.

Core principles:

It's really fascinating that these early systems already recognized the core principles we are trying to implement today:

1. Standardized data collection:

Healthcare providers at sentinel sites should use standardized digital forms to capture essential demographic, clinical, and temporal information. We want to ensure consistency across all participating facilities and enable accurate epidemiological analysis. We should always try to adhere to WHO surveillance standards with clearly predefined entities & attributes.

2. Data processing and storage:

The platform should handles real-time data streaming between microservices, enabling immediate processing of incoming surveillance data. Each microservice should ensures data integrity through automated validation rules. We should always check for completeness, consistency, and quality. Validated data should be stored in a database with proper indexing for efficient retrieval.

3. Information sharing:

The epidemiology unit generates documents consolidating validated surveillance data for the Ministry of Health. The platform integrates with third parties to share aggregated data with the Ministry of Health. The platform also provides interactive dashboards for stakeholders.

4. Alerting and notifications:

The platform should ensure immediate notification delivery to relevant actor via email and dashboard notifications. Whenever needed, the platform should automatically generates notifications through multiple channels—email alerts, in-app notifications, and dashboard warnings.

5. Integration with existing systems:

The platform should seamlessly integrate with the regional health data aggregation system and any other existing platform used by Ministry of Health. The system must also integrate easily with any biobank systems that may exist.

A drawing is worth more than thousand words:

                                         [Health facility]   [Hospital]         [Clinic]
                                               |                 |                  |
                                               |                 |                  |
                                          +----+-----------------+------------------+----+
                                          |              4S platform                     |
                                          | • Data collection (Patients, symptoms, ...)  |
                                          | • Data validation & storage                  |
                                          | • Data treatment (Analyses, studies, ...)    |
                                          | • Data sharing (Reports, alerts, ...)        |
                                          | • Data dashboards (Statistics, plots, ...)   |
                                          +----+-------------------+-------------+-------+
                                               |                   |             |
                                          +----+----+              |        +----+----+
                                          | Reports |              |        | Alerts  |
                                          |Analytics|              |        |Warnings |
                                          +----+----+              |        +----+----+
                                               |                   |             |
                                               |                   |             |
                                         +-----+---------+   +-----+----+    +---+-------+
                                         | Public Health |   | Research |    | Emergency |
                                         |   Department  |   |          |    | Response  |
                                         +---------------+   +----------+    +-----------+

This is the platform we are trying to build :) I can stop the post here, thanks for reading ...

The technological stack:

Frontend:

The 4S platform frontend leverages AngularJS with TypeScript, providing a dynamic and type-safe user interface that ensures smooth interactions for healthcare providers.

One of the main component of our architecture is Keycloak, an identity & access management solution that handles user authentication, authorization accross the entire platform. Customizing the Django Rest Framework's authentication classes to handle different tokens sent by different realms was an adventure worthy of an article.

Backend:

The backend infrastructure centers around Python, Django, and Django Rest Framework. With each passing day, as we make progress on the project, I become more and more convinced that it was the right choice. Django is an amazing piece of software, thanks a lot to all its contributors.

Data management is handled by PostgreSQL, our primary database system. I can proudly say that I'm the one who made this choice. Postgresql is known for its reliability, strong adherence to SQL standards, powerful extensibility, support for complex data types, ACID compliance, robust security features, and excellent community support.

To handle real-time communication and event-driven processes such as email notifications, lab result updates, and system alerts, we've integrated Apache Kafka. Kafka ensures that critical healthcare information flows efficiently between different system components without data loss.

In order to detect outbreaks, samples (blood, urine, stool, etc.) must be collected. And to effectively manage vast biological sample collections, ensure data integrity, and streamline complex workflows, we need a Biobank and a Biobanking software. A biobank is essentially an organized library of biological samples (like blood and tissue) that are stored alongside health and personal information, all used to support medical research. Mbiolims is the biobanking software we use to manage our samples and their associated data.

Here too, I would need to write a really long article telling you how we've managed to do the integration between Django and MBioLIMS :) Hehe!

Operations:

All code versioning and collaboration is managed through Git & Bitbucket, ensuring secure source code management, proper access controls and audit trails. While Bitbucket provides comprehensive version control and collaboration features, I'm still convinced it was not the right choice. Gitlab ? Gitea ? Github ? Come on!

The 4S platform is in fact a constallation of multiple running containers. Those containers are built using Docker and Podman technologies, enabling consistent deployment across different environments while simplifying development and testing workflows. I mean, I'm the only one using Podman while all my teammates are still using Docker :\

Our continuous integration and deployment pipeline is powered by Jenkins. Jenkins automates testing and deployment processes to maintain code quality and enable rapid, reliable updates. I prefer GitlabCI by 13 parsecs.

Kubernetes is used for container orchestration, deployment and operations. K8s provides automatic scaling, load balancing, and high availability to ensure our platform can handle varying loads and maintain uptime. My opinions on this tool are mixed.

In one side, K8s will allow us to easily scale our applications up or down based on demand. This is a critical requirement for managing sensitive healthcare information. In the other side, I live in a part of the world where most medical facilities (most people, let's be honest) do not even have a working domain name, website, and email address. How can we ask them for the DevOps expertise to manage a Kubernetes cluster? I will see to what extent I can write an article about Kubernetes.

Even more:

There are other essential components of the system which I am not really familiar with.

DHIS2 is widely adopted across West Africa for integrated health information systems, disease surveillance, education management, and public health emergency response. We simply cannot ask a country that already uses DHIS2 to use the 4S platform without it being able to transfer data back to a DHIS2 instance.

We have an Apache Superset instance for data visualization and exploration. With Apache Superset, we can easily let a third party consult some information, without modifying our system.

Our platform generate lots of files, analysis reports & statistics. We needed something to store vast amounts of unstructured data, like images, videos, pdf documents, etc... We choosed MinIO for this use case.

The diagram we've seen above can also be seen using this different perspective:

    +-----------------------------------------------------------------------------------------------------------------------+
     |                                                      KUBERNETES CLUSTER                                               |
     +-----------------------------------------------------------------------------------------------------------------------+
     |                                                                                                                       |
     |  +--------------------------------------------------+   <-- (Auth/Roles) -->  +------------------+                    |
     |  |             FRONTEND CLIENTS                     |------------------------>|     KEYCLOAK     |                    |
     |  |          (AngularJS + TypeScript)                |<------------------------|   (Auth Server   |                    |
     |  +------------------^-------------------------------+                         +-----^------------+                    |
     |                     |                                                               |                                 |
     |              (API Requests)                                                         | (Auth/Roles)                    |
     |                     |                                                          +----+                                 |
     |                     |                                                          |                                      |
     +---------------------v----------------------------------------------------------v--------------------------------------+
     |  +--------------------------------------------------------------------------------+      +--------------------------+ |
     |  |                BACKEND MICROSERVICES (Django Rest Framework)                   |      |    Biobank (Mbiolims)    | |
     |  |      Microservices exchanges data via a message broker (Kafka in our case)     |<---->|   Stores Samples infos   | |
     |  |   [Microservice A]  <--Data-->  [Microservice B] <--Data--> [Microservice N-1] |      |                          | |
     |  +-------------------------^-----------------------^-------------------------^----+      +--------------------------+ |
     |                            |                       |                         |                                        |
     |              (Store/Retrieve)       (Write/Open Files)      (Sends data aggregations)                                 |
     |                            |                       |                         |                                        |
     +----------------------------v-----------------------v-------------------------v----------------------------------------+
     | +--------------------------------+ +--------------------------------+ +--------------------------------------------+  |
     | |        POSTGRESQL DB           | |             MINIO              | |                   DHIS2                    |  |
     | |     (Primary Data Store)       | |      (Object/File Storage)     | |           (External Data Recipient)        |  |
     | +--------------------------------+ +--------------------------------+ +--------------------------------------------+  |
     |               | (Reads Data)                                                                                          |
     |               |                                                                                                       |
     | +-------------v--------------------+                                                                                  |
     | |    APACHE SUPERSET (Dataviz)     |                                                                                  |
     | +----------------------------------+                                                                                  |
     |                                                                                                                       |
     +-----------------------------------------------------------------------------------------------------------------------+

The engineering team:

This article is also for my fellow developers who built the 4S network under impossible constraints—limited budgets, legacy infrastructure, unrealistic timelines. My colleagues engineers writing scripts to detect disease patterns, configuring servers that never get enough resources. My teammates who's written documentation that nobody reads but everyone depends on. I'm talking about the people who made the story:

Cisse:

Cisse a.k.a "Celui qui fait bouger le monde", plays a crucial role in coordinating activities across the Institution Digital Factory team. His expertise extends to managing and preserving the integrity of biomedical data within the Institution. He works closely with stakeholders to validate specifications of new health applications. To uncover more about him, follow this link.

Mame Astou:

As an integral member of the development team, Mame Astou a.k.a "Madame La Directrice", craft robust and dynamic web applications while maintaining a keen eye for intuitive user interface design. She knows how to create user experiences that prioritize both aesthetic appeal and seamless usability. What would we be without her? Absolutely nothing. To uncover more about her, please visit this page.

Diom:

Diom, a.k.a "Le Grand Marabout de l'Architecture", is a DevOps engineer who bridges the gap between development and operations. With expertise in containerization technologies, cloud platforms, and monitoring solutions, he design and maintain scalable deployment environments while ensuring system reliability and security.

Abdoul Karim:

Abdoul Karim is probably the pillar of the team, always ready to make concessions and take responsibility. His technical experience spans multiple domains. It would take me hours to explain why he is an excellent engineer. Just go and find out by yourself how great he is.

Mariama:

Mariama a.k.a "La Sauveuse De Tous Les Hommes", has a deep knowledge of Scrum and agile principles. She guide the team through iterative development cycles while fostering continuous improvement and team collaboration. We are still waiting for that one day she will finally write a few lines of TypeScript. Access Mariama's comprehensive profile by following this link.

Birante:

Birante has an exceptional expertise that encompasses everything from custom UI animations and state management to integrating native device features and third-party services. He is able to deliver native-quality experiences across both iOS, Android and Web platforms. He secretly dreams about rewriting all the 4s platform using RoR applications. Here is a link to get the complete picture about Birante.

Tidiane:

Tidiane, a.k.a "Le Sage", a.k.a "The Biobank expert", is probably the wisest and most reserved member of the team. Tidiane is a seasoned backend developer who brings deep expertise in Django framework development. He excel at designing robust APIs and implementing complex business logic that powers modern web applications. If you know anyone who works at Mbiolims, please tell them they should hire Tidiane. His background can be found here.

Adrien:

Aux âmes bien nées, la valeur n'attend point le nombre d'années. Adrien, the youngest in the team, is a software engineer with an extensive experience in developing RESTful APIs, managing database migrations, and implementing automated testing frameworks within Django projects. For additional information about Adrien, visit this link.

Guys, your dedication, your resilience, your ingenuity deserve recognition far beyond what they receive. Thank you.

The roadmap:

Right now we're developing comprehensive features related to "case management". Those features will enable healthcare providers to track patient journeys from initial consultation through treatment completion, ensuring continuity of care and improved outcomes.

We plan to extend our platform to handle a wider spectrum of medical conditions and syndromes , transforming it from a specialized tool into a comprehensive healthcare management solution.

Our platform will also align with the One Health approach , recognizing the interconnection between human, animal, and environmental health.

Data exchange with other healthcare systems, electronic health records, and medical devices, is something we need to work towards. We really need to meet evolving healthcare standards and prepare for international expansion. While I agree we didn't put enough efforts on this, compliance and interoperability remain central to our development strategy.

DEV Community: Nsukami

Vectors

Disclaimers :

Hook

ToC

The classical approach: data as structure

The vector approach: data as position

Why this is not just for AI projects

Both approaches in Django: a concrete example

Setup

The traditional model

Generating the embedding

The two query styles, side by side

Wiring it into views

When to use which

Going further

1. Fine-tune your own embedding model on domain data

2. Retrieval-augmented generation (RAG) on top of our vector store

3. Vectors for anomaly detection in surveillance data

4. Multimodal embeddings

5. The theory underneath: why vector spaces work

Representation models: other ways to represent data

How to practice

More on this topic

While I still have your attention

Syndromic Surveillance! (Part 1)

Disclaimers :

TOC:

Hook:

Definition, context, history:

2007:

2012:

2015:

2020:

2022:

2023:

2025:

4S overview:

Core principles:

1. Standardized data collection:

2. Data processing and storage:

3. Information sharing:

4. Alerting and notifications:

5. Integration with existing systems:

The technological stack:

Frontend:

Backend:

Operations:

Even more:

The engineering team:

Cisse:

Mame Astou:

Diom:

Abdoul Karim:

Mariama:

Birante:

Tidiane:

Adrien:

The roadmap:

More on the topic: