DEV Community: Samuel Komfi

Amanzi - Know Your Water

Samuel Komfi — Sun, 01 Mar 2026 22:18:12 +0000

This is a submission for the DEV Weekend Challenge: Community

The Community

South Africa is in a water crisis that has taken many years to manifest into daily life.

I live in the Johannesburg area, where water is not something you take for granted. Gauteng imports nearly 98% of its water from outside the province, the Vaal Dam fluctuates dangerously, and municipal restrictions can change overnight. In townships like Alexandra and Diepsloot, flooding and water outages are a lived reality, not a news headline. Nationwide, 47% of South Africans experience regular water disruptions, and the country loses an estimated 35% of its treated water to leaking infrastructure before it ever reaches a tap.

Yet for the average resident, getting reliable, consolidated water information means digging through the Department of Water & Sanitation's PDF reports, checking three different municipal websites, and hoping the news covers it. There is no single, accessible, community-facing app that brings it all together.

The community I built Amanzi AquaSA for is every South African who has ever woken up to no water and/or tried to figure out whether they can legally fill their borehole in a Level 2 restriction area. That's a community of tens of millions of people and they deserve better tools.

What I Built

Amanzi AquaSA is a Flutter mobile app (Android & iOS) that turns South Africa's fragmented water data into one clear, beautiful, and genuinely useful experience. The tagline is simple: Know Your Water.

The app is organized into five core areas:

Dashboard - A live at-a-glance summary of national water health. Dam levels, active restrictions, rainfall this month vs. historical average, and a critical alerts banner if something urgent is happening in your area. It answers the question every Gauteng resident asks on a dry summer morning: "How bad is it today?"

Dams & Rivers - Detailed profiles for South Africa's 10 major dams (Vaal, Sterkfontein, Gariep, Theewaterskloof, and more) with historical level charts you can switch between 1-year and 10-year views. River health cards cover the Vaal, Olifants, Limpopo and other major waterways - including honest pollution and health status ratings. Catchment area maps show where the water actually comes from.

My Water Usage - A personal water calculator that lets you tally your real daily usage: glasses of water, coffee, shower minutes, toilet flushes, laundry loads, geyser, garden watering and more. It compares your total against the SA average (237L/day) and the humanitarian minimum (50L/day), estimates your monthly municipal bill, and gives you targeted tips based on where you're using the most.

Alerts - A filterable feed of water alerts: flood warnings, restriction level changes, dam level thresholds, quality advisories, and maintenance outages. Critical alerts surface across the whole app with a pulsing red banner so nothing urgent gets missed.

Explore - The bigger picture. National water infrastructure entities (Rand Water, Umgeni, Lepelle), active government water projects with budgets and timelines, dam levels by province, and an educational self-sufficiency hub covering borehole drilling (process, costs, legal requirements), DIY water treatment methods (UV, RO, ceramic filters, SODIS), and desalination techniques from emergency solar stills to municipal-scale reverse osmosis.

An interactive map ties it all together; toggling between dam markers, river health overlays, catchment area polygons, and flood-risk zones across the country.

Demo

Video demo link

APK Download

Code

Github Code

How I Built It

Amanzi AquaSA is built entirely in Flutter, targeting Android and iOS from a single codebase. Here's the technical breakdown:

Framework & Language Flutter 3.x with Dart, using a feature-first folder structure. Each feature (dams, rivers, rainfall, etc.) is fully self-contained with its own models, repository, and screen files - making the codebase easy to extend as live APIs become available.

Navigation go_router for declarative, URL-like routing with deep linking support - important for eventually sharing direct links to specific dam or river detail pages.

Maps flutter_map with OpenStreetMap tiles (no API key, no cost, works offline). Custom CustomPainter layers for catchment area polygons and color-coded river polylines. Dam markers are dynamically colored by level percentage - teal for healthy, amber for low, red for critical.

Charts fl_chart powers the historical dam level line charts, monthly rainfall bar charts, and the usage history graph. The dam detail screen supports switching between 1-year and 10-year data views with smooth animated transitions between datasets.

Usage Calculator Built with stateful widget steppers and a real-time litre accumulator. Toilet flush mode toggles between old (9L) and new (4.5L) cistern volumes. Shower time uses a Slider rather than a stepper since duration is continuous. Results are persisted locally with shared_preferences to show a 7-day usage history.

Self-Sufficiency Hub The rainwater harvesting calculator uses a simple formula: roof area (m²) × rainfall (mm) × 0.8 efficiency = litres collected. Users input their roof size and select their region's monthly rainfall to get a realistic monthly collection estimate - surprisingly powerful for planning tank sizing.

Water security isn't a future problem in South Africa - it's today's problem, and it's getting harder to ignore. Amanzi is a small piece of infrastructure for a community that needs better tools to understand, track, and protect the water they depend on.

Know your water.

Unlocking Data Insights: A Semantic Search App with MindsDB and Django

Samuel Komfi — Tue, 01 Jul 2025 10:04:50 +0000

Imagine querying your vast, disparate datasets using plain English, getting not just exact matches, but semantically relevant information. That is the power of MindsDB Knowledge Bases, and I'm excited to share how I integrated them into a Django application.

Introduction

Data is everywhere, but truly understanding it often requires complex SQL queries or specialized tools. What if you could ask your data questions naturally, like "Show me clients in high-income urban areas" or "Summarize suspicious transactions"? This is exactly what MindsDB's new Knowledge Bases (KBs) aim to enable.

This article details how a django transaction application can turn a dashboard of information. I decided to build a simple, yet powerful, Django web application that allows users to perform semantic searches on mock financial data and even get AI-powered summaries. The application only has three tables to keep it simple. It is based on the dataset from the following: Kaggle Dataset We will start by building a normal Django application with postgres as the backend database. We will later then show how to make your data available to MindsDB. Django knowledge is neccessary to follow along as I will not explain the framework.

Bridging Natural Language and Structured Data

Traditional databases excel at structured queries (e.g., SELECT * FROM clients WHERE age > 30). However, they fall short when you want to query based on meaning or context, like finding "high-risk clients" or "transactions indicative of fraud." This gap often requires complex machine learning pipelines.

MindsDB Knowledge Bases address this by:

Semantic Search: Understanding the meaning behind your queries, not just keywords.
Unified Access: Querying data across different sources (like my PostgreSQL DB).
Metadata Filtering: Combining semantic search with traditional SQL filters.

Project Overview: A Django App for Financial Data Insights

My application is a Django-based web interface designed to interact with a PostgreSQL database containing mock financial data:

Client: Information about individual clients (age, gender, address, income).
Card: Details about credit/debit cards associated with clients.
Transaction: Records of financial transactions (amount, merchant, date, location).

The core of the application's intelligence comes from MindsDB, which transforms this structured data into semantically searchable Knowledge Bases. The Django app then acts as a user-friendly frontend to query these KBs.

Django Application Setup

The Django project, named transaction_dashboard_proj or any other name, uses PostgreSQL as its database and python-dotenv for environment variable management. The name of the app will be called finance and swiftly added to a list of apps in the settings.py file.

Herer is the requirements file for the project:

Django>=5.2.3
psycopg2-binary>=2.9.9
python-dotenv>=1.0.0 
pandas>=2.0.0
mindsdb_sdk>=1.0.0

The finance/models.py defines the database schema:

# core/models.py  
from django.db import models  

class Client(models.Model):
    """Client model - separate from Django User model since clients cannot login"""
    current_age = models.IntegerField()
    retirement_age = models.IntegerField()
    birth_year = models.IntegerField()
    birth_month = models.IntegerField()
    gender = models.CharField(max_length=10)
    address = models.TextField()
    latitude = models.DecimalField(max_digits=9, decimal_places=6)
    longitude = models.DecimalField(max_digits=9, decimal_places=6)
    per_capita_income = models.DecimalField(max_digits=12, decimal_places=2)
    yearly_income = models.DecimalField(max_digits=12, decimal_places=2)
    total_debt = models.DecimalField(max_digits=12, decimal_places=2)
    credit_score = models.IntegerField(default=0)
    num_credit_cards = models.IntegerField(default=0)

    class Meta:
        db_table = 'client'
        ordering = ['id']

    def __str__(self):
        return f"Client {self.id}"


class Card(models.Model):
    """Card model linked to clients"""
    CARD_BRAND_CHOICES = [
        ('visa', 'Visa'),
        ('mastercard', 'Mastercard'),
        ('amex', 'American Express'),
        ('discover', 'Discover'),
    ]

    CARD_TYPE_CHOICES = [
        ('credit', 'Credit'),
        ('debit', 'Debit'),
        ('prepaid', 'Prepaid'),
    ]

    client = models.ForeignKey(Client, on_delete=models.CASCADE, related_name='cards')
    card_brand = models.CharField(max_length=20, choices=CARD_BRAND_CHOICES)
    card_type = models.CharField(max_length=25, choices=CARD_TYPE_CHOICES)
    card_number = models.CharField(max_length=20)  # Masked card number
    expires = models.CharField(max_length=12)
    cvv = models.CharField(max_length=4)  # Should be encrypted in production
    has_chip = models.BooleanField(default=True)
    num_cards_issued = models.IntegerField(default=1)
    credit_limit = models.DecimalField(max_digits=12, decimal_places=2, null=True, blank=True)
    acct_open_date = models.CharField(max_length=8, default="01/1970")
    year_pin_last_changed = models.IntegerField(default=2025)
    card_on_dark_web = models.CharField(default="No")


    class Meta:
        db_table = 'cards'
        ordering = ['id']

    def __str__(self):
        return f"Card {self.id} - {self.card_brand} {self.card_type}"


class Transaction(models.Model):
    """Transaction model linked to clients and cards"""

    CHIP_CHOICES = [
        ('chip', 'Chip Transaction'),
        ('swipe', 'Swipe Transaction'),
    ]

    date = models.DateTimeField()
    client = models.ForeignKey(Client, on_delete=models.CASCADE, related_name='transactions')
    card = models.ForeignKey(Card, on_delete=models.CASCADE, related_name='transactions')
    amount = models.DecimalField(max_digits=10, decimal_places=2)
    use_chip = models.CharField(default="Chip Transaction", choices=CHIP_CHOICES)
    merchant_id = models.CharField(max_length=50)
    merchant_city = models.CharField(max_length=100)
    merchant_state = models.CharField(max_length=100, null=True, blank=True)
    zip = models.CharField(max_length=10, null=True, blank=True)
    mcc = models.IntegerField(default=1000)
    errors = models.CharField(max_length=200, null=True, blank=True)

    class Meta:
        db_table = 'transactions'
        ordering = ['-date', 'id']

    def __str__(self):
        return f"Transaction {self.id} - ${self.amount} on {self.date}"

Note: Clients are not Django users and do not have login capabilities in this application, focusing purely on data querying. Some of the fields may not be exactly PCI compliant or best practice, but we working with the dataset provided for testing.

Run Django migrations to create the tables in the database:

python manage.py makemigrations
python manage.py migrate

The Kaggle Dataset

Download the dataset from the Financial Transactions Dataset: Analytics from Kaggle, specifically targeting only 3 files needed for the table models above. They are the Transaction data (transactions_data.csv), Card information( cards_dat.csv) and the Users data(users_data) which is renamed client for our Django app to avoid confusion with the django auth_user. Please note the transactions_data file is massive and will need to be split into 3 or more files for easier handling. You can find a script online to split it up.

After downloading the files we will copy them into the database by using the copy command in psql. Start with client data(users_data.csv), followed by card data, and finally transaction data(as it has card & client foreign keys).

Log into psql:

\connect fraud_mindsdb;

\copy client(id,current_age,retirement_age,birth_year,birth_month,gender,address,latitude,longitude,per_capita_income,yearly_income,total_debt,credit_score,num_credit_cards) FROM '/tmp/users_data.csv' DELIMITER ',' CSV header;

\copy cards(id, client_id, card_brand, card_type, card_number, expires, cvv, has_chip, num_cards_issued, credit_limit, acct_open_date, year_pin_last_changed, card_on_dark_web) FROM '/tmp/cards_data.csv' DELIMITER ',' CSV header;

\copy transactions(id, date,client_id, card_id, amount,use_chip,merchant_id, merchant_city, merchant_state, zip, mcc, errors) '/tmp/transaction_data.csv' DELIMITER ',' CSV header;

Connecting MindsDB to PostgreSQL

We are going to run MindsDB locally instead of using the cloud version. You will need docker installed. To run MindsDB Web Studio locally execute the following:

docker run -p 47334:47334 -p 47335:47335 mindsdb/mindsdb

If you get a 'Connection refused' error run the following to allow MindsDB to access your local Postgres instance:

docker run --network=host mindsdb/mindsdb

The first step in MindsDB is to connect it to our existing PostgreSQL database. This allows MindsDB to access the Client, Card, and Transaction tables. I ran this command directly in the MindsDB Web Studio:

CREATE DATABASE finance_db
WITH ENGINE = 'postgres',
PARAMETERS = {
  "host": "your_db_host",
  "port": 5432,
  "user": "your_user",
  "password": "your_password",
  "database": "your_postgresdb"
};

This creates a virtual database finance _db inside MindsDB, mirroring our PostgreSQL schema.

Creating MindsDB Knowledge Bases

Next, we define two Knowledge Bases: one for Client data (client_kb) and one for Transaction data (transaction_kb). The key here is defining the CONTENT_COLUMN (what MindsDB will semantically understand) and METADATA_COLUMNS (additional data to retrieve and filter on).


-- For Clients:  
CREATE KNOWLEDGE_BASE client_kb
USING 
    embedding_model = {
        "provider": "gemini",
        "model_name": 'gemini-embedding-exp-03-07',
        "api_key": "xxxxxx"
    },
    reranking_model = {
        "provider": "gemini",
        "model_name": "gemini-pro",
        "api_key": "xxxxxx"
    },
    content_column = 'address',
    metadata_columns = ['per_capita_income', 'current_age', 'gender', 'birth_year']; 

-- For Transactions:  
CREATE KNOWLEDGE_BASE transaction_kb
USING 
    embedding_model = {
        "provider": "gemini",
        "model_name": 'gemini-embedding-exp-03-07',
        "api_key": "xxxxxx"
    },
    reranking_model = {
        "provider": "gemini",
        "model_name": "gemini-pro",
        "api_key": "xxxxxx"
    },
    content_column = ['merchant_city', 'merchant_state'],
    metadata_columns = ['amount', 'use_chip', 'date', 'client_id'];

After defining the KBs, we ingest the existing data in the postgres database of the Django app:

INSERT INTO client_kb (content, metadata)
SELECT 
    address as content,
    JSON_OBJECT(
        'id', id,
        'per_capita_income', per_capita_income,
        'current_age', current_age,
        'gender', gender,
        'birth_year', birth_year
    ) as metadata
FROM finance_db.core_client;

-- Insert transaction data with merchant location as content
INSERT INTO transaction_kb (content, metadata)
SELECT 
    CONCAT(merchant_city, ', ', merchant_state) as content,
    JSON_OBJECT(
        'id', id,
        'amount', amount,
        'use_chip', use_chip,
        'date', date,
        'client_id', client_id
    ) as metadata
FROM finance_db.core_transaction;

Indexing for Performance

To ensure efficient semantic queries, it's crucial to create indexes on the Knowledge Bases:

CREATE INDEX ON client_kb;  
CREATE INDEX ON transaction_kb;

This optimizes the underlying vector search for faster retrieval.

Automated Data Ingestion with MindsDB JOBS

For a dynamic application, new data is constantly added. In our case as we inserted the data from CSV files we can simulate that by spliting the files and inserting the last CSV file to simulate new transactions or we can add them in the Django admin app in the backend. MindsDB JOBS allow for periodic updates of the Knowledge Bases. We will set up a job to automatically insert new transactions into transaction_kb every hour:

CREATE JOB update_transaction_kb_job (  
    INSERT INTO transaction_kb SELECT * FROM finance_db.transactions WHERE date > LAST;  
) EVERY 1 HOUR;

The WHERE date > LAST clause is a powerful MindsDB feature that intelligently fetches only records added since the last job run or ingestion, ensuring efficiency.

Multi-step Workflows: KB + AI Table for Summarization

One of the most exciting aspects is combining KBs with MindsDB AI Tables for multi-step workflows. We will create an AI Table using Google Gemini (you can use OpenAI if you got moola) to summarize transaction details. You need to configure your Gemini engine in MindsDB first.

First, set up your Gemini engine (if you haven't already):

CREATE ML_ENGINE google_gemini_engine
FROM google_gemini
USING
    api_key = 'AIzaSyA0for-example';

Then, define the summarization model:

CREATE ML_MODEL summarize_transactions_model
PREDICT summary
USING
    engine = 'google_gemini_engine', 
    model_name = 'gemini-pro',
    prompt_template = 'Summarize the following transaction details into a concise overview, highlighting key aspects: {{transaction_details}}.';

Now, the magic happens in a combined query, this will be called in the view of the Django application(You don't need to call it through the MindsDB web view):

SELECT  
    t.id AS transaction_id,  
    t.amount,  
    t.date,  
    t.merchant_city,  
    t.merchant_state,  
    t.use_chip,  
    t.client_id,  
    s.summary -- The predicted summary from the AI model  
FROM  
    transaction_kb AS t  
JOIN  
    summarize_transactions_model AS s ON 1=1 -- Join the KB with the AI model  
WHERE  
    t.content LIKE 'unusual large spending' AND -- Semantic search for transactions  
    s.transaction_details = CONCAT( -- Feed relevant columns into the AI model's prompt  
        'Transaction ID: ', t.id,  
        ', Amount: $', t.amount,  
        ', Date: ', t.date,  
        ', Merchant City: ', t.merchant_city,  
        ', Merchant State: ', t.merchant_state,  
        ', Used Chip: ', t.use_chip,  
        ', Client ID: ', t.client_id  
    );

This query first finds transactions semantically related to "unusual large spending" from the transaction_kb, and then for each found transaction, it sends a concatenated string of its details to the summarize_transactions_model to generate a summary.

MindsDB Integration in Django (mindsdb_util.py)

To bridge Django and MindsDB, we create a mindsdb_util.py file within my Django core app. This utility uses the mindsdb_sdk Python library to connect to and query MindsDB. All the select queries will be called from a django view and generate data to be shown in the dashboard.

import os
from typing import List, Dict, Any, Optional
from django.conf import settings
from django.db import models
import mindsdb_sdk
from .models import Transaction, Client, Card


class MindsDBUtil:
    """
    Utility class for executing MindsDB SQL queries using mindsdb_sdk
    """

    def __init__(self):
        """Initialize the MindsDB connection using mindsdb_sdk"""
        try:
            # Get MindsDB connection parameters from environment or settings
            host = getattr(settings, 'MINDSDB_HOST', 'localhost')
            port = getattr(settings, 'MINDSDB_PORT', 47334)
            username = getattr(settings, 'MINDSDB_USER', None)
            password = getattr(settings, 'MINDSDB_PASSWORD', None)

            # Initialize MindsDB connection
            if username and password:
                self.connection = mindsdb_sdk.connect(
                    host=host,
                    port=port,
                    username=username,
                    password=password
                )
            else:
                self.connection = mindsdb_sdk.connect(
                    host=host,
                    port=port
                )

            self.server = self.connection.get_server()
            print(f"Successfully connected to MindsDB at {host}:{port}")

        except Exception as e:
            print(f"Warning: Could not initialize MindsDB connection: {e}")
            self.connection = None
            self.server = None

    def execute_query(self, query: str) -> List[Dict[str, Any]]:
        """
        Execute a MindsDB SQL query and return results

        Args:
            query: SQL query to execute

        Returns:
            List of dictionaries containing query results
        """
        if not self.connection:
            raise Exception("MindsDB connection not available")

        try:
            # Execute the query
            result = self.connection.query(query)

            # Convert to list of dictionaries
            if hasattr(result, 'fetchall'):
                columns = [desc[0] for desc in result.description]
                rows = result.fetchall()
                return [dict(zip(columns, row)) for row in rows]
            else:
                # Handle different result formats
                return result.to_dict('records') if hasattr(result, 'to_dict') else []

        except Exception as e:
            print(f"Error executing query: {e}")
            raise

    def find_wealthy_clients(self, min_age: int = 40, min_income: float = 70000) -> List[Dict[str, Any]]:
        """
        Find clients in wealthy areas with age and income filtering

        Note: This assumes your client_kb has columns: id, address, current_age, per_capita_income, gender
        You may need to adjust based on your actual KB schema
        """
        query = f"""
        SELECT
            c.id AS client_id,
            c.address,
            c.current_age,
            c.per_capita_income,
            c.gender
        FROM
            client_kb AS c
        WHERE
            c.current_age > {min_age} AND                    
            c.per_capita_income > {min_income};
        """
        return self.execute_query(query)

    def find_travel_expenses(self, min_amount: float = 500, use_chip: bool = True) -> List[Dict[str, Any]]:
        """
        Find transactions related to travel with amount and chip usage filtering

        Note: This assumes your transaction_kb has appropriate columns
        """
        # Use proper boolean value for MindsDB
        chip_value = 'true' if use_chip else 'false'
        query = f"""
        SELECT
            t.id AS transaction_id,
            t.amount,
            t.date,
            t.merchant_city,
            t.merchant_state,
            t.use_chip,
            t.client_id
        FROM
            transaction_kb AS t
        WHERE
            t.amount > {min_amount} AND                   
            t.use_chip = {chip_value};
        """
        return self.execute_query(query)

    def find_online_shopping(self, state: str = 'California') -> List[Dict[str, Any]]:
        """
        Find transactions for online shopping in a specific state
        """
        query = f"""
        SELECT
            t.id AS transaction_id,
            t.amount,
            t.merchant_city,
            t.merchant_state,
            t.client_id
        FROM
            transaction_kb AS t
        WHERE
            t.merchant_state = '{state}';
        """
        return self.execute_query(query)

    def semantic_search_transactions(self, search_term: str, limit: int = 10) -> List[Dict[str, Any]]:
        """
        Perform semantic search on transaction knowledge base

        Args:
            search_term: Natural language search term (e.g., "suspicious activity", "travel expenses")
            limit: Maximum number of results to return
        """
        query = f"""
        SELECT 
            *
        FROM transaction_kb 
        WHERE 
            MATCH('{search_term}')
        LIMIT {limit};
        """
        return self.execute_query(query)

    def semantic_search_clients(self, search_term: str, limit: int = 10) -> List[Dict[str, Any]]:
        """
        Perform semantic search on client knowledge base

        Args:
            search_term: Natural language search term (e.g., "wealthy suburban areas")
            limit: Maximum number of results to return
        """
        query = f"""
        SELECT 
            *
        FROM client_kb 
        WHERE 
            MATCH('{search_term}')
        LIMIT {limit};
        """
        return self.execute_query(query)

    def get_transaction_summary(self, transaction_id: int) -> List[Dict[str, Any]]:
        """
        Get AI-generated summary for a specific transaction

        Note: This assumes you have a model named 'summarize_transactions_model'
        and that the model expects transaction_id as input
        """
        query = f"""
        SELECT 
            transaction_id,
            summary
        FROM summarize_transactions_model 
        WHERE 
            transaction_id = {transaction_id};
        """
        return self.execute_query(query)

    def analyze_suspicious_patterns(self, client_id: Optional[int] = None) -> List[Dict[str, Any]]:
        """
        Use AI model to analyze suspicious transaction patterns

        This is a more realistic approach than the original join-based method
        """
        where_clause = f"WHERE client_id = {client_id}" if client_id else ""

        query = f"""
        SELECT 
            t.id,
            t.amount,
            t.date,
            t.merchant_city,
            t.merchant_state,
            t.client_id,
            p.risk_score,
            p.risk_reason
        FROM transaction_kb t
        JOIN pattern_analysis_model p ON t.id = p.transaction_id
        {where_clause}
        ORDER BY p.risk_score DESC;
        """
        return self.execute_query(query)

    def get_knowledge_base_stats(self) -> Dict[str, int]:
        """
        Get statistics about the knowledge bases
        """
        try:
            client_count = self.execute_query("SELECT COUNT(*) as count FROM client_kb;")
            transaction_count = self.execute_query("SELECT COUNT(*) as count FROM transaction_kb;")

            return {
                'client_kb_count': client_count[0]['count'] if client_count else 0,
                'transaction_kb_count': transaction_count[0]['count'] if transaction_count else 0
            }
        except Exception as e:
            print(f"Error getting knowledge base stats: {e}")
            return {'client_kb_count': 0, 'transaction_kb_count': 0}

    def custom_semantic_search(self, 
                             search_term: str, 
                             kb_type: str = 'transaction',
                             filters: Dict[str, Any] = None,
                             limit: int = 10) -> List[Dict[str, Any]]:
        """
        Perform custom semantic search on knowledge bases with optional filters

        Args:
            search_term: Natural language search term
            kb_type: 'transaction' or 'client'
            filters: Dictionary of filters to apply
            limit: Maximum number of results

        Returns:
            List of matching results
        """
        kb_name = f"{kb_type}_kb"

        # Build the base semantic search query
        query = f"SELECT * FROM {kb_name} WHERE MATCH('{search_term}')"

        # Add filters if provided
        if filters:
            filter_conditions = []
            for key, value in filters.items():
                if isinstance(value, str):
                    # Escape single quotes in string values
                    escaped_value = value.replace("'", "''")
                    filter_conditions.append(f"{key} = '{escaped_value}'")
                elif isinstance(value, bool):
                    filter_conditions.append(f"{key} = {'true' if value else 'false'}")
                else:
                    filter_conditions.append(f"{key} = {value}")

            if filter_conditions:
                query += " AND " + " AND ".join(filter_conditions)

        query += f" LIMIT {limit};"
        return self.execute_query(query)

    def test_connection(self) -> bool:
        """
        Test if MindsDB connection is working
        """
        try:
            if not self.connection:
                return False

            # Try a simple query
            result = self.execute_query("SELECT 1 as test;")
            return len(result) > 0 and result[0].get('test') == 1

        except Exception as e:
            print(f"Connection test failed: {e}")
            return False

    def list_available_tables(self) -> List[str]:
        """
        List all available tables/knowledge bases in MindsDB
        """
        try:
            result = self.execute_query("SHOW TABLES;")
            return [row.get('table_name', row.get('Tables_in_mindsdb', '')) for row in result]
        except Exception as e:
            print(f"Error listing tables: {e}")
            return []

    def describe_table(self, table_name: str) -> List[Dict[str, Any]]:
        """
        Get schema information for a specific table
        """
        try:
            return self.execute_query(f"DESCRIBE {table_name};")
        except Exception as e:
            print(f"Error describing table {table_name}: {e}")
            return []


# Global instance for easy access
mindsdb_util = MindsDBUtil()

Django views then simply import mindsdb_client and call its query() method:

from django.shortcuts import render  
from .mindsdb_util import mindsdb_client  

def client_kb_search_view(request):  
    query_text = request.GET.get('query', '')  
    if query_text:  
        sql_query = f"SELECT id, address, current_age, per_capita_income FROM client_kb WHERE content LIKE '{query_text}' LIMIT 10;"  
        results_df = mindsdb_client.query(sql_query)  
        # Convert to list of dicts for template  
        results = results_df.to_dict('records') 
        
    return render(request, 'finance/client_kb_search.html', {'query': query_text, 'results': results})

This application demonstrates several powerful use cases for MindsDB Knowledge Bases in a financial context:

Customer Segmentation & Targeted Marketing: Semantically search for "affluent clients in suburban areas" to tailor product offerings.
Fraud Detection: Query "suspicious transactions in unusual locations" or "unusually large spending patterns" and get AI summaries for quick analysis by human analysts.
Risk Assessment: Identify clients or transaction types that semantically match "high-risk profiles."
Compliance & Audit: Quickly retrieve all data points related to specific semantic concepts for reporting.

Conclusion

MindsDB Knowledge Bases significantly lower the barrier to integrating advanced semantic search and AI capabilities into existing data infrastructure. The ability to combine natural language queries with traditional SQL filtering, and to chain these with AI Tables, opens up a world of possibilities for building intelligent data applications.

This project was a great experience in leveraging MindsDB's powerful features to transform raw data into actionable, semantically queryable insights within a familiar Django environment.

GitHub Repository: [https://github.com/samaras/django-mindsdb-transactions]
MindsDB Docs: https://mindsdb.com/docs/

Greedy algorithm

Samuel Komfi — Fri, 14 Jun 2024 19:41:26 +0000

This is a submission for DEV Computer Science Challenge v24.06.12: One Byte Explainer.

Explainer

A greedy algorithm is like a kid in a candy store: it grabs the best candy (local optimum) it sees without thinking ahead, hoping to end up with the most candy (global optimum). It’s used in problems like finding the shortest path or scheduling tasks.

Additional Context

Greedy but effective

How to search for files in Linux

Samuel Komfi — Fri, 23 Sep 2022 21:06:04 +0000

Introduction

This guide was created to help you understand how to search for files in a Linux system. The tutorial was based on a Linux Mint 18.3 installation.
Linux uses different commands to help you look for files using regular expressions. Imagine looking for a file, but don't dont remember
much except a small part of the name of the file.

Prerequisites

Before you begin this guide you'll need the following:

Desktop/laptop/server with Ubuntu/Linux Mint installed or any other Linux distribution really.

Step 1 — Using the grep command

Grep is used to search for characters inside files. It also has support for regular expressions. The family of grep commands include the
following commands: grep, egrep, fgrep.
egrep and grep are more or less the same with just minor differences. fgrep uses fixed strings to extract text from files. Grep can also take in input from the standard input.

Search for pattern in files:

$ grep [options] pattern files

The commands will read each line one at time.

\* will match any character
? or . will only match a single character
[xxx] will match a range of characters
\x will escape characters such as ? or \ which are used in other regexp
^pattern will match anything that starts with pattern
$pattern will match anything that ends with pattern

Search recursively for pattern in the home directory:

$ grep -r pattern /home

Display the find commands in your bash_history with the following:

$ grep -a find .bash_history

You'll see the following output:

[Output]

find /home -size +10000k
man find
man find > find.txt
gedit find.txt
find -type f -mtime -1 -print

Now transition to the next step by telling the reader what's next.

The find command

The find command is a slow but reliable option if you are searching for a file in your system. If the file exists, find is more likely to find it eventually. It has the following format:

$ find [path] [options]

$ find /home/tom -name 'index*'

Find can used to search for files in choosen directories. You can specify a list of directories to search in.
The find command can also search using the following criteria:

Search by size of the file
Search by name of the file
Search by permission of the files -
Search by group the file owner belongs to -gid
Search by the owner of the file -uid & -user
Search by type -type The possible arguments to -type are b{block device}, c {character device}, d {directory}, p {named pipe}, f {regular file}, l {symbolic link}

Find files names that start with "FB":

$ find /home -name 'FB*'

Find files that are larger than 10 MB in the home folder:

$ find /home -size +10000k

Find files by the type of permission allowed:

$ find /home -perm

Find files by the group of the owner of the file belongs to:

$ find -user root

<$>[warning]
Warning: The -user option will only work if the user exists and is not removed. Use the -uid argument to reference the user.
<$>

The -maxdepth levels option is used to limit how far deep the find command will search in the subdirectories.

The -exec options allows you to execute other commands on the files found. The following will remove any .tmp files found in the home directory.

$ find /home -name "*.tmp" -exec rm {} \;

To list files of a certain type which have also been accessed in the past 24 hours you can run the follwoing command:

$ find /home -mtime -1 \! -type d

The -type options specifies the type of file. In this case a directory. You add a ! to list any file type except a directory.

Locate and the whereis command

The whereis command is used to search for files in the systems path.

$ whereis elixir

The locate command is for finding files using just a simple string or filename. The command assumes you have called the updatedb. The updatedb command creates or updates a database file which has all the files attached to the linux root directory. Locate will use this database to give you the location of the file.

Find all instances of filename:

$ locate filename

ls command

The ls command is used to list the contents of the current working directory. Its one of the most used linux commands.

The most common options used with ls are the -a and -l options.
The ls -a command is used to list all files including hidden files, while the ls -l command is for listing all files with all their size, permissions and ownership information. You can also use special characters to abbreviate file names, this is called 'globbing'. It should not be confused with regular expressions which is used in the grep command.

$ ls -l filename

To list all text files in the directory run:

$ ls *.txt

The following will list any file that has a number as its name including the extension. If you omit the last star at the end, only
files ending with numbers will be listed e.g ls *[0-9].

$ ls *[0123456789]*
$ ls *[0-9]*

I got the following output on my home folder:

[digit_files Output]
buffalo_0.18.2_Linux_x86_64.tar.gz
Calculus Made Easy33283-pdf.pdf
ccenh.mp4
Cogele El Golpe - King Bongo [360p].mp4
com.keramidas.virtual.WIFI_AP_LIST-20190908-150043.tar.gz
COR14.3 (7).pdf
De Mthuda - Wamuhle ft Njelic.mp3
DesignPatternsInPython_ver0.1.pdf
design-principles4.pdf
Django 2.0 release notes _ Django documentation _ Django.pdf
djangoadvent-articles06_object-permissions.rst
DomAC6HW0Accg_6.jpg
Dxmd1ybWsAEIfc8.jpg
ejabberd 2.1.pdf
FB_IMG_1431202962739.jpg
FB_IMG_1450904477927.jpg
FB_IMG_1458068934007.jpg

$ ls *[-a-z][0-9]*

$ ls FB_IMG_1*[0-9]*.jpg

I got the following output on my home folder:

[secondary_label Output]

FB_IMG_1431202962739.jpg
FB_IMG_1450904477927.jpg
FB_IMG_1458068934007.jpg

I could also have gotten the same output using an asterik representing any zero or more characters after the 'FB_IMG_' string.

$ ls FB_IMG_*

You can escape characters with a slash "\"

$ ls *\?*

Nugget

You can search the manual pages for help in looking for a particular name and description using the apropos command. This will search the whatis database for files that have the specified string.

$ apropos search

The following output lists all the commands I can use to 'search' on my computer.

[Output]
ALTER_TEXT_SEARCH_CONFIGURATION (7) - change the definition of a text search configuration
ALTER_TEXT_SEARCH_DICTIONARY (7) - change the definition of a text search dictionary
ALTER_TEXT_SEARCH_PARSER (7) - change the definition of a text search parser
ALTER_TEXT_SEARCH_TEMPLATE (7) - change the definition of a text search template
anemotaxis (6x)      - directional search on a plane.
apropos (1)          - search the manual page names and descriptions
badblocks (8)        - search a device for bad blocks
bsearch (3)          - binary search of a sorted array
bzegrep (1)          - search possibly bzip2 compressed files for a regular expression
bzfgrep (1)          - search possibly bzip2 compressed files for a regular expression
bzgrep (1)           - search possibly bzip2 compressed files for a regular expression
CREATE_TEXT_SEARCH_CONFIGURATION (7) - define a new text search configuration
CREATE_TEXT_SEARCH_DICTIONARY (7) - define a new text search dictionary
CREATE_TEXT_SEARCH_PARSER (7) - define a new text search parser
CREATE_TEXT_SEARCH_TEMPLATE (7) - define a new text search template
docker-search (1)    - Search the Docker Hub for images
DROP_TEXT_SEARCH_CONFIGURATION (7) - remove a text search configuration
DROP_TEXT_SEARCH_DICTIONARY (7) - remove a text search dictionary
DROP_TEXT_SEARCH_PARSER (7) - remove a text search parser
DROP_TEXT_SEARCH_TEMPLATE (7) - remove a text search template
FcConfigGetCacheDirs (3) - return the list of directories searched for cache files
find (1)             - search for files in a directory hierarchy
git-bisect (1)       - Use binary search to find the commit that introduced a bug
hsearch (3)          - hash table management
hsearch_r (3)        - hash table management
lfind (3)            - linear search of an array
lsearch (3)          - linear search of an array
lzegrep (1)          - search compressed files for a regular expression
lzfgrep (1)          - search compressed files for a regular expression
lzgrep (1)           - search compressed files for a regular expression
manpath (1)          - determine search path for manual pages
res_nsearch (3)      - resolver routines
res_search (3)       - resolver routines
strpbrk (3)          - search a string for any of a set of bytes
tsearch (3)          - manage a binary tree
wcschr (3)           - search a wide character in a wide-character string
wcscspn (3)          - search a wide-character string for any of a set of wide characters
wcspbrk (3)          - search a wide-character string for any of a set of wide characters
wcsrchr (3)          - search a wide character in a wide-character string
wmemchr (3)          - search a wide character in a wide-character array
wx-config (1)        - wxWidgets configuration search and query tool
XtFindFile (3)       - search for a file using substitutions in the path list
XtResolvePathname (3) - search for a file using standard substitution
xzegrep (1)          - search compressed files for a regular expression
xzfgrep (1)          - search compressed files for a regular expression
xzgrep (1)           - search compressed files for a regular expression
zegrep (1)           - search possibly compressed files for a regular expression
zfgrep (1)           - search possibly compressed files for a regular expression
zgrep (1)            - search possibly compressed files for a regular expression
zipgrep (1)          - search files in a ZIP archive for lines matching a pattern

Conclusion

The article introduced the most frequently used command to search for files in linux. I hope you learned something new by reading this article,
and can now fully embrace the power of some the commands listed here. For more information on the commands there is the man pages which will provide more technical
information. The ls command alone has over 70 options.