DEV Community

Cover image for Build a Live Search Feature with Django using HTMX and PostgreSQL FTS (in 10 minutes)
Zach
Zach

Posted on • Originally published at circumeo.io

Build a Live Search Feature with Django using HTMX and PostgreSQL FTS (in 10 minutes)

We're going to build a simple app that uses advanced PostgreSQL full-text search features, with results that update as the user enters their query.

This includes:

  • using HTMX to fetch results as the user types (without extra JavaScript)
  • allowing the user to enter search engine style queries (i.e. "python or django")
  • adding a Django management command to quickly insert test data

Here's an image of the working product:

Full text search example

Previewing the project

Want to see a live version of the app? You can create a copy of this project now and try it out.

Copy the FullTextSearchApp Project

Setting up the Django app

Install packages and create the Django application.

pip install --upgrade django faker
django-admin startproject eventsearch .
python3 manage.py startapp core
Enter fullscreen mode Exit fullscreen mode

Add core to the INSTALLED_APPS list.

# settings.py
INSTALLED_APPS = [
  "core",
    ...
]
Enter fullscreen mode Exit fullscreen mode

Adding the templates

  • Create a directory named templates within the core app.
  • Create a file named search.html within the templates directory.
<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8">
        <script src="https://unpkg.com/htmx.org"></script>
        <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/css/bootstrap.min.css"
              rel="stylesheet"
              integrity="sha384-EVSTQN3/azprG1Anm3QDgpJLIm9Nao0Yz1ztcQTwFspd3yD65VohhpuuCOmLASjC"
              crossorigin="anonymous">
        <link href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.10.0/font/bootstrap-icons.css"
              rel="stylesheet">
        <style>
      html,
      body {
        height: 100%;  
      }

      #content {
        display: flex;
        flex-direction: column;
        height: 100%;
      }

      #events-content {
        background-color: rgb(246, 247, 248);
        flex: 1;
      }

      #events-content-inner {
        max-width: 600px;
        width: 100%;
      }

      #event-search-input {
        border-top-left-radius: 8px;
        border-bottom-left-radius: 8px;
        border: 1px solid rgb(217, 217, 217);
        border-right: none;
        max-width: 600px;
        outline: none;
      }

      #search-submit-btn {
        background-color: rgb(246, 88, 88);
        border-top-right-radius: 8px;
        border-bottom-right-radius: 8px;
        border: none;
        color: #fff;
        width: 42px;
      }

      .search-result {
        border-bottom: 1px solid #ccc;
      }

      .search-result-inner {
         display: flex;
         gap: 1rem;
      }

      .event-thumbnail {
         border-radius: 8px;
         width: 160px;
         height: 100px;
      }

      .event-description {
         font-size: 13px;  
      }

       .event-date {
         color: rgb(124, 111, 80);
         font-weight: 500;
       }
        </style>
    </head>
    <body>
        <div id="content">
            <form method="post">
                {% csrf_token %}
                <div id="event-search" class="d-flex justify-content-center p-3">
                    <div class="d-flex justify-content-center w-50">
                        <input id="event-search-input"
                               name="q"
                               class="w-100 p-2"
                               type="text"
                               value="{{ query }}"
                               placeholder="Search events"
                               hx-post="/"
                               hx-select="#events-content-inner"
                               hx-swap="outerHTML"
                               hx-trigger="input changed delay:500ms, search"
                               hx-target="#events-content-inner" />
                        <button id="search-submit-btn" type="submit">
                            <i class="bi bi-search"></i>
                        </button>
                    </div>
                </div>
            </form>
            <div id="events-content" class="d-flex justify-content-center">
                <div id="events-content-inner" class="p-4">
                    {% for event in events %}
                        <div class="search-result py-4">
                            <div class="search-result-inner">
                                <div>
                                    {% if event.image_url %}<img class="event-thumbnail" src="{{ event.image_url }}" />{% endif %}
                                </div>
                                <div>
                                    <div class="event-date">{{ event.event_date|date:"l, F d" }} {{ event.event_time }}</div>
                                    <div>{{ event.name }}</div>
                                    <div class="event-description mt-2">{{ event.description }}</div>
                                </div>
                            </div>
                        </div>
                    {% empty %}
                    <div class="mx-auto text-center py-4">
                      <p>No events found.  Have you run the management command yet?</p>  
                      <p>Click Shell in the IDE and then Connect.</p>
                      <pre class="mt-5">python3 manage.py populate_events</pre>
                    </div>
                    {% endfor %}
                </div>
            </div>
        </div>
    </body>
</html>
Enter fullscreen mode Exit fullscreen mode

Adding the views

Copy and paste the following into views.py within the core directory.

from django.contrib.postgres.search import SearchQuery
from django.shortcuts import render
from core.models import Event


def search(request):
    events, q = [], ""

    if request.method == "POST":
        q = request.POST.get("q")
        if q:
            events = Event.objects.filter(
                description_vector=SearchQuery(q, search_type="websearch")
            )

    if not events:
        events = Event.objects.all()

    return render(request, "core/search.html", {"query": q, "events": events})
Enter fullscreen mode Exit fullscreen mode

Updating URLs

Create urls.py in the core directory.

from django.urls import path
from .views import search

urlpatterns = [
    path("", search, name="search")
]
Enter fullscreen mode Exit fullscreen mode

Update the existing urls.py within the project eventsearch directory.

from django.contrib import admin
from django.urls import include, path

urlpatterns = [
    path("admin/", admin.site.urls),
    path("", include("core.urls")),
]
Enter fullscreen mode Exit fullscreen mode

Adding the database models

Overwrite the existing models.py with the following:

from psycopg2.extensions import register_adapter, AsIs

from django.db import models
from django.db.models import F
from django.contrib.auth.models import User

from django.contrib.postgres.search import SearchVectorField
from django.contrib.postgres.indexes import GinIndex

class PostgresDefaultValueType:
    pass

# Wrap the built-in SearchVectorField so that we pass DEFAULT when value is None.
# Otherwise, the Django ORM will try to update the `description_vector` field, which
# PostgreSQL won't allow because it is a GENERATED column.

# Credit for this approach belongs here:
# https://code.djangoproject.com/ticket/21454#comment:28
class DelegatedSearchVectorField(SearchVectorField):
    def get_prep_value(self, value):
        return value or PostgresDefaultValueType()

register_adapter(PostgresDefaultValueType, lambda _: AsIs('DEFAULT'))

class Event(models.Model):
    name = models.CharField(max_length=100)
    description = models.TextField()
    description_vector = DelegatedSearchVectorField()

    image_url = models.URLField(max_length=1024, blank=True, null=True)
    event_date = models.DateField()
    event_time = models.TimeField()
Enter fullscreen mode Exit fullscreen mode

Adding a Manual Migration for the Search Vector Field

Run the migration command to generate a new empty migration.

python3 manage.py makemigrations core --empty
Enter fullscreen mode Exit fullscreen mode

Populate the empty migration with the following.

from django.db import migrations

class Migration(migrations.Migration):

    dependencies = [
        ('core', '0001_initial'),
    ]

    operations = [
        migrations.RunSQL(
            """
            ALTER TABLE core_event ADD COLUMN description_vector tsvector GENERATED ALWAYS AS (
                setweight(to_tsvector('english', coalesce(name, '')), 'A') ||
                setweight(to_tsvector('english', coalesce(description, '')), 'B')
            ) STORED;
            CREATE INDEX IF NOT EXISTS description_vector_gin ON core_event USING GIN(description_vector);
            """,
            reverse_sql=
            """
            ALTER TABLE core_event DROP COLUMN description_vector;
            """,
        ),
    ]
Enter fullscreen mode Exit fullscreen mode

The migration will create an automatically updated tsvector by using a GENERATED column. Django 5.0 supports the GeneratedField type, but the SQL expression is too complex to handle that way, so we'll use a manual migration to handle it without the ORM.

Adding a Django Management Command for Test Data

  • Create the directory structure management/commands within the core folder.
  • Open a file named populate_events.py within the new directory and enter the following.
from datetime import datetime, timedelta
import random

from django.core.management.base import BaseCommand
from django.contrib.auth.models import User
from django.utils.timezone import make_aware
from faker import Faker

from core.models import Event

class Command(BaseCommand):
    def handle(self, *args, **options):
        self.stdout.write(self.style.SUCCESS('Starting database population...'))

        faker = Faker()

        for i in range(10):
            first_name = faker.first_name()
            last_name = faker.last_name()
            User.objects.create_user(username=f'user{i}',
                                     email=f'user{i}@example.com',
                                     password='password',
                                     first_name=first_name,
                                     last_name=last_name)

        event_names = [
            'Django for Beginners',
            'Python Data Science Meetup',
            'React Native Workshop',
            'Machine Learning Basics',
            'Introduction to IoT',
            'Blockchain 101',
            'Advanced Django Techniques',
            'Full-stack Development with Python and JavaScript',
            'Mobile App Development Trends',
            'Cybersecurity Fundamentals',
        ]

        descriptions = [
            'Learn Django through practical examples, from setting up your environment to building and deploying a web application.',
            'Explore data science techniques using Python, including statistical analysis, machine learning algorithms, and data visualization.',
            'Build your first React Native app by learning the basics of this cross-platform framework and creating a simple application.',
            'An introduction to machine learning concepts, covering foundational principles, algorithms, and real-world applications.',
            'Getting started with the Internet of Things (IoT): Learn how to connect devices and collect data for analysis and insights.',
            'Understanding blockchain technology, from the basics of decentralized networks to smart contracts and cryptocurrency.',
            'Deep dive into Django features, exploring advanced functionalities that can help in building robust and scalable web applications.',
            'Learn full-stack development from front to back, covering everything from basic HTML/CSS to backend programming and databases.',
            'What\'s new in mobile app development: Explore the latest trends, tools, and technologies shaping the future of mobile applications.',
            'Protect your digital assets with cybersecurity basics, learning about threats, safeguards, and best practices for online security.',
        ]

        image_urls = [
          'https://circumeo.io/static/images/server-racks.png',
          'https://circumeo.io/static/images/typewriter.jpg',
          'https://circumeo.io/static/images/library.jpg',
          'https://circumeo.io/static/images/landscape.jpg',
          'https://circumeo.io/static/images/laptop.jpg',
          'https://circumeo.io/static/images/latte.jpg',
        ]

        for name, description in zip(event_names, descriptions):
            image_url = random.choice(image_urls)

            event_date = datetime.now() + timedelta(days=random.randint(1, 90))
            event_time = datetime.now().time()

            Event.objects.create(name=name, description=description, image_url=image_url, event_date=make_aware(event_date), event_time=event_time)

        self.stdout.write(self.style.SUCCESS('Database population completed successfully!'))
Enter fullscreen mode Exit fullscreen mode

Open a shell session in order to run the Django management command.

python3 manage.py populate_events
Enter fullscreen mode Exit fullscreen mode

Up and Running with Full Text Search

That's it! You now have an app up and running that incorporates powerful full-text search capabilities. Not to mention, the search is dynamic as well, without the need for additional JavaScript, thanks to HTMX.

Top comments (0)