When you need to use GraphQL with Django, a common practice is to use Graphene-Django, which allows you to query Django models. However, directly querying Django models might be too slow, especially when you have many complex relations. To speed that up, you can add one more layer of abstraction and index your Django models to an Elasticsearch server. In this article, I will show you how to create a GraphQL interface for querying posts from the Elasticsearch server.
Elasticsearch Server
The easiest way to install the Elasticsearch server locally is using EVM - Elasticsearch Version Manager:
$ sudo curl -o /usr/local/bin/evm https://raw.githubusercontent.com/duydo/evm/master/evm
$ sudo chmod +x /usr/local/bin/evm
Then you can install Elasticsearch 7.17.27 (or other version) with EVM, as follows:
$ evm install 7.17.27
By default, the Elasticsearch server will raise a warning if you don't have HTTPS or basic authentication. You can change one setting to turn off those warnings for localhost. To do so, add a line to ~/.evm/elasticsearch-7.17.27/config/elasticsearch.yml
:
xpack.security.enabled: false
Now, you can start the server with:
$ evm start
If you need more Elasticsearch versions on the same computer, you can easily install and switch between them.
Python Packages
Install the latest packages of Django Elasticsearch DSL for connecting to Elasticsearch server, and Graphene-Django for GraphQL implementation:
(venv)$ pip install Django
(venv)$ pip install django-elasticsearch-dsl
(venv)$ pip install graphene-django
Django Settings
Add the installed apps to INSTALLED_APPS
in the myproject/settings.py. Also, add a few more settings that we will need in this exercise:
INSTALLED_APPS = [
# Contributed Django apps
# ...
# Third-party apps
"django_elasticsearch_dsl",
"graphene_django",
# Local apps
"posts",
]
PROJECT_NAME = "myproject"
ENVIRONMENT = "dev"
ELASTICSEARCH_DSL = {
"default": {
"hosts": "http://localhost:9200",
}
}
GRAPHENE = {
"SCHEMA": "elasticsearch_graphql.schema.schema",
}
Models and Administration for the Posts App
Let's create a posts app with two models: Category
and Post
:
The posts/models.py file will look like this:
from django.db import models
class Category(models.Model):
title = models.CharField("Title", max_length=255)
slug = models.SlugField("Slug", max_length=255)
class Meta:
verbose_name = "Category"
verbose_name_plural = "Categories"
ordering = ["title"]
def __str__(self):
return self.title
class Post(models.Model):
title = models.CharField("Title", max_length=255)
slug = models.SlugField("Slug", max_length=255)
content = models.TextField("Content")
categories = models.ManyToManyField(Category)
class Meta:
verbose_name = "Post"
verbose_name_plural = "Posts"
ordering = ["title"]
def __str__(self):
return self.title
Also let's create administration posts/admin.py to be able to add some entries easily:
from django.contrib import admin
from .models import Category, Post
@admin.register(Category)
class CategoryAdmin(admin.ModelAdmin):
list_display = ["title", "slug"]
prepopulated_fields = {"slug": ["title"]}
@admin.register(Post)
class Post(admin.ModelAdmin):
list_display = ["title", "slug"]
list_filter = ["categories"]
filter_horizontal = ["categories"]
prepopulated_fields = {"slug": ["title"]}
Create the database schema and the superuser, then run the local development server with the following Django management commands:
(venv)$ python makemigrations
(venv)$ python migrate
(venv)$ python createsuperuser
(venv)$ python runserver
Add some posts and categories in the administration:
http://localhost:8000/admin/
Elasticsearch Documents for the Posts
Let's create PostDocument
definition for the Elasticsearch index of the Post
model and its categories at posts/documents.py:
import graphene
from django.conf import settings
from django_elasticsearch_dsl import Document
from django_elasticsearch_dsl.registries import registry
from django_elasticsearch_dsl import fields
from .models import Post
@registry.register_document
class PostDocument(Document):
title = fields.TextField()
slug = fields.KeywordField()
content = fields.TextField()
categories = fields.NestedField(
properties={
"title": fields.TextField(),
"slug": fields.KeywordField(),
}
)
class Index:
name = f"{settings.PROJECT_NAME}_{settings.ENVIRONMENT}_posts"
settings = {"number_of_shards": 1, "number_of_replicas": 0}
class Django:
model = Post
ignore_signals = False
auto_refresh = True
queryset_pagination = 5000
def prepare(self, instance):
data = super().prepare(instance)
# Prepare categories field
data["categories"] = [
{
"title": category.title,
"slug": category.slug,
}
for category in instance.categories.all()
]
return data
To index the posts on the Elasticsearch server, run the following:
(venv)$ python manage.py search_index --rebuild
Let's see if the entries were indexed and can be retrieved.
At first, run the Django shell:
(venv)$ python manage.py shell
Then try to list out items from the Elasticsearch index:
>>> from posts.documents import PostDocument
>>> for doc in PostDocument.search():
... print(doc.title)
It will list up to 10 posts (that's the default pagination) from the index:
Creating AI-based Summaries in a Django Website
Creating Open Graph Images in Django for Improved Social Media Sharing
HTTPS for Django Development Environment
GraphQL Types and Queries for the Posts App
Now, create the GraphQL schema posts/schema.py for the categories and posts, as follows:
import operator
from functools import reduce
import graphene
from .documents import PostDocument
class ElasticsearchObjectType(graphene.ObjectType):
@classmethod
def from_elasticsearch(cls, elasticsearch_document):
instance = cls()
instance.elasticsearch_document = elasticsearch_document
return instance
class ElasticsearchCategoryType(ElasticsearchObjectType):
title = graphene.String()
slug = graphene.String()
def resolve_title(self, info):
return self.elasticsearch_document.title
def resolve_slug(self, info):
return self.elasticsearch_document.slug
class ElasticsearchPostType(ElasticsearchObjectType):
title = graphene.String()
slug = graphene.String()
content = graphene.String()
categories = graphene.List(of_type=ElasticsearchCategoryType)
def resolve_title(self, info):
return self.elasticsearch_document.title
def resolve_slug(self, info):
return self.elasticsearch_document.slug
def resolve_content(self, info):
return self.elasticsearch_document.content
def resolve_categories(self, info):
return [
ElasticsearchCategoryType.from_elasticsearch(
elasticsearch_document=category
)
for category in self.elasticsearch_document.categories or []
]
class Query(object):
elasticsearch_posts = graphene.List(
ElasticsearchPostType,
limit=graphene.Int(),
offset=graphene.Int(),
search=graphene.String(),
categories=graphene.List(of_type=graphene.String),
)
def resolve_elasticsearch_posts(
self,
info,
limit=10,
offset=0,
search=None,
categories=None,
):
from elasticsearch_dsl.search import Q
search_obj = PostDocument.search()
queries = []
if search:
queries.append(
Q(
"multi_match",
query=search,
fields=[
"title^5",
"slug^4",
"content^3",
],
type="best_fields",
tie_breaker=0.3,
fuzziness="AUTO",
fuzzy_transpositions=True,
lenient=True,
)
)
if categories:
queries.append(
Q(
"nested",
path="categories",
query=Q("terms", categories__slug=categories),
)
)
if queries:
# All filters AND-ed
search_obj = search_obj.query(reduce(operator.iand, queries))
search_obj = search_obj[offset : offset + limit if limit else None]
return [
ElasticsearchPostType.from_elasticsearch(elasticsearch_document=result)
for result in search_obj.execute()
]
Here, we have an ElasticsearchObjectType
that provides a custom method from_elasticsearch
to initiate GraphQL objects from Elasticsearch instead of Django models.
The query for elasticsearch_posts
has two filters: by a search string and by a list of OR-ed category slugs. The search query looks into title
, slug
, and content
fields with different boosting weights and searches for the best match. The fuzziness set to "AUTO"
allows mistakes in the search string by two letters.
At the project level, create Graphene Relay - a schema class that uses a Query class inheriting from all the Query classes from all Django apps. The file can be located at myproject/schema.py and have this content:
import graphene
import posts.schema
class Query(posts.schema.Query, graphene.ObjectType):
pass
schema = graphene.Schema(query=Query)
Finally, enable the GraphQL endpoint in the myproject/urls.py:
from django.contrib import admin
from django.urls import path
from django.views.decorators.csrf import csrf_exempt
from graphene_django.views import GraphQLView
urlpatterns = [
path("graphql/", csrf_exempt(GraphQLView.as_view(graphiql=True))),
path("admin/", admin.site.urls),
]
Checking the GraphQL Queries
Navigate to http://localhost:8000/graphql/
Graphene-Django comes with a GraphiQL user interface, which, when accessed from a browser, allows the user to query GraphQL and see results in the same window:
A search query for "AI" with categories "Intermediate" or "Advanced" can look like:
query FilteredPosts {
elasticsearchPosts(
search: "AI"
categories: ["intermediate", "advanced"],
limit: 10,
offset: 0,
) {
title
slug
categories {
title
slug
}
}
}
It will produce results in JSON just like this:
{
"data": {
"elasticsearchPosts": [
{
"title": "Creating AI-based Summaries in a Django Website",
"slug": "creating-ai-based-summaries-in-a-django-website",
"categories": [
{
"title": "Basics",
"slug": "basics"
},
{
"title": "Django",
"slug": "django"
},
{
"title": "Gemini",
"slug": "gemini"
},
{
"title": "Intermediate",
"slug": "intermediate"
},
{
"title": "Large Language Models",
"slug": "large-language-models"
},
{
"title": "PyBazaar",
"slug": "pybazaar"
},
{
"title": "Simplemind",
"slug": "simplemind"
}
]
},
{
"title": "HTTPS for Django Development Environment",
"slug": "https-for-django-development-environment",
"categories": [
{
"title": "DevOps",
"slug": "devops"
},
{
"title": "Development",
"slug": "development"
},
{
"title": "Django",
"slug": "django"
},
{
"title": "HTTPS",
"slug": "https"
},
{
"title": "Intermediate",
"slug": "intermediate"
},
{
"title": "SSL",
"slug": "ssl"
},
{
"title": "TLS",
"slug": "tls"
}
]
}
]
}
}
Conclusion
Using Elasticsearch with GraphQL is not trivial, but it provides excellent performance, especially for complex data structures. The best way to implement that in Django that I know of is combining Django Elasticsearch DSL and Graphene-Django.
Cover photo Angela Roma
Top comments (0)