As a developer working on APIs or front-end applications, you need data in your database to test API endpoints and front-end components effectively. This article will show you how to import JSON data into Django models, a skill that's useful for other formats like CSV or text files in various backend frameworks.
Recently, I worked on an API development project where my data was stored in a JSON file. I built a backend using Django and Django Rest Framework to serve this data through API endpoints to a React-based front-end application. While Django offers tools like the Admin app, dumpdata, and loaddata (Tutorial) for data management, these tools can be complex for beginners and don’t handle file processing by default.
To simplify the process, I used a straightforward Python script to load my JSON data into Django models.
This article assumes you have already set up a Django project and configured it to serve data from your models via an API endpoint. However, for completeness and clarity, I have included the necessary steps and provided examples of the models.py and views.py files. For this, article is divided into two parts (Part I** & Part II), if you have a Django app with REST setup, you can skip directly to Part II: Dumping data (books.json) to Book model (Book()).
Part I: Steps to Set Up Your Django Environment**
1.Create a virtual environment using either conda or venv with Python 3.10 or later:
Example my env name is djangorestapi
Using venv:
$python -m venv myenv
$python -m venu djangorestapi
Using conda:
$conda create --name myenv python=3.10
$conda create --name djangorestapi python=3.10
2.Activate your virtual environment (Ubuntu/Linux):
$source djangorestapi/bin/activate
$conda activate djangorestapi
3.Install Django and Django Rest Framework
Offical Website:
Django
Note: Notice the activated virtual environment djangorestapi and project folder name devrestapi.
**(djangorestapi)**ajit/**devrestapi**$pip install django djangorestframework
4.Create a Project using Django admin command (we will use popular bookstore example so will have project name as bookstore and later there will be a app name as book and Model name will Book())
Note: For simplicity, I have removed the virtual environment name and project folder, but throughout this tutorial, ensure you are working within your virtual environment.
$django-admin startproject bookstore
5.Create a new app and call it book.
Note: You need to move inside the project folder (bookstore) created after running the command in step 4.
$cd bookstore
$python manage.py startapp book
6.Add the book app to the project’s settings by editing the INSTALLED_APPS list in the bookstore/settings.py file.
INSTALLED_APPS = [
...
'book',
...
]
7.Similar to book app you also need to add restframework app to use REST features in Django application.
INSTALLED_APPS = [
...
'rest_framework',
'book',
...
]
Make and run migration
It is always good practice to create and run migration after adding a new app to the Django project or creating or making changes in the models.py. So, you should do that as follows:
$python manage.py makemigrations
$python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
....
The step 1-7 will provide all the boilerplate code to get started with your API application.
Now, you can create a model Book in your book/models.py as below:
from django.db import models
class Book(models.Model):
title = models.CharField(max_length=200)
author = models.CharField(max_length=100)
price = models.DecimalField(max_digits=6, decimal_places=2)
def __str__(self):
return self.title
Serving the Book Model via API
To serve the Book model via an API, you need to accomplish two key tasks. First, create a serializer to convert model data to and from JSON. Second, create a view to map the serializer to your API endpoints.
Let's walk through these steps:
Create a Serializer:create a serializers.py file within your book app as follow:
#book/serializers.py
from rest_framework import serializers
from .models import Book
class BookSerializer(serializers.ModelSerializer):
class Meta:
model = Book
fields = ['id', 'title', 'author', 'price']
The views will handle the API requests and responses using the serializer.
We have two views for two API endpoints:
BookListAPIView(): To create a view that serves all Book model information via an API endpoint (/api/books/)
BookDetail(): To create a view that show details of one book based on the id. (books/int:pk/)
#book/views.py
from rest_framework import generics
from .models import Book
from .serializers import BookSerializer
class BookListCreate(generics.ListCreateAPIView):
queryset = Book.objects.all()
serializer_class = BookSerializer
class BookDetail(generics.RetrieveUpdateDestroyAPIView):
queryset = Book.objects.all()
serializer_class = BookSerializer
Once your model and view are ready, you can link your app route through urls.py (within the book app and the project urls.py).
#book/urls.py
from django.urls import path
from .views import BookListCreate, BookDetail
urlpatterns = [
path('books/', BookListCreate.as_view(), name='book-list-create'),
path('books/<int:pk>/', BookDetail.as_view(), name='book-detail'),
]
Now, map your app routes to the project route, using urls.py from project.
#bookstore/urls.py
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('api/', include('book.urls')),
]
Note: It is good time to create and run your migrations again.
$python manage.py makemigrations
Output:
Migrations for 'book':
book/migrations/0001_initial.py
- Create model Book
Output: migration
$python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, book, contenttypes, sessions
Running migrations:
Applying book.0001_initial... OK
By following these steps, you will have set up API endpoint /api/books/ and /api/book/int:pk/ that retrieves all Book model data or a single book data in JSON format when accessed via a GET request.
Let check that every thing is working fine by running Django webserver and going to the api endpoints.
$python manage.py runserver
Go to the browser and open localhost url as :
http://127.0.0.1:8000/api/books/
Now, if you try other API endpoint that is for getting the detail of a specific book id. such as:
http://127.0.0.1:8000/api/books/1
But, you will have a error because there is no data about any book and now in the next part we will dump data to Book Model.
Part II: Dumping data (books.json) to Book model (Book())
By default, Django utilizes SQLite for data storage. If you haven't set up another database, the following steps will work seamlessly. Any adjustments needed for specific databases will be minimal.
install and import required python modulesdefault modules in Python 3.10
$pip install sqlite3
$pip install json
import sqlite3
import json
In Django, the default database name is db.sqlite3, and you can find this file in the main project directory after performing migrations.
You can use any SQLite database viewer tool, or alternatively, the following online tool is recommended for visualizing tables.
https://inloop.github.io/sqlite-viewer/
It will surprise you to see there are so many tables along with a table for the Book() model. Don't worry, Django create these tables for Admin app that we were talking at the beginning of the article.
Note: Django create a table for each model and by default the naming is done with app name as namespace and the model name (However, it can be change using META class option, So be mindful how your tables naming is done in the project.). So, the table for Book model will be named as 'book_books'.
You can connect with db.sqlite3 database using following python lines:
conn = sqlite3.connect('db.sqlite3')
Now, you can load your json data using following line:
with open('data.json', 'r') as json_file:
data = json.load(json_file)
Now dumping data into the model is similar to a SQL insert operation, so you can loop through your data object and make insert into the book_books table as follows:
for item in data:
title = item["title"]
author = item["author"]
price = item["price"]
conn.execute("INSERT INTO 'book_books' (title, author, price) VALUES (?, ?, ?)",
(title, author,price ))
It is important to perform commit and close the connection after a database operation.
conn.commit()
conn.close()
You can see the data in sqlite table as below:
**
(Note: if you are wondering how the id field came, then you must know , Django add the id field with each model if the default is not overwritten.) **
Now, if you try the following endpoints, you will details:
http://127.0.0.1:8000/api/books/1
This will insert all the data into the table and now those data will be available via the endpoints. It is good to run a migration to make things good after a database operation.
You can run the Django server and test the api endpoints via browser or using any tools like postman or curl etc.
You can find the complete code via my github repo
All questions, and queries are welcome in the comment.
Top comments (2)
@ajitkumar Such a useful post, thanks for sharing! As a FSD i encounter this situation regularly, where i need to add any test data to my DB to test my endpoints, you made it look smooth and simple👍🏼.
Thank you. I hope it will help.