Django Import Export From Basic To Advance

Areeba Seher — Thu, 13 Oct 2022 13:14:54 +0000

Table of Content

Installation and configuration
Models
Resources
Exporting Data
Exporting Data as JSON
Exporting Data as CSV
Data Filtering + Exporting Data
Dealing with the model relationship
Foreign Key Relationship
Method 1: Using ForeignKeyWidget.
Method 2: Using fields
Many To Many Key Relationships
Method 1: By data manipulation on export
Method 2: By adding ManyToManyWidget
Advanced data manipulation on export.
Adding new columns
Changing headers before exporting
Export order
Admin Integration

django-import-export is a library used to handle importing and exporting of data. This package supports multiple formats.

csv
xls
xlsx
tsv
ods
json
yml
html This Library also has a Django admin integration.

For this article, I am using django==4.0 and django-import-export==2.9.0

Installation and configuration

pip install django-import-export

Update your settings.py file.

INSTALLED_APPS = [
    .....
    'import_export',
]

Models.

Now add our models. Here we have 3 simple models. I am using Foreign Key and Many to Many relations to cover these in this article.

# models.py
from django.db import models


class Author(models.Model):
    name = models.CharField(max_length=100)

    def __str__(self):
        return self.name


class Category(models.Model):
    name = models.CharField(max_length=100)

    def __str__(self):
        return self.name


class Book(models.Model):
    name = models.CharField('Book name', max_length=100)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)
    published = models.DateField('Published')
    categories = models.ManyToManyField(Category)

    def __str__(self):
        return self.name

Now run migrations.

Resources

Create a new file resources.py in your app. In this file, we will add the Model Resource classes which are similar to how Django handles forms.py.

# resources.py
from import_export import resources
from .models import Author, Category, Book


class BookResource(resources.ModelResource):
    class Meta:
        model = Book

This is the simplest configuration. We can add more, like fields, and exclude in Meta just like forms.py configuration. We will discuss it below.

Exporting data

Run python manage.py shell. It will open the shell.

Exporting Data as JSON.

from app.resources import BookResource

book_resource = BookResource()
data = book_resource.export()
data.json

Output:

[
    {"id": 1, "name": "Django for APIs", "author": 1, "published": "2022-09-16", "categories": "2"},
    {"id": 2, "name": "Beginning Django E-Commerce", "author": 2, "published": "2022-09-16", "categories": "1,3"},
    {"id": 3, "name": "Django Unleashed", "author": 3, "published": "2022-09-16", "categories": "1,2"},
    {"id": 4, "name": "Django Design Patterns and Best Practices", "author": 4, "published": "2022-09-16", "categories": "1,4"}
]

Exporting Data as CSV.

from app.resources import BookResource

book_resource = BookResource()
data = book_resource.export()
data.csv

Output:

id,name,author,published,categories
1,Django for APIs,1,2022-09-16,2
2,Beginning Django E-Commerce,2,2022-09-16,"1,3"
3,Django Unleashed,3,2022-09-16,"1,2"
4,Django Design Patterns and Best Practices,4,2022-09-16,"1,4"

Data Filtering + Exporting Data.

We can also export the filtered data.

from app.models import Book
from app.resources import BookResource


query = Book.objects.filter(name="Django for APIs")
book_resource = BookResource()
data = book_resource.export(query)
data.json

Output:

[
    {"id": 1, "name": "Django for APIs", "author": 1, "published": "2022-09-16", "categories": "2"}
]

Views and URLs.

# views.py
from django.http import HttpResponse
from .resources import BookResource


def export_csv(request):
    book_resource = BookResource()
    data = book_resource.export()
    response = HttpResponse(data.csv, content_type='text/csv')
    response['Content-Disposition'] = 'attachment; filename="books.csv"'
    return response

# urls.py

from django.urls import path
from .views import export_csv

urlpatterns = [
    path('export-csv', export_csv, name='export_csv')
]

Go to this URL:

http://127.0.0.1:8000/export-csv (It will download the books.csv file)

Note:
For json export use content_type='application/json' and filename=books.json

For excel (xslx) export use content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' and filename=books.xlsx

You can check more content_type from here:

Dealing with the model relationship.

When defining ModelResource fields it is possible to follow model relationships:

1) Foreign Key Relationship.

In the above CSV file picture, you can see that in the author column we have author ids like 1,2,3,4, not the author name.

1. Method 1: Using ForeignKeyWidget.

class import_export.widgets.ForeignKeyWidget(model, field='pk', *args, **kwargs)

fields = 'pk' is the default lookup criterion for the lookup field. But we can customize it. As in our example, we are using field = "name" as a lookup field.

I prefer this method because it's a widget so you can use this widget in any resource class.

# resources.py

from import_export import resources
from .models import Author, Category, Book
from import_export.fields import Field
from import_export.widgets import ForeignKeyWidget

class BookResource(resources.ModelResource):

    author = Field(
        column_name='author',
        attribute='author',
        widget=ForeignKeyWidget(model=Author, field='name'))

    class Meta:
        model = Book

Output

name,published,categories,author
Django for APIs,2022-09-16,2,William S. Vincent
Beginning Django E-Commerce,2022-09-16,"1,3",James McGaw
Django Unleashed,2022-09-16,"1,2",Andrew Pinkham
Django Design Patterns and Best Practices,2022-09-16,"1,4",Arun Ravindran

2. Method 2: Using fields

Adding author__name in fields under Meta will give the author name instead of the author id.

# resources.py

class BookResource(resources.ModelResource):
    class Meta:
        model = Book
        fields = ('name', 'author__name', 'published', 'categories')

Output:

name,published,categories,author__name
Django for APIs,2022-09-16,2,William S. Vincent
Beginning Django E-Commerce,2022-09-16,"1,3",James McGaw
Django Unleashed,2022-09-16,"1,2",Andrew Pinkham
Django Design Patterns and Best Practices,2022-09-16,"1,4",Arun Ravindran

Read more from here. How to export Foreign key relation in django

2) Many To Many Key Relationships.

If we check the output then we observed that the categories column contains ids of categories like 2, "1,3" etc

We have 2 methods to do this.

1) Method 1: By data manipulation on export:

We can manipulate data (any field) before exporting. We will perform some more manipulation in this article.

Not all data can be easily extracted from an object/model attribute. In order to turn complicated data model into a (generally simpler) processed data structure on export, dehydrate_ method should be defined: source: Documentation

# resources.py

class BookResource(resources.ModelResource):

    def dehydrate_categories(self, book):
        data = []
        for i in book.categories.all():
            data.append(i.name)
        joined_string = ",".join(data)
        return str(joined_string)

    class Meta:
        model = Book

Output:

id,name,author,published,categories
1,Django for APIs,1,2022-09-16,Coding
2,Beginning Django E-Commerce,2,2022-09-16,"Programming,ecommerce"
3,Django Unleashed,3,2022-09-16,"Programming,Coding"
4,Django Design Patterns and Best Practices,4,2022-09-16,"Programming,Best practices"

2) Method 2: By adding ManyToManyWidget:

class import_export.widgets.ManyToManyWidget(model, separator=None, field=None, *args, **kwargs)

I prefer this method.

# resources.py

from import_export import resources
from .models import Author, Category, Book
from import_export.widgets import ManyToManyWidget
from import_export.fields import Field


class BookResource(resources.ModelResource):
    categories = Field(
        column_name='categories', attribute='categories',
        widget=ManyToManyWidget(model=Category, separator=',', field='name'))

    class Meta:
        model = Book

Output

categories,id,name,author,published
Coding,1,Django for APIs,1,2022-09-16
"Programming,ecommerce",2,Beginning Django E-Commerce,2,2022-09-16
"Programming,Coding",3,Django Unleashed,3,2022-09-16
"Programming,Best practices",4,Django Design Patterns and Best Practices,4,2022-09-16

Read in detail from here.

Advanced data manipulation on export.

Adding new columns.

Let's take an example. You have 2 column names or fields i.e first name and last name. And you want to add a new column "Full Name" by combining the first and last name in the exported file.
In our case, we are adding 2 new columns full_title and description.

# resources.py

class BookResource(resources.ModelResource):

    full_title = Field(column_name='Full Title')
    description = Field()

    def dehydrate_full_title(self, book):
        book_name = getattr(book, "name", "unknown")
        author_name = getattr(book.author, "name", "unknown")
        return '%s by %s' % (book_name, author_name)

    def dehydrate_description(self, book):
        book_name = getattr(book, "name", "unknown")
        author_name = getattr(book.author, "name", "unknown")
        published = getattr(book, "published", "unknown")
        return "%s is published by %s in %s" % (book_name, author_name, published)


    def dehydrate_categories(self, book):
        data = []
        for i in book.categories.all():
            data.append(i.name)
        joined_string = ",".join(data)
        return str(joined_string)

    class Meta:
        model = Book

Output:

Full Title,description,id,name,author,published,categories
Django for APIs by William S. Vincent,Django for APIs is published by William S. Vincent in 2022-09-16,1,Django for APIs,1,2022-09-16,Coding
Beginning Django E-Commerce by James McGaw,Beginning Django E-Commerce is published by James McGaw in 2022-09-16,2,Beginning Django E-Commerce,2,2022-09-16,"Programming,ecommerce"
Django Unleashed by Andrew Pinkham,Django Unleashed is published by Andrew Pinkham in 2022-09-16,3,Django Unleashed,3,2022-09-16,"Programming,Coding"
Django Design Patterns and Best Practices by Arun Ravindran,Django Design Patterns and Best Practices is published by Arun Ravindran in 2022-09-16,4,Django Design Patterns and Best Practices,4,2022-09-16,"Programming,Best practices"

Changing headers before exporting.

We can also change the headers before exporting our data.

# resources.py

from import_export import resources
from .models import Author, Category, Book
from .widgets import CustomManyToManyWidget
from import_export.fields import Field


class BookResource(resources.ModelResource):

    def get_export_headers(self):
        headers = super().get_export_headers()
        for i, h in enumerate(headers):
            if h == 'author':
                headers[i] = "Author"
            if h == 'name':
                headers[i] = "Book Name"
        return headers

    class Meta:
        model = Book

Output:

id,Book Name,Author,published,categories
1,Django for APIs,1,2022-09-16,2
2,Beginning Django E-Commerce,2,2022-09-16,"1,3"
3,Django Unleashed,3,2022-09-16,"1,2"
4,Django Design Patterns and Best Practices,4,2022-09-16,"1,4"

Export Order.

You can set export_order in the Meta class.

# resources.py

class Meta:
        model = Book
        export_order = (
            'name', 'author', 'full_title', 'published', 'categories', 'description'
        )

Output:

name,author,Full Title,published,categories,description
Django for APIs,William S. Vincent,Django for APIs by William S. Vincent,2022-09-16,Coding,Django for APIs is published by William S. Vincent in 2022-09-16
Beginning Django E-Commerce,James McGaw,Beginning Django E-Commerce by James McGaw,2022-09-16,"Programming,ecommerce",Beginning Django E-Commerce is published by James McGaw in 2022-09-16
Django Unleashed,Andrew Pinkham,Django Unleashed by Andrew Pinkham,2022-09-16,"Programming,Coding",Django Unleashed is published by Andrew Pinkham in 2022-09-16
Django Design Patterns and Best Practices,Arun Ravindran,Django Design Patterns and Best Practices by Arun Ravindran,2022-09-16,"Programming,Best practices",Django Design Patterns and Best Practices is published by Arun Ravindran in 2022-09-16

You can also use exclude in the Meta class. I will recommend you go through the documentation. Because django-import-export documentation is really good and easy to understand.

Admin Integration.

# admin.py

from django.contrib import admin
from import_export.admin import ImportExportModelAdmin
from .models import (Author, Category, Book)
from .resources import BookResource


admin.site.register(Author)
admin.site.register(Category)


class BookAdmin(ImportExportModelAdmin):
    resource_class = BookResource

admin.site.register(Book, BookAdmin)

Then go to the admin panel. You will see the following options in the admin panel under Book Model.

You can select a different format and press submit. Your data will be exported in your required format.

That's it for now. If you like this article then please show some love in the comment section or by supporting the content. You can also suggest more articles in the comment section. I will definitely write your suggested articles.
Thank you.

If you like my post then you can support me by buying me a coffee ☕

Happy Coding.

This artical is originally published here. https://www.letscodemore.com/

DEV Community: Areeba Seher

Django Import Export From Basic To Advance

Table of Content

Installation and configuration

Models.

Resources

Exporting data

Exporting Data as JSON.

Exporting Data as CSV.

Data Filtering + Exporting Data.

Views and URLs.

Dealing with the model relationship.

1) Foreign Key Relationship.

1. Method 1: Using ForeignKeyWidget.

2. Method 2: Using fields

2) Many To Many Key Relationships.

1) Method 1: By data manipulation on export:

Advanced data manipulation on export.

Adding new columns.

Changing headers before exporting.

Export Order.

Admin Integration.

If you like my post then you can support me by buying me a coffee ☕