nbdev formating and linting

• Introduction
• nbqa
• black
• isort and mypy
• Automating your flows with Makefile
• pre-commit
  • install and configure pre-commit
  • create .pre-commit-config.yaml
• reproducible
  environments with Pipenv/Poetry/virtualenv
• Concusion

Introduction

nbdev is a great tool.

However some of you may not be comfortable with the default fastai
codestyle, or maybe your team is
already using a different one.

This post will show you how to integrate your nbdev workflows with
industry standard tools like:

• black , The uncompromising code formatter
• isort , A Python utility / library to sort imports.
• mypy , Optional static typing for Python
• pre-commit , A framework for managing and maintaining multi-language pre-commit hooks.

Note

An astute reader may notice that one can enable
black_formatting = True in settings.ini

However,

• this will only format the code in the module, not the code in the
  notebook.

• this does not allow you to use other tools like isort or mypy

nbqa

use nbqa to format and verify
code in your notebooks

nbqa is a tool that allows you to run any tool (formatter/linter) on
your notebooks, without having to worry about the details.

black

Black is a python code formatter that is becoming the de-facto standard

pip install nbqa

nbqa black nbs/

isort and mypy

isort will sort your imports, and mypy will check your type annotations

pip install isort mypy

nbqa isort nbs/
nbqa mypy nbs/

Automating your flows with Makefile

But you are already using nbdev command-line arguments like
nbdev_export and nbdev_test, who wants to remember more commands?

Make is a great tool for
automating your workflows, usually comes pre-installed on most unix
systems.

Create a Makefile in the root of your project and add the following
lines:

Makefile

# Makefile
format:
    nbqa black nbs/
    nbqa isort nbs/
    nbqa mypy nbs/

Now you can run make format to format your notebooks.

Let’s add more commands to our Makefile:

Makefile

format:
    nbqa black nbs/
    nbqa isort nbs/
    nbqa mypy nbs/

export:
    nbdev_export

mypy:
    nbqa mypy nbs/ --ignore-missing-imports --check-untyped-defs

test:
    nbdev_test --n_workers 4

prepare:
    # Export, test, and clean notebooks, and render README if needed
    nbdev_prepare

docs:
    nbdev_docs

And now the cherry on top: Makefile supports dependecies, so we can
chain commands together.

add this to your Makefile:

Makefile

all: format prepare mypy

Running make all will format your notebooks, export them, run mypy,
run tests and update README.md

pre-commit

Want to automate your formatting even more?

Force your team to format their notebooks before committing?

Use pre-commit

install and configure pre-commit

pip install pre-commit # install 

pre-commit install # enable for this repo

create .pre-commit-config.yaml

.pre-commit-config.yaml

repos:
  - repo: https://github.com/fastai/nbdev
    rev: 2.2.10
    hooks:
      - id: nbdev_clean
      - id: nbdev_export

Add nbqa hooks

.pre-commit-config.yaml

  - repo: https://github.com/nbQA-dev/nbQA
    rev: 1.5.3
    hooks:
      - id: nbqa-mypy
        args: ["--ignore-missing-imports", "--check-untyped-defs"]
        additional_dependencies: ["mypy", "types-toml", "types-requests"]
      - id: nbqa-black
      - id: nbqa-isort

control your hooks behaviour

.pre-commit-config.yaml

  - repo: https://github.com/nbQA-dev/nbQA
    rev: 1.5.3
    hooks:
      - id: nbqa-mypy
        args: ["--ignore-missing-imports", "--check-untyped-defs"] # pass args to mypy
        exclude: "directory_to_exclude/.*" # eclude any directory/files from formatting
        additional_dependencies: ["mypy", "types-toml", "types-requests"] # install additional dependencies
      - id: nbqa-black
        exclude: "directory_to_exclude/.*" # eclude any directory/files from formatting
      - id: nbqa-isort
        exclude: "directory_to_exclude/.*" # eclude any directory/files from formatting

reproducible environments with Pipenv/Poetry/virtualenv

Sometimes you may run into reproducibility issues, where your notebook
works on your machine, but not on your team’s.

The best way to avoid it is to use a dependency manager, such as:

• Pipenv
• Poetry
• virtualenv

But how do you integrate it with nbdev, since it uses settings.ini to
manage dependencies?

In this blog post I show an example of
how to use Pipenv with nbdev.

Concusion

I hope you find this post useful.

These changes may make nbdev more friendly to a team environment.

Please don’t hesitate to post any questions or comments below.