DEV Community: Dinesh S

Build side projects instead of tutorials

Dinesh S — Mon, 28 Nov 2022 08:54:52 +0000

Why building side projects is a better way to learn programming than following tutorials.

You want to get into programming and you started by following tutorials. They are fine in the initial days, when you are just starting out. But after you get comfortable, you have to start building projects to see more benefits.

Here are a few reasons why building side projects is better

When you build projects, you learn by doing and you gain practical experience that can't be gained by following tutorials.
Building projects also forces you to think creatively and solve problems on your own, which is an essential skill for any programmer.
By contrast, tutorials can often be too simplistic or too specific, and they don't always give you a chance to practice your skills in a real-world context.
Projects also tend to be more engaging and enjoyable than tutorials, so you'll be more motivated to stick with them (and learning programming) in the long run.
Building projects, leaves room for self exploration and fosters creativity.

Benefits of side projects.

Working on side projects can help you learn new programming concepts and sharpen your skills.
Completing a side project can give you a sense of accomplishment and boost your confidence.
Working on side projects can help you explore different programming languages and tools.
Side projects can also be a great way to land your dream job or get freelance clients.
You have something to show in a job interview.

How to get started with building your own projects.

Start small. You can build simple projects at first.
Find inspiration for projects from things you're interested in or want to learn more about.
Try to automate a manual workflow that you do everyday.
Use online resources and Google searches to help you figure out how to build what you want.
Build a tool that you want for yourself if it doesn't exist already.

Are you still stuck with tutorials? Now is the time to go build something. Do you need help, feel FREE to DM me on Twitter.

10 Advanced Git workflows that will come in handy

Dinesh S — Mon, 07 Nov 2022 09:13:38 +0000

We all know the common Git commands which we use everyday, like commit, push, pull etc.

Here are 10 less known commands which can be very useful.

1⃣ Undo a git rebase

You did a rebase & now you want to undo it. Do git reflog and find the point where you want to go. (Ex: HEAD@{10}) You will find something like the below.

git reflog
<sha> HEAD@{9}: rebase: action
<sha> HEAD@{10}: commit: action

HEAD@{9} is the point where rebase was done. So to revert that, we have to do a hard reset to the previous point.

Use the value to do a hard reset.

git reset --hard HEAD@{10}

The rebase you did is undone.

2⃣ Hunt a mysterious bug in your git history

This command uses a binary search algorithm to find which commit in your project’s history introduced a bug. You use it by first choosing a "bad" commit that is known to contain the bug, and a "good" commit that is known to be clean, before the bug was introduced. Git bisect picks a commit between those two points and asks you whether the selected commit is "good" or "bad". It continues narrowing down the range until it finds the exact commit that introduced the change.

git bisect

3⃣ Modify the commit message of a specific commit

git rebase -i head~2

The above command display's interactive rebase for the last 2 commits from head. Choose the number based on where your commit lies from head. If it is the 2th commit, choose head~2. You will get something like below

pick <sha> <commit message>
pick <sha> <commit message>

Change the command to edit instead of pick from the above, corresponding to the commit you want to modify

pick <sha> <commit message>
edit <sha> <commit message>

Now go ahead & save and the rebase will stop at the 2nd commit. You can amend the commit to change the commit message. Do git rebase --continue to finish the rebase.

git commit --amend
git rebase --continue

4⃣ Amend last commit

This comes in very handy if you want to change the commit message of the last commit. It can also be used to add new changes to the previous commit, instead of creating a new commit.

git commit --amend

5⃣ Recover a dropped stash

You have dropped a stash, but now you want it back. If you know the hash of the dropped stash you can do the below to retrieve it.

git stash apply $stash_hash

When you drop a stash, a hash of it is printed on the screen. If you haven't closed your terminal, you can locate it by scrolling to the top of your terminal window. If you can't find it, execute the below command.

gitk --all $( git fsck --no-reflog | awk '/dangling commit/ {print $3}' )

This will open gitk (git gui). To spot stash commits, look for commit messages that look like this.

WIP on somebranch: commithash Some old commit message

Once you know the hash of the commit you want, you can apply it as a stash

git stash apply $stash_hash

6⃣ Copy a file from one branch to another branch

If you want to copy an entire file from one branch into another branch, do the below

git checkout bugfix README.md

bugfix is the source branch where the file README.md is located.

Execute the above command from the branch (destination), where you want the file to be copied.

7⃣ Display the changes in a particular commit or a stash

git show <sha>
git show stash{0}

8⃣ Get the number of commits (Commit count)

To get the count for a specific revision

git rev-list --count <revision>

For example git rev-list --count main will display the commit count in main branch.

To get the count across all branches

git rev-list --all --count

9⃣ Revert a specific commit

Get the of the commit you want to revert and do this.

git revert <sha>

🔟 Rename a local branch

If you want to rename a branch that is different from the current branch you are in, do this

git branch -m <oldname> <newname>

If you want to rename the current branch you are in, you can do

git branch -m <newname>

I built my own Tweet scheduler which costs less than 1$ per month

Dinesh S — Sat, 29 Oct 2022 06:20:11 +0000

I wanted a simple scheduler that will post tweets from my Notion database to Twitter.

I wanted an app without any bells & whistles, just a dumb scheduler. I did not want to pay 10$/mo just for that. So as any developer would do, I decided to write my own scheduler 😂.

Here's how I did it.

I wrote a script tweeter.js which does the following

Fetch content from a Notion database.
Prepare the tweet
Send it to Twitter based on the scheduled time slot
Mark the tweet as posted in Notion.

I have hosted the scheduler in AWS and it runs as a container in ECS.

I use Amazon EventBridge rule to create a new container every 30 minutes. This container when started executes tweeter.js and terminates. This way, I don't have a server running all the time, which in turn results in reduced costs.

It costs less than 1$ a month and I can schedule as many tweets/threads I want.

One caveat is that the tweets schedule time is not accurate to the specific minute. It depends on the frequency at which a new container is created. I have set it to 30 minutes, so if you schedule a tweet between 10:00 & 10:30 all of them would be tweeted at the same time.

References

ECS infrastructure setup - https://github.com/vthub/scheduled-ecs-task

Why do we need coding standards?

Dinesh S — Wed, 29 Jun 2022 14:36:46 +0000

Why do we need coding standards?

Many beginner developers ask me why they should worry about coding standards when their code works as expected.

Here are a few important points. 👇

1. Consistency

Using similar coding styles throughout the codebase is a win for everyone on the team. It reduces the time to think & to decide how to name a variable/class/function or how to structure code. Every developer who codes doesn't have to spend time thinking.

2. Maintainability

Codebases are mostly contributed by a lot of developers and that keeps changing. Maintaining a constant standard throughout makes it easy for anyone to get started and contribute without friction. Having a set standard helps to keep code complexity in check.

3. Clean code

The code looks clean. Imagine opening a codebase just to find code written haphazardly. It would be a nightmare to work on such a codebase. This eventually becomes tech debt.

Coding standards that are enforced, have a cleaner codebase than others.

4. Developer sanity

If every developer follows their own standards, the codebase would be a mess. It would be very frustrating for another developer to fix bugs or complete features started by others, because of different coding styles.

5. Reduced development time

When there is a consistent standard, the code is readable and familiar, which saves a lot of time. Developers can start coding right away.

Reviewing Pull Requests becomes easier as well.

Testing pyramid in software development

Dinesh S — Sun, 23 Jan 2022 10:17:49 +0000

We have all been hearing about the importance of writing tests and test-driven development.

✅ Here's how you can write better tests by implementing the testing pyramid strategy.

A testing pyramid is made up of 3 layers

Unit Tests
Integration Tests
End to End Tests

Testing pyramid tells you what type of tests you should be more focussed on.

Unit tests

A unit test verifies that an individual piece of code functions as expected. They don't have any external dependencies.

Unit tests form the base of the pyramid. Ideally they should constitute bulk of your tests.

They are quite fast to execute, hence devs are encouraged to run it as often as required.

Integration

Next layer on the pyramid are the integration tests.

Integration tests validate how a piece of code functions when it interacts with other code. These tests communicate with external dependencies like database's & external API's.

They are slow compared to unit tests. They also need the external services to be running.

End to End (E2E)

At the top of the pyramid, we have End 2 End tests.

These tests interact with pages just like a user would, for example by clicking buttons and typing into text boxes, and they check that pages respond appropriately from the user's perspective.

They test the functioning of an app in its entirety. They are the slowest to run and given its dependency over multiple things, they are non-deterministic in nature. Ideally, you should be having only a few of these.

They are non-deterministic, the automation software must wait for the browser to complete the page load and for the particular element to appear.

For example, suppose that you write a test that clicks a button to open a modal and then clicks a button inside the modal to close it. Sometimes, the modal will open before the test tries to click the close button, so the test will pass.

Summary

We've looked at the types of tests above.

At a broad level, there are 2 aspects of a test type.

Speed
Test surface

You can see that speed is inversely proportional to the test area. Unit tests cover a short area, hence they are fast to run. But they cannot guarantee the functioning of a feature from the perspective of a user.

On the other hand E2E tests, validate the functioning of an entire feature as an end user would use it, but they are very slow to run and have lot of dependencies.

A good test suite will have a mix of the 3 types of tests outlined above. Unit tests are more reliable and fast, on the other hand E2E tests verify the functioning of an entire app, but are slow and non-deterministic.

How I built SetProgress - Part 4 Webhooks

Dinesh S — Thu, 30 Dec 2021 12:08:41 +0000

In the last few posts I have explained how I built waitlist, progress bar & the automation. In this post, I will talk about how & why I added Webhooks to the app.

Webhook Integration

In the previous post I wrote about how I automated 100DaysOfCode progress. That automation is very limited as it just works only based on HashTags. The idea of adding a Webhooks endpoint was born from there.

Implementing this is quite simple. All I have to do is to expose an api /api/users/external-progress which accepts currentProgress in the request body. Users have to send their currentProgress to this endpoint to update their progress.

The api is pretty simple, but how do we identify the user whose progress is to be updated?

Here is where the unique secret comes in handy. Each user gets a secret which is unique to them. They have to set this key in the header of the endpoint. You might ask why secret and why not user id? User Id can be obtained easily which allows a bad actor to update the progress of another user. We don't want that, hence the secret. If user feels that their secret is compromised, then they can generate a new one from the settings page.

Now that we have an endpoint, users can make use of NoCode tools like Zapier, Integromat, Integrately etc to automate their progress updates.

Use cases

Update MRR
Update your running mileage

How I built SetProgress - Part 3 Automation

Dinesh S — Sat, 25 Dec 2021 03:18:32 +0000

In the previous post, I explained how I built SetProgress.

In this post, I am going to explain how I automated progress updates.

Automation was not part of my initial plan. I just wanted to create a simple app which allows a user to design & push a progress bar to Twitter.

Sumudu Siriwardana asked me if the app could calculate & update progress for her #100DaysOfCode challenge.

I thought about it, and I realized that the app doesn't offer much automation. User has to update their progress manually everyday or every week.

Automating #100DaysOfCode

Let me give you a bit of context on 100DaysOfCode.

It has 2 rules

Code minimum an hour every day for the next 100 days.
Tweet your progress every day with the #100DaysOfCode hashtag.

Anyone who is doing this challenge will tweet everyday tagging 100DaysOfCode and a little bit about what they did that day.

Solution

We can fetch all the tweets of a user and check if any one of those have the hashtag 100DaysOfCode.

I used the Twitter search tweets API to get the tweets of a user containing a specific hashtag and in a given date range.

This is how the request looks.

https://api.twitter.com/1.1/search/tweets.json?q=from%3Asdinesh91%20since%3A2021-12-24%20until%3A2021-12-25%20#100DaysOfCode%20-filter%3Aretweets

These are the query parameters I used

from:sdinesh91 since:2021-12-24 until:2021-12-25 #100DaysOfCode -filter:retweets

This query returns tweets of a specific user, tweeted within a specific date range and having the specified hashtag. We also don't need to fetch the retweets, so I filter them out.

We don't have to count multiple tweets containing the hastag in a single day. 100DaysOfCode is progress made in days and not in the number of times tweeted.

Having fetched the tweets it is very simple to count the progress. If I find a tweet for any given day, I increment the count. Once I have the count, I update the progress in the database and push the new progress bar to Twitter.

One important aspect of writing a backend job like this, is to make it Idempotent. Being idempotent, ensures that we obtain the same result irrespective of the number of times we run the code. This is critical as backend jobs might fail or timeout. In such cases, we should be able to just rerun the job and things should work as expected.

To make tracker job idempotent, I store the tracker's last run date to each user's config. This allows the tracker to resume processing from where it last left by comparing the last run date.

Tech

I have hosted the tracker on Netlify as a function. I also use EasyCron to trigger the tracker function every day.

How I built SetProgress - Part 2 Progress bar

Dinesh S — Fri, 17 Dec 2021 03:06:12 +0000

This is the 2nd post in the series "How I built SetProgress".

Before we dive in, let me explain the features of SetProgress.
SetProgress allows you to display your progress to your Twitter Audience. If you are an active user on Twitter you might have noticed accounts with a progress bar in their profile. The most common one's are MRR (Monthly recurring revenue), #100DaysOfCode and many more.

Progress bar examples

Before I built this app, I saw a lot of people on Twitter sharing their progress using progress bars in their bio. I asked a few of them how they were doing it and they said, they were creating those blocks manually. That works fine if your progress doesn't change frequently. But if you are doing #100DaysOfCode or something similar, progress changes quite frequently. It would also be cumbersome for a user to calculate the number of bars in the progress bar.

Let's say you are doing 100DaysOfCode and your have completed 56 days, what would the progress bar look like? How many blocks do you want to have in the bar and how do you fill them with full/partial blocks? There is a bit of mathematic calculation here. In 2021, nobody should be doing this sort of thing manually.

I decided to build an app to automate this. There are 2 things to be solved here.

Create the progress bar based on the current progress, target progress and the number of bars in the progress bar. (As configured by the user in the app)

Push the progress bar to Twitter.

SetProgress Architecture

To use the app, user has to login using Twitter. User profile information is fetched from Twitter and displayed in the app. Signing in with Twitter also enables the app to push the progress bar to Twitter.

Progress bar design

The first step in designing a progress bar is to configure it. It is made of blocks which display the progress. There are 3 types of blocks

Empty
Partial
Full

The blocks can be made of any emoji. For convenience, I have collected commonly used one's into a dropdown, which can be selected while configuring.

The bar is calculated based on the current and the target progress. Just a bare progress bar doesn't convey much meaning. So you also configure what should be displayed to its left and right.

Save to DB

Users can configure the progress and save it to the database. By doing this, they don't have to configure the progress every time they come to the app. If you think about it, once you do the initial configuration, you would just update the current progress, everything else remains the same.

Push to Twitter

Once the user Sign's In with Twitter, the app receives the user's access token which is required for updating the user profile Twitter doc. Progress bar is nothing but a plain string. When user clicks on Push to Twitter, an API call is made to Twitter to update their profile.

Tech stack

I am using Next.js for backend along with FaunaDB as the datastore.

I am using ReactJS and Tailwindcss for frontend.

The app is hosted on Vercel. It has a mix of Server Side rendered pages and API's. You don't want to expose API keys and database secrets in the client. So I proxy the requests through API's that require Twitter tokens (For pushing bar to Twitter) and database credentials (For storing the progress bar configuration).

7 tips to help you deploy code with confidence

Dinesh S — Fri, 10 Dec 2021 03:44:47 +0000

Writing code is great, but how do you deploy your code with confidence?

Here are 7 tips that will help you with it.

1. Code reviews

Get your code reviewed by your peers. Common patterns and bugs can be spotted in these reviews by senior devs.

2. Deploy small features/chunks of code

Deploying a small feature set reduces the number of errors that might come up. This also restricts issues to the feature being deployed, which makes it easy to debug and test.

3. Keep an eye out for database migration changes

You need to be careful when deploying database migrations. Identify if they are destructive or not and employ migration deployment strategies accordingly. Deployment with a migration change makes it challenging to do rollbacks.

4. Write unit tests & integrate them with CI system

Having unit tests validates the requirement and makes sure any new code changes don't break existing functionality. Make sure your CI system runs these on all PR's to validate that the new changes don't break the existing code.

5. Testing in test/gamma environment

You can have automated integration tests to verify if the code works when integrated with other services. Also manually testing in Gamma/Staging env is always better as these environments mimic production.

6. Have a rollback strategy ready

Before you deploy your code, have a rollback plan ready. Let's say there is a bug after deployment, how do you roll back to the previous version? There won't be time to think when production is broken. This will save you from panic & distress.

7. Have a good error-monitoring system in place.

Don't wait for users to notify you of any issues in your application. Have an error-monitoring system which allows you to respond to issues promptly & proactively.

How I built SetProgress - Part 1 Waitlist

Dinesh S — Wed, 08 Dec 2021 11:42:00 +0000

This is going to be a multi part series. In this post, I am going to explain how I created a waitlist page for my app SetProgress.

For a simple waitlist, you require only 2 things, a landing page and a way to collect & store emails or phone numbers.

Waitlist Architecture

Landing page

For the landing page, I used Unicorn platform. I chose an existing template and made some minor tweaks to it. I did not want to spend time building a page myself since waitlist is a temporary thing and I could spend that time on building the app itself.

I took their Maker pricing plan, so that I can export the code and host it myself. I did not want to depend on another platform for hosting and keep paying them just for that.

Collecting emails

I wrote a few lines of JavaScript code which will make a call to an API which will have the email the user entered in the waitlist input box. On success, I show the user an alert message with the text Added to waitlist successfully. I just used javascript alert(success) for it, no fancy pop up's.

Storing waitlist emails

I decided to store the emails in Airtable. I created an automation in Integromat which will receive the emails through a webhook and write them to Airtable. The automation is a Webhook to Airtable integration. The ajax call from the waitlist form is made to the Webhook API url provided by Integromat.

Integromat integration

Creating the logo

I created a simple logo from Figma.

App logo

I created it using emoji's 😄

Tools used for the waitlist

Unicorn platform, for building the landing page.
Figma, for creating the logo.
Storyset, for illustration. I used Fast-loading illustration.
Airtable, to store the emails
Integromat, to create a webhook and write data to Airtable
Vercel to deploy the waitlist.

I used the free tier of all the above tools, except Unicorn platform for which I did one time payment of 8$ for a month, just to export the code.

Summary

As you can see, a waitlist doesn't have to be complicated. It can be done in a few simple steps. I don't recommend anyone to spend a lot of time on the waitlist. Focus your time & energy on the actual app itself.

Here's how the waitlist page looks.

Build an app using Next.js and Twitter API

Dinesh S — Fri, 03 Dec 2021 06:46:16 +0000

In this post I am going to walk you through the Next.js Twitter starter template I have created. Using this guide, you can build your own app using Next.js and Twitter API.

Let's get started.

Clone the starter kit

Dineshs91 / nextjs-twitter-starter

A starter kit for building apps using Twitter API with Next.js

This is a Next.js Twitter starter kit.

Packages

Getting Started

Create .env.local file in the project root and add the following content in it

TWITTER_CONSUMER_KEY=
TWITTER_CONSUMER_KEY_SECRET=
TWITTER_BEARER_TOKEN=
TEST_TWITTER_HANDLE=

To get the Twitter keys, visit https://developer.twitter.com/en/portal/dashboard and create a standalone app. Fetch the consumer key, secret and bearer token and add it to the .env.local file.

Add your twitter handle for TEST_TWITTER_HANDLE. This is used in the twitter sample page.

App vs. User authentication

Twitter has two different authentication options:

App: higher rate limits. Great for building your own Twitter App.
User: lower rate limits. Great for making requests on behalf of a User.

User authentication requires:

consumer_key
consumer_secret
access_token_key
access_token_secret

App authentication requires:

bearer_token

For the example in this starter, we use App authentication which makes use of the bearer_token

Install dependencies

yarn install

Run the development server:

yarn dev

Open http://localhost:3000 with your browser to…

View on GitHub

First, clone the repository to your local computer.

git clone https://github.com/Dineshs91/nextjs-twitter-starter.git

This kit contains the following packages for you to get started.

Next
React
Twitter-lite
Tailwindcss

Install dependencies

cd into the newly cloned repository and run the following command.

yarn install

Note: If you don't have yarn installed, install it by following the instructions provided here https://classic.yarnpkg.com/lang/en/docs/install

Set up local configuration

Create a file to store the local environment variables.

Create a file in the root directory with the name .env.local and add the following in it.

TWITTER_CONSUMER_KEY=
TWITTER_CONSUMER_KEY_SECRET=
TWITTER_BEARER_TOKEN=
TEST_TWITTER_HANDLE=

You can get the values of TWITTER_CONSUMER_KEY, TWITTER_CONSUMER_KEY_SECRET & TWITTER_BEARER_TOKEN from Twitter developer portal.

Navigate to https://developer.twitter.com/en/portal/dashboard and create a new standalone app. Once you create the app, you will get the required values which you can add in the .env.local file.

TEST_TWITTER_HANDLE can be any Twitter handle. This key is used in the demo page to fetch their public profile information.

Local development

To start your local development server, run the following command

yarn dev

Once the server starts, navigate to http://localhost:3000/.

Checkout the demo page at http://localhost:3000/twitter. The information you see there, is pulled from Twitter using their official API.

Develop

The starter kit contains one example for both SSR and API.

It has one server side rendered (SSR) page and one api api/twitter-user.

SSR page pages/index.js
API pages/api/twitter-user.js

You can start editing the page by modifying pages/index.js. The page auto-updates as you edit the file.

Test the API from postman, by sending a post request with a request body in the below format.

{
    "twitter_handle": "SDinesh91"
}

The response will be

{
    "screen_name": "SDinesh91",
    "name": "Dinesh",
    "profile_image_url": "https://pbs.twimg.com/profile_images/1421346630456922112/fVyiui9f_normal.jpg"
}

Make sure the header Content-Type is set to application/json

Now, go ahead and try it out with your own twitter handle.

The pages/api directory is mapped to /api/*. Files in this directory are treated as API routes instead of React pages.

Checkout the example located at pages/twitter.js. You can navigate to http://localhost:3000/twitter in your browser to see it.

Hacking

That's it. You are all set to build your own app using Twitter API. Share your app with me on Twitter.

Deployment

The easiest way to deploy your app is to use the Vercel Platform from the creators of Next.js.

Check out Next.js deployment documentation for more details.

Authentication

If you would like to make requests on behalf of another user, you will need to generate a separate set of Access Tokens for that user using the 3-legged OAuth flow, and pass that user's tokens with your OAuth 1.0a User Context requests.

Check out next-auth.js for OAuth. This package is not included in the starter, you will have to add it manually.

Django Soft delete options

Dinesh S — Mon, 29 Nov 2021 09:03:03 +0000

In this post, I try and explore different ways of doing soft delete's in Django, either using a library or from the models directly.

The reason I am trying out all these options, is to make sure that the consequences of any framework or approach, are well known before making a choice. And accidental deletion of production data is no joke.

Approaches:

Paranoia model
Django safe delete https://github.com/makinacorpus/django-safedelete

For all the approaches, we will verify how the following cases are handled.

GET
DELETE
Queryset GET and DELETE
Relations

Paranoia model:

I found this code in sentry
The idea here is to create a custom model manager, which includes a custom queryset.
We create ParanoiaModel, which will serve as the base model.

class ParanoidModel(models.Model):
    class Meta:
        abstract = True

    deleted_on = models.DateTimeField(null=True, blank=True)

    def delete(self):
        self.deleted_on=timezone.now()
        self.save()

Any model which needs safe delete, can inherit this base model. This works well, only when an individual object has to be deleted. But would fail, when delete is issued on a queryset.

So we add a custom queryset ParanoidQuerySet.

class ParanoidQuerySet(QuerySet):
    """
    Prevents objects from being hard-deleted. Instead, sets the
    ``date_deleted``, effectively soft-deleting the object.
    """

    def delete(self):
        for obj in self:
            obj.deleted_on=timezone.now()
            obj.save()

class ParanoidManager(models.Manager):
    """
    Only exposes objects that have NOT been soft-deleted.
    """

    def get_queryset(self):
        return ParanoidQuerySet(self.model, using=self._db).filter(
            deleted_on__isnull=True)

class ParanoidModel(models.Model):
    class Meta:
        abstract = True

    deleted_on = models.DateTimeField(null=True, blank=True)
    objects = ParanoidManager()
    original_objects = models.Manager()

    def delete(self):
        self.deleted_on=timezone.now()
        self.save()

We also add a custom manager. It helps us in 2 ways. We can access the original manager, which will return the soft deleted objects and second, queries that return queryset will filter the soft deleted objects without the need for us to specify it in each query.

Now delete's work on both individual objects and querysets.

class Post(ParanoidModel):
    title = models.CharField(max_length=100)
    content = models.TextField()

post = Post(title="soft delete strategies", content="Trying out various soft delete strategies")
post.delete()
# Will soft delete the post

Post.objects.all().delete()
# Will also soft delete all the posts.

Post.objects.get()
# Will not return any post and will raise an exception.

Post.original_objects.get()
# Will return the soft deleted post.

Post.original_objects.all()
# Returns soft deleted objects as well, along with the undeleted ones.

This strategy works very well for the first 3 criterion. But how does this work across relations ?

Let's add another model to the above example, to understand how delete's work across relations.

post = Post(title="soft delete strategies", content="Trying out various soft delete strategies")

class Comment(ParanoidModel):
    post = models.ForeignKey(Post, on_delete=models.CASCADE, related_name='comments')
    message = models.TextField()

comment = Comment(post, message="Well written blog post")
# post is the object we created earlier.

post.delete()
# Soft delete the post.

print(Comment.objects.count())
# The comment of the post still exists.

From the above example, it is clear that the soft delete is not propagated to the relations. Deleting a post, does not delete the comments related to it. They still can be accessed independently, but cannot be accessed from the post, since the post is soft deleted.

So summarising this approach, everything works well, other than the relations handling. This implementation is good enough, if the relation models are not queried directly. For example, once we delete the post, the comments related to the post become irrelevant. Comments don't mean a thing without their parent post.

The other thing to note here is, if we decide to restore a soft deleted object, we don't have to worry about its relations, since they have not been deleted (Neither soft/hard).
If you want restore option, you can add undelete method to the base model ParanoiaModel.

class ParanoidModel(models.Model):
    class Meta:
        abstract = True

    deleted_on = models.DateTimeField(null=True, blank=True)
    objects = ParanoidManager()
    original_objects = models.Manager()

    def delete(self):
        self.deleted_on=timezone.now()
        self.save()

    def undelete(self):
        self.deleted_on=None
        self.save()

You can also add this to the custom queryset.

class ParanoidQuerySet(QuerySet):
    """
    Prevents objects from being hard-deleted. Instead, sets the
    ``date_deleted``, effectively soft-deleting the object.
    """

    def delete(self):
        for obj in self:
            obj.deleted_on=timezone.now()
            obj.save()

    def undelete(self):
        for obj in self:
            obj.deleted_on=None
            obj.save()

Note: I've made some changes to the code found from sentry, like changing the field name deleted_on

Django safe delete

This framework provides lot of options for soft deleting. They have the following policies

HARD_DELETE
SOFT_DELETE
SOFT_DELETE_CASCADE
HARD_DELETE_NOCASCADE
NO_DELETE

Policies apply to how the delete is handled and stored in the database.

They have the following visibility options

DELETED_INVISIBLE (Default)
DELETED_VISIBLE_BY_FIELD

Visibility options apply for retrieving data.

I will focus only on the soft delete policies, as other options are irrelevant for this post. You can check out their documentation if you are interested

HARD_DELETE

This is similar to the Django default behaviour, with some more options. I am not going to discuss them here. You can checkout their documentation.

SOFT_DELETE

This policy just soft deletes the object being deleted. The related objects remain untouched.

Let's start by creating some models

from django.db import models
from safedelete.models import SafeDeleteModel
from safedelete.models import SOFT_DELETE


class Article(SafeDeleteModel):
    _safedelete_policy = SOFT_DELETE
    title = models.CharField(max_length=100)
    content = models.TextField()
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

class Comment(SafeDeleteModel):
    _safedelete_policy = SOFT_DELETE
    article = models.ForeignKey(Article, on_delete=models.CASCADE, related_name='article_comments')
    text = models.TextField()

Now we will try deleting an article

# First we create an article
article = Article.objects.create(
    title="article 1 title",
    content="article 1 content"
)
article.delete()
# Will soft delete the article.

Article.objects.all().delete()
# Will soft delete all the articles.

Article.objects.all()
# Will return objects which are not deleted (Either soft/hard)

Article.objects.all_with_deleted()
# Will fetch all the objects including the deleted one's

Article.original_objects.all()
# Will fetch all the objects including the deleted one's using our custom manager.

We can restore the soft deleted object with the following code.

article.undelete()

In this approach, first 3 criterion work well, but fails for relations. So soft deleting an object, doesn't soft delete its relations.

This strategy is almost similar to the paranoia design we discussed above.

SOFT_DELETE_CASCADE

This is almost similar to the above, except that it soft delete's the related objects as well.

Let's start by creating some models

from django.db import models
from safedelete.models import SafeDeleteModel
from safedelete.models import SOFT_DELETE_CASCADE


class User(SafeDeleteModel):
    _safedelete_policy = SOFT_DELETE_CASCADE
    full_name = models.CharField(max_length=100)
    email = models.CharField(max_length=100)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

class UserLogin(SafeDeleteModel):
    _safedelete_policy = SOFT_DELETE_CASCADE
    user = models.ForeignKey(User, on_delete=models.CASCADE, related_name="user_logins")
    login_time = models.DateTimeField(auto_now=True)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

Now we will try deleting the user

user = User.objects.create(
    full_name="sam kin",
    email="sam@gm.com"
)
UserLogin.objects.create(
    user=user
)
UserLogin.objects.create(
    user=user
)

user.delete()

User.objects.count()
# User count will be 0

UserLogin.objects.count()
# UserLogin count will also be 0. (Since this is cascade soft delete)
# Both user and user login are soft deleted.

We can see that soft delete's are propagated to the relations as well.

Here, restoring user object will restore all of its login's. So all the related objects are restored.

user.undelete()

This approach handles all our criterions.

NO_DELETE

This policy prevents any sort of delete soft/hard. The only way to delete, is through raw sql query. This can be useful in places, where any kind of delete is not allowed, from the application.

Summary

All the approaches fall under 2 categories

Supports relations
Doesn't support relations

If you want your soft-delete's, to be propagated to the relations use soft-delete-cascade. If this is not required, then you can choose any of the above approaches.

You can find the code samples with tests in my Github repo soft-delete-options-in-django