How to Add Subscription Based Throttling to a Django API

Python was a natural choice when I started SocialSentiment.io. It let me use the same language for both the machine learning algorithms and web development. And I had used Django previously for other projects. The Django Rest Framework (DRF) is a great package to quickly and easily extend a Django project to offer APIs. Today we'll look at how to extend its capabilities to support custom throttling based on user subscriptions.

Subscription Model

First let's define our application's subscription model and throttling requirements:

• A free tier allowing a few hundred API requests per day
• A low cost paid tier offering a few thousand requests per day
• A higher cost tier offering unlimited requests
• All tiers limited to 5 requests per second

This is a very common use case for a modern SaaS application.

Custom Throttling Class

One great thing about Django Rest Framework is it includes many built-in options for authentication and throttling. Each can be applied globally or to specific endpoints. If you desire any type of dynamic throttling options you'll need to extend it. Fortunately the architecture of DRF lets you override just about any part of it.

Let's start by writing a custom class that overrides DRF's UserRateThrottle:

from rest_framework.throttling import UserRateThrottle

class SubscriptionRateThrottle(UserRateThrottle):
    # Define a custom scope name to be referenced by DRF in settings.py
    scope = "subscription"

    def __init__(self):
        super().__init__()

    def allow_request(self, request, view):
        """
        Override rest_framework.throttling.SimpleRateThrottle.allow_request

        Check to see if the request should be throttled.

        On success calls `throttle_success`.
        On failure calls `throttle_failure`.
        """
        if request.user.is_staff:
            # No throttling
            return True

        if request.user.is_authenticated:
            user_daily_limit = get_user_daily_limit(request.user)
            if user_daily_limit:
                # Override the default from settings.py
                self.duration = 86400
                self.num_requests = user_daily_limit
            else:
                # No limit == unlimited plan
                return True

        # Original logic from the parent method...

        if self.rate is None:
            return True

        self.key = self.get_cache_key(request, view)
        if self.key is None:
            return True

        self.history = self.cache.get(self.key, [])
        self.now = self.timer()

        # Drop any requests from the history which have now passed the
        # throttle duration
        while self.history and self.history[-1] <= self.now - self.duration:
            self.history.pop()
        if len(self.history) >= self.num_requests:
            return self.throttle_failure()
        return self.throttle_success()

What we're doing is dynamically looking up the user-specific throttle at the key moment to override the default DRF picks up from your settings file. Define a method get_user_daily_limit to look up the value. I highly recommend using Django's cache methods if this is stored in a database for performance.

Settings

Next let's see what's required in settings.py:

REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': [...],
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.IsAuthenticated'
    ],
    'DEFAULT_THROTTLE_CLASSES': [
        'rest_framework.throttling.UserRateThrottle',
        'app.throttling.SubscriptionDailyRateThrottle'
    ],
    'DEFAULT_THROTTLE_RATES': {
        'user': '5/second',
        'subscription': '200/day'
    }
}

Here we set up two types of throttling. The built-in UserRateThrottle will handle the global 5 requests per second limit. It finds that setting in DEFAULT_THROTTLE_RATES with key user. Our custom throttle class is also enabled and defaults to the subscription value if a user subscription isn't found. Of course the application should be written so this never happens, but it's good to have a fallback plan if a user isn't configured properly.

Subscriptions

How you code and model your subscriptions is up to you. In my case I wrote static classes that define the details of each subscription tier. A Subscription model links the user to a specific plan with details such as start time, payment details, etc.

The nice thing is Django and DRF don't dictate how you design your user subscriptions. Any way you choose to model it they'll handle because you can customize every aspect of authorization and throttling.

Conclusion

So far I only have good things to say about the flexibility of Django and DRF and the customizations they allow. They took the right approach in offering a wide variety of built-in capabilities while allowing developers the opportunity to easily extend or override them. It's been working great for SocialSentiment.io and our APIs. I'd like to hear how others have added their own features to Django Rest Framework in the comments below.

Comments

[Injemam]

Hi Matthew !
Just wanted to thank you for this amazing post. I'm working on a saas project and also a beginner in drf. My project is almost completed and I'm stucked in implementing rate limit. Can you please make a tutorial post how to integrate this in an api project.
I did searched the whole internet in past 4 days and in this weekend i wanted to implement this functionality. Lots of other people also need this tutorial because i have seen many questions related to this but not a answers for them.
The only thing i found is your post.
I read it 5-6 times and tried to implement but failed.
Please make a detailed post.
Thankyou

[alimp5]

Tnx you man :X
but you could write a step by step guide for beginners.
again thanks you.