DEV Community: KagemaNjoroge

How One Line of Code Almost Cost My Client Their OpenAI Credits

KagemaNjoroge — Fri, 25 Jul 2025 15:07:36 +0000

A few months ago, I started offering consulting services as a Django developer. One of my first clients had an urgent issue:

Our OpenAI credits are draining at a crazy rate, and we don’t know why.

They suspected misuse. I dug into their codebase and deployment setup — and one of the first red flags was this:

# settings.py
DEBUG = True

Their production server was running with debug mode enabled.

It didn't take long to find something more worrying...

An Endpoint Was Leaking Their OpenAI API Key

Django's debug mode is helpful during development. But in production, it exposes detailed stack traces and local variable values in the browser when an exception occurs.

In their case, one endpoint was trying to parse JSON from a request body...but when the request was malformed(for example, accessing the endpoint via GET instead of POST, a malformed JSON body etc), it raised a JSONDecodeError. Here is a simplified version of the code:

from django.http import HttpRequest, JsonResponse
from dotenv import load_dotenv
import os
import json

load_dotenv() # loads env variables from a .env file

def index(request: HttpRequest) -> JsonResponse:
    open_ai_key = os.getenv("OPENAI_KEY")
    data = json.loads(request.body.decode("utf-8"))  # Boom on bad requests
    return JsonResponse(data)

With DEBUG=True, that simple mistake triggered Django’s detailed error page, including:

The full stack trace
Request info
Local variables — including open_ai_key 😱

Here’s a screenshot from a proof of concept (POC) I recreated to demonstrate what was happening:

Fix

I immediately advised two things:

Rotate the OpenAI key — assume it’s compromised.
Re-deploy the app with DEBUG = False in settings.py.

DEBUG = False

With DEBUG set to false, Django shows a standard error 500 page without leaking any sensitive data.

I also recommended wrapping JSON parsing in a try/except block to avoid unhandled exceptions:

try:
    data = json.loads(request.body.decode("utf-8"))
except json.JSONDecodeError:
    return JsonResponse({"error": "Invalid JSON"}, status=400)

Key takeaways

Never deploy Django with Django=True
Wrap dangerous logic (like JSON parsing) in proper error handling.

What looked like a billing issue turned out to be a security hole. If you’re building with Django (or any framework), treat debug tools like loaded guns: safe in dev, dangerous in prod.

Happy Django-ing 👋

I met my girlfriend through a GitHub Pull Request

KagemaNjoroge — Tue, 21 Jan 2025 11:37:46 +0000

How cool is that?

3 Books every Python developer should read

KagemaNjoroge — Thu, 20 Jun 2024 11:08:28 +0000

Books are an incredible resource on the journey to becoming a successful programmer. Here are three that have been invaluable to me, and I believe they'll be just as beneficial for you.

1. Clean Code by Robert C. Martin

Despite being a bit old, "Clean Code" is packed with principles that help you write clean, readable, and maintainable code. Ignore the naysayers – this book is a must-read for anyone serious about coding, regardless of the language.

2. The Pragmatic Programmer by Andrew Hunt and David Thomas

This classic is filled with practical advice and tips that will make you a better programmer. It covers topics from debugging and testing to project management and career development. While the first two chapters may seem a bit dry, the rest is pure gold. This book’s insights are especially valuable for Python developers aiming to enhance their coding practices.

3. Code Complete by Steve McConnell

My favorite out of all these books, "Code Complete" dives deep into software construction, covering everything from design to debugging. It is filled with best practices and timeless principles. For Python developers, this book offers a comprehensive guide to writing robust and efficient code.

Happy debugging! 😊

Super-charging Django: Tips & Tricks

KagemaNjoroge — Sun, 16 Jun 2024 14:43:32 +0000

Django encourages rapid development and clean, pragmatic design. However, as your application scales, ensuring optimal performance becomes crucial. In this article, we'll go beyond the basics and delve deeper into advanced techniques and tools to optimize your Django application.

1. Database Optimization

a. Indexing and Query Optimization:

Proper indexing can significantly enhance query performance. Use Django’s db_index and unique constraints appropriately. Analyze and optimize your SQL queries to avoid redundant data retrievals.

class Hikers(models.Model):
    name = models.CharField(max_length=100, db_index=True)
    unique_code = models.CharField(max_length=50, unique=True)

Monitoring tools like pg_stat_statements for PostgreSQL can help identify slow queries and further optimize them.

b. Advanced Query Techniques:

Leverage Django's ORM capabilities with complex queries, such as annotate(), aggregate(), and Subquery to perform calculations directly in the database, minimizing data transfer to your application.

from django.db.models import Count, Subquery, OuterRef

subquery = Hiker.objects.filter(related_model_id=OuterRef('pk')).values('related_model_id')
annotated_queryset = RelatedModel.objects.annotate(count=Count(subquery))

Use the Prefetch object to optimize querying of related objects and avoid the N+1 problem.

from django.db.models import Prefetch

prefetched_hikers = Hiker.objects.prefetch_related(Prefetch('related_model', queryset=RelatedModel.objects.all()))

Always consider performing database operations at the lowest level:

Performing operations in a QuerySet:

# QuerySet operation on the database
# fast, because that's what databases are good at
  hikers.count()

Performing operations using Python:

# counting Python objects
# slower, because it requires a database query anyway, and processing
# of the Python objects
len(hikers)

Performing operations using template filters:

<!--
Django template filter
slower still, because it will have to count them in Python anyway,
and because of template language overheads
-->
 {{ hikers|length }}

c. Database Connection Pooling:

Database connection pooling reduces the cost of opening and closing connections by maintaining a pool of open connections.

# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'mydatabase',
        'USER': 'mydatabaseuser',
        'PASSWORD': 'mypassword',
        'HOST': 'localhost',
        'PORT': '5432',
        'CONN_MAX_AGE': 500,  # Use persistent connections with a timeout
    }
}

Consider using third-party packages like django-postgrespool2 for more advanced pooling features if needed.

2. Caching Strategies

a. Advanced Caching Techniques:

Utilize different caching strategies like per-view caching, template fragment caching, and low-level caching to optimize your Django app. Use Redis or Memcached for fast, in-memory data storage.

# settings.py
CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.redis.RedisCache',
        'LOCATION': 'redis://127.0.0.1:6379/1',
    }
}

# views.py
from django.views.decorators.cache import cache_page

@cache_page(60 * 15)  # Cache view for 15 minutes
def my_view(request):
    ...

Include an example of low-level caching:

from django.core.cache import cache

def my_view(request):
    data = cache.get_or_set('my_data_key', expensive_function, 300)
    return JsonResponse(data)

b. Distributed Caching:

For large-scale applications, implement distributed caching to share cache data across multiple servers, ensuring consistency and scalability.

c. Using Cached Sessions:

For better performance, store session data using Django’s cache system. Ensure you have configured your cache system properly, particularly if using Memcached or Redis.

# settings.py
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'

3. Efficient Middleware Use

a. Custom Middleware Optimization:

Write custom middleware efficiently, avoiding unnecessary processing. Minimize the number of middleware layers to reduce overhead.

class SimpleMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        response = self.get_response(request)
        return response

# settings.py
MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    # Add only necessary middleware
]

b. Asynchronous Middleware:

Django 3.1+ supports asynchronous views and middleware. Use async middleware for I/O-bound tasks to improve throughput.

class AsyncMiddleware:
    async def __call__(self, request, get_response):
        response = await get_response(request)
        return response

4. Static and Media File Optimization

a. Serve Static and Media Files Efficiently:

Use a dedicated service like Amazon S3, or a CDN to serve static and media files, reducing load on your Django application server.

b. Static File Compression and Minification:

Compress and minify your static files (CSS, JS) to reduce load times. Use tools like django-compressor and whitenoise.

# settings.py
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

5. Asynchronous Processing and Task Queues

a. Celery for Background Tasks:

Offload long-running tasks to Celery to keep your application responsive. Configure Celery with Redis or RabbitMQ as the message broker.

# tasks.py
from celery import shared_task

@shared_task
def my_background_task(param):
    # Perform time-consuming task
    return param * 2

An example of setting up periodic tasks with Celery Beat:

# celery.py
from celery import Celery
from celery.schedules import crontab

app = Celery('my_project')

app.conf.beat_schedule = {
    'task-name': {
        'task': 'my_app.tasks.my_periodic_task',
        'schedule': crontab(minute=0, hour=0),  # every day at midnight
    },
}

b. Async Views for High I/O Operations:

Use Django’s async views to handle high I/O operations efficiently.

from django.http import JsonResponse
import asyncio

async def async_view(request):
    await asyncio.sleep(1)  # Simulating a long I/O operation
    data = {'message': 'This is an async view'}
    return JsonResponse(data)

Consider integrating Django with FastAPI for asynchronous endpoints if your application needs high-performance, non-blocking I/O operations.

6. Load Balancing and Scalability

a. Horizontal Scaling:

Distribute your application load across multiple servers using load balancers. Compare different load balancers (e.g., Nginx vs. HAProxy) and their benefits.

b. Kubernetes for Container Orchestration:

Set up Kubernetes to manage and scale your Django application.

# Kubernetes Deployment Example
apiVersion: apps/v1
kind: Deployment
metadata:
  name: django-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: django
  template:
    metadata:
      labels:
        app: django
    spec:
      containers:
      - name: django
        image: my_django_image
        ports:
        - containerPort: 8000

7. Monitoring and Profiling

a. Comprehensive Monitoring:

Implement monitoring tools like Prometheus and Grafana for real-time metrics and alerting. Set up alerts and dashboards to monitor application health proactively.

b. In-depth Profiling:

Use profiling tools like Django Debug Toolbar, Silk, and Py-Spy to analyze performance bottlenecks and optimize your code.

# Profiling with Py-Spy
py-spy top --pid <django-process-pid>

8. Optimizing Template Performance

a. Cached Template Loader:

Use Django’s cached template loader to cache compiled templates, reducing rendering time.

# settings.py
TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'OPTIONS': {
            'loaders': [
                ('django.template.loaders.cached.Loader', [
                    'django.template.loaders.filesystem.Loader',
                    'django.template.loaders.app_directories.Loader',
                ]),
            ],
        },
    },
]

b. Efficient Template Design:

Use Django’s select_related and prefetch_related in views to optimize data retrieval for templates. Avoid unnecessary template tags and filters to keep rendering fast.

# views.py
from django.shortcuts import render
from .models import Hiker

def hiker_list(request):
    hikers = Hiker.objects.select_related('related_model').all()
    return render(request, 'hikers/list.html', {'hikers': hikers})

Note that using {% block %} is faster than using {% include %}.

9. Advanced Python Techniques

a. Lazy Evaluation:

Utilize Django’s @cached_property decorator for caching expensive computations within a model instance.

from django.utils.functional import cached_property

class MyModel(models.Model):
    @cached_property
    def expensive_property(self):
        return expensive_computation()

b. Using PyPy:
PyPy is an alternative Python implementation that can execute code faster for CPU-bound operation

Consider using PyPy for performance improvements, but be aware of compatibility issues with certain Django dependencies. Test thoroughly before deploying to production.

10. Using the Latest Version of Django

Always use the latest version of Django to benefit from performance improvements, security patches, and new features. Keep your dependencies updated alongside Django for optimal performance and security.

Conclusion

Remember, optimization is an ongoing process. Continuously monitor your application’s performance and be proactive in identifying and addressing bottlenecks. Happy coding! 😊

Django Admin - Zero to Hero

KagemaNjoroge — Sun, 26 May 2024 15:01:56 +0000

One of the most powerful parts of Django is the automatic admin interface. It reads metadata from your models to provide a quick, model-centric interface where trusted users can manage content on your site.
The admin’s recommended use is limited to an organization’s internal management tool. It’s not intended for building your entire front end around. In this article, we will cover everything you need to know to become a Django Admin pro. We will start from scratch and gradually move to advanced topics.
By the end of this article, you will be able to manage your Django project efficiently using Django Admin.
This will be a long article, feel free to jump to the section you are interested in:

Activating Django Admin
Creating Superuser
Accessing Django Admin Site
An Overview of Django Admin Interface
Creating and registering models
ModelAdmin class
ModelAdmin Actions

Activating Django Admin

The admin is enabled in the default project template used by startproject. If you’re not using the default project template, here are the requirements:

Add django.contrib.admin and its dependencies - django.contrib.auth, django.contrib.contenttypes, django.contrib.messages, and django.contrib.sessions - to your INSTALLED_APPS setting.
Add admin.site.urls to your URLconf. Modify the urls.py file in your project folder to include the following line:

urlpatterns = [
    path('admin/', admin.site.urls),
]

Creating Superuser account

To access the Django admin, you need to create a superuser. Run the following command in your terminal:

python manage.py createsuperuser

You will be prompted to enter a username, email address, and password for the superuser.

Accessing Django Admin Site

Run the Django development server:

python manage.py runserver

By default, Django development server will run on port 8000. Open your browser and navigate to http://localhost:8000/admin/. You will see the Django admin login page.

Enter the superuser credentials you created earlier to log in.

An Overview of Django Admin Interface

Without any other registered models, Django will display the “Groups” and “Users” models. These are part of Django’s authentication framework. You can use these models to manage users and groups in your Django project. Please note how they are categorized under the “Authentication and Authorization” section.

Creating and registering models

In this article let's work with the Books model located in the books app. The Books model has the following fields:

title - The title of the book.
author - The author of the book.
price - The price of the book.
published - A boolean field to indicate if the book has been published.

Assuming you have already created the books app, let's create the Books model in the models.py file of the books app:

class Book(models.Model):
    title = models.CharField(max_length=100)
    author = models.CharField(max_length=100)
    price = models.DecimalField(max_digits=10, decimal_places=2)
    published = models.BooleanField(default=False)

    def __str__(self):
        return self.title # Display the title of the book in the Django admin site
    class Meta:
        # the attributes below are very useful for the Django admin interface, they are not required but are good practice
        verbose_name = 'Book' # Singular name for the model
        verbose_name_plural = 'Books' # Plural name for the model

In your admin.py file of the books app, register the Books model:

from .models import Book
admin.site.register(Book)

Save the file and make migrations:

python manage.py makemigrations
python manage.py migrate

Restart the Django development server:

python manage.py runserver

Head over to the Django admin site and you will see the Books model listed under the Books section. You can now add, edit, and delete books using the Django admin interface.
Make sure to create some books using the Django admin interface.

How cool is that?

A little further - ModelAdmin class

It is possible to register a model with just the admin.site.register(ModelName) method.

Perhaps you need more control over how the model is displayed in the admin interface. This is where the ModelAdmin class comes in.

The ModelAdmin class is the representation of a model in the admin interface. It allows you to customize how the model is displayed, how it can be interacted with, and how it can be filtered.

You can customize the ModelAdmin class by creating a class that inherits from admin.ModelAdmin and then registering it with the model.

In our case, we want to display the Books model with a list of books sorted by the price in ascending order. We also want to display the title and author fields in the list view.

Let's create a BookAdmin class in the admin.py file of the books app:

class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'author') # Display title and author fields in the list view
    ordering = ('price',) # Sort by price in ascending order

admin.site.register(Book, BookAdmin)

In the end the admin.py file of the books app should look like this:

from django.contrib import admin
from .models import Book
class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'author') # Display title and author fields in the list view
    ordering = ('price',) # Sort by price in ascending order
admin.site.register(Book, BookAdmin)

Head over to the Django admin site and you will see the Books model listed under the Books section. The books will be displayed with the title and author fields in the list view, sorted by price in ascending order.

Let me teach you an even concise way to register the ModelAdmin using the @admin.register decorator.

@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
    list_display = ('title', 'author') # Display title and author fields in the list view
    ordering = ('price',) # Sort by price in ascending order

The above code is equivalent to the previous code where we registered the BookAdmin class with the Books model but in a more concise way.

ModelAdmin Actions

If you have noticed, selecting multiple books provides a dropdown with the action "Delete selected books". This is an example of a ModelAdmin action.

That sounds limited, right? What if you want to create your own custom actions?

Custom ModelAdmin Actions

The easiest way to explain actions is by example, so let’s dive in.
A common use case for admin actions is the bulk updating of a model’s fields.
To create an action, you need to define a method in the ModelAdmin class and decorate it with the @admin.action decorator.

Let's create a custom action to mark selected books as published.

class BookAdmin(admin.ModelAdmin):
    list_display = (
        "title",
        "author",
    )  # Display title and author fields in the list view
    ordering = ("price",)  # Sort by price in ascending order
    actions = ["mark_as_published"]  # Add the custom action

    @admin.action(description="Mark selected books as published")
    def mark_as_published(self, request, queryset):
        queryset.update(published=True)


admin.site.register(Book, BookAdmin)

Admin model actions take three arguments: self, request, and queryset. The queryset argument is a list of all the selected objects. In our case, it will be a list of selected books.

That is just a tip of the iceberg. There is so much more you can do with Django Admin. I hope this article has given you a good foundation to start exploring Django Admin. Feel free to explore the official Django documentation to learn more about Django Admin.

Happy coding! 🚀

Are you still exposing your Django SECRET_KEY?

KagemaNjoroge — Mon, 20 May 2024 22:05:27 +0000

Django settings.py contains the most sensitive information of your project. Hard coding key details such as SECRET_KEY,ALLOWED_HOSTS, database settings such database password, database host, database port, database username in your codebase is a security risk. In this article, we will discuss how to secure your Django SECRET_KEY by using environment variables and the python dotenv library. While I will focus on the SECRET_KEY, the same principles can be applied to other sensitive information in your Django project.

Why is the Django `SECRET_KEY` important?

Django uses the SECRET_KEY to provide cryptographic signing, protection against CSRF attacks, and other security features. If an attacker gains access to your SECRET_KEY, they can potentially compromise the security of your Django project. It is crucial to keep the SECRET_KEY confidential and secure.

Installing the `python-dotenv` library

The python-dotenv library allows you to load environment variables from a .env file into your Django project. To install the python-dotenv library, you can use pip:

pip install python-dotenv

If you do have a requirements.txt file, you can add the python-dotenv library to it.

Creating a `.env` file

Create a .env file in the root directory(the same directory as manage.py) of your Django project. Add the SECRET_KEY to the .env file in the following format:

SECRET_KEY="your_secret_key_here"

Adding the `.env` file to `.gitignore`

The .env file contains sensitive information, and you should not commit it to your version control system. To prevent the .env file from being accidentally committed, add it to your .gitignore file. Here is an example of how you can add the .env file to your .gitignore file:

# .gitignore
.env

Loading the `SECRET_KEY` from the `.env` file

In your settings.py file, you will need to import the os and dotenv modules and load the SECRET_KEY from the .env file. Here is an example of how you can do this:

import os
from dotenv import load_dotenv

load_dotenv()

load_dotenv() will load the environment variables from the .env file into your Django project. After reading from the .env file, load_dotenv() will update the os.environ dictionary with the key-value pairs from the .env file.

It is important to call load_dotenv() before accessing the SECRET_KEY in your settings.py file.

Accessing the `SECRET_KEY` in your `settings.py` file

Now that you have loaded the SECRET_KEY from the .env file, you can access it in your settings.py file using the os module:

SECRET_KEY = os.getenv('SECRET_KEY')

By using os.getenv('SECRET_KEY'), you can access the SECRET_KEY value stored in the .env file. This ensures that the SECRET_KEY is not hard-coded in your codebase and is kept secure.

Please note the above steps can be applied to other sensitive information in your Django project.

Happy coding! 🚀🐍

Why You Shouldn’t Be Worried About AI

KagemaNjoroge — Fri, 08 Mar 2024 14:13:57 +0000

Let’s be honest, when the topic of AI comes up, a lot of us jump straight to those sci-fi disaster movies. Robots with glowing red eyes, AI taking over the world — it’s the stuff of nightmares! And thanks to endless headlines about AI doing amazing (and sometimes scary) things, it’s easy to feel a little uneasy.

But here’s the thing: AI is not some self-aware villain waiting to happen. It’s a tool, and a staggeringly powerful one, but a tool nonetheless. Think of it like this: a hammer can be used to build a house or to cause harm. It’s the intent that matters, not the tool itself.

The reality is, AI is already woven into our lives. It recommends movies for you to watch, helps doctors spot diseases earlier, and even helps with tasks like writing emails. It’s not out to get you — it’s out to make things easier.

Misconceptions

Of course, it’s wise to be aware of the potential pitfalls of AI, just like any technology. Concerns about bias, misuse, and job displacement are valid. One prevalent fear is that AI will inevitably lead to widespread job loss and economic upheaval. While it’s true that AI may disrupt certain industries, history has shown that technological advancements ultimately create more jobs than they eliminate.

For instance, while AI may automate routine tasks, it also creates opportunities for new types of jobs. Roles such as AI ethicists, data privacy officers, and AI trainers are emerging as crucial components of the AI workforce. These jobs require human skills such as critical thinking, creativity, and empathy, which AI cannot replicate.

As Andrej Karpathy, wisely puts it, “We should see AI as a copilot.” It’s a perfect analogy: AI isn’t replacing us, it’s there to augment our abilities. The future belongs to those who adapt, who see how AI can augment what they do — not do it for them.

The Role of Ethics and Regulation & the future of AI

Now, let’s be real — safeguards are needed. Just like we don’t let just anyone build a nuclear power plant, we need sensible AI regulations. Global cooperation on this is vital. Think about it: we need agreements to prevent the use of autonomous AI weapons and ensure that these powerful tools are used ethically. If we do not have AI regulations, some armies of the world might take AI to the battlefield. Regulation isn’t about stifling innovation; it’s about making sure AI serves the greater good.

AI can evoke a sense of unease. However, I argue that rather than fearing AI, we should embrace its potential as a transformative force for good.

What are your thoughts on AI? Do you find the possibilities exciting, the potential risks concerning, or both?

Why do people think coding is difficult?

KagemaNjoroge — Tue, 05 Mar 2024 15:37:39 +0000

Simple amateur coding is trivial and most people can do it. Probably making ten or twenty lines of code in a programming language is something most people could do I think.

Factors mess that up:

Figuring out what to code. You just get asked, say, ‘Can you make it email me if more than five expense claims get rejected on the same day for the same person?’. How do you code that?
Fitting it in to existing code. You have to figure out how to fit your new code in to the existing code without breaking it. It might be millions of lines you have never seen. it might be very badly constructed and might break on small changes. That bit is hard.

And then, we want to do better as professionals.

We want our code to have automated tests, which changes how you design it a bit.

We want it to be easy to deploy to computers, so you have to design that in.

We want it free from security holes, to keep data safe.

We want to never corrupt any data.

We want our colleagues to easily read and easily change our code, which needs designing in.

We want our code to be robust and correctly handle bad data, bad user input and anything else unexpected. We have to learn to expect that.

Those bits are pretty hard, too.

Then, our bosses want to know when all that is going to be ready, so they can plan around it. Once the TV advert goes live, that software better be running.

That’s hard.

We have to juggle competing priorities of adding features, managing tech debt and fixing bugs.

There’s no right answer for that. Somebody is always disappointed.

We have to cope with floods of new users and feature requests that are difficult to build. The whole scalability piece is hard.

The code needs to stay running 24/7 or close to it - even if half the computers, disk drives, cooling systems, internet switches and electrical power fails.

That adds a bit of trickiness for sure.

We need to use existing libraries wherever we can, so we need to understand code we’ve not written. Tricky. But trickier still, we need to be part-time lawyers, because we have to check we are legally allowed to use that code without it affecting the business. Is it Creative Commons License or is it GNU?, that's another level of complexity.

We need to remember somebody pays for all the computing our code runs on, so we have to make it play nice.

We have to work for and with people who range from fantastic humans to the sort you’d rather not spend time with. And with all ranges of skill levels.

That’s a real mixed bag.

We have to know so much about computing, engineering, the problem area we are solving, testing.

That’s the really hard bit.

Django Framework is Dead 😵

KagemaNjoroge — Mon, 04 Mar 2024 22:26:22 +0000

Ladies and gentlemen, gather 'round, for I bring grave news from the world of Python web development. It is with a heavy heart that I declare the passing of our dear friend, Django framework.

But before you shed a tear or organize a funeral procession, let me assure you – Django is very much alive and kicking!

Yes, you read that correctly. Despite the ominous title, Django is far from dead. In fact, it's thriving more than ever, and here's why.

As we bid adieu to the era of monolithic frameworks and embrace the brave new world of micro services and serverless architecture, some may have prematurely written off Django as a relic of the past. But let me remind you of Django's timeless virtues – its elegance, its simplicity, its "batteries-included" philosophy that has endeared it to developers for decades.

While other frameworks may come and go, Django stands the test of time. Its robust ORM, its sleek admin interface, its powerful authentication system – these are not features of a dying framework, but the hallmarks of a mature, battle-tested tool-set that continues to evolve with the times.

So, why the provocative title, you ask? Well, dear reader, it's all about grabbing your attention and sparking a conversation. Sometimes a little irony is the best way to cut through the noise and make a point.

And the point is this – Django may not be the shiny new toy on the block, but it's the dependable workhorse that gets the job done. Whether you're building a simple blog or a complex enterprise application, Django has your back, providing a solid foundation upon which to build your dreams.

So, let's raise a toast to Django – not in mourning, but in celebration of its enduring legacy and the countless projects it has powered over the years. If you are a perfectionist with a deadline to meet, you should definitely try Django. Long live Django!

What are the differences between Django and Flask?

KagemaNjoroge — Mon, 04 Mar 2024 22:03:13 +0000

Django and Flask are both popular Python web frameworks, each with its own strengths and use cases. Here are some key differences between them:

Complexity and Learning Curve

Django is a full-stack framework that follows the "batteries-included" philosophy, providing a wide range of built-in functionalities such as authentication, admin site, ORM, and URL routing out of the box. While this makes Django powerful and feature-rich, it also contributes to a steeper learning curve, especially for beginners.
Flask, on the other hand, is a lightweight micro-framework that offers more flexibility and minimalism. It provides only the essentials for building web applications, allowing developers to choose and integrate additional libraries or extensions as needed. Flask's simplicity makes it easier to pick up and learn, making it a preferred choice for smaller projects or developers new to web development.

Flexibility and Customization

Flask's minimalist approach allows for greater flexibility and customization compared to Django. Developers have more control over the architecture and components of their applications, making Flask suitable for building APIs, micro-services, and other lightweight web applications where fine-grained control is desired.
Django, while less flexible in terms of its predefined structure, offers a more opinionated approach to web development, which can be advantageous for larger projects or teams. Its built-in components and conventions help maintain consistency and scalability across projects, making it well-suited for complex, enterprise-level applications.

Industry Adoption and Community Support

Django is widely considered an industry standard for Python web development, with a large and active community of developers, extensive documentation, and a rich ecosystem of third-party packages and extensions. Its robust feature set, stability, and scalability have made it a popular choice for a wide range of web applications, including content management systems, e-commerce platforms, and social networks.
Flask, while not as ubiquitous as Django, also enjoys significant adoption and community support. It is favored for its simplicity, flexibility, and ease of integration with other tools and libraries. Flask's lightweight nature and extensibility make it particularly well-suited for rapid prototyping, small to medium-sized projects, and applications requiring specialized functionality.

Django's comprehensive feature set and industry-standard status make it a solid choice for large-scale applications requiring stability and scalability. However, Flask's simplicity, flexibility, and ease of learning make it an excellent option for smaller projects, prototyping, and developers seeking more control over their application architecture. Ultimately, the choice between Django and Flask depends on the specific requirements, complexity, and constraints of your project. Read my article on why Django can ruin your life for more dev stories about Django.

The Day I Realized I Was a Noob Dev: Lessons from an Anagram Mistake

KagemaNjoroge — Thu, 22 Feb 2024 11:32:53 +0000

Every programmer has their moments of humility, and I had mine on a day when I thought I had it all figured out. It all started with a seemingly simple programming problem - checking if two strings were anagrams. Little did I know that this challenge would reveal just how much I had to learn.
The Problem That Fooled Me
The problem was straightforward: Given two strings, s and t, determine if t is an anagram of s.
An anagram, as we all know, is a word or phrase formed by rearranging the letters of another word or phrase, typically using all the original letters exactly once. Piece of cake, right?

Here's the question in its entirety:
LeetCode link to the problem

'''
Given two strings s and t, return true if t is an anagram of s, 
and false otherwise.
An Anagram is a word or phrase formed by rearranging the letters
of a different word or phrase, typically using all the original
letters exactly once.

Example 1:

Input: s = "anagram", t = "nagaram"
Output: true
Example 2:

Input: s = "rat", t = "car"
Output: false


Constraints:

1 <= s.length, t.length <= 5 * 104
s and t consist of lowercase English letters.

'''

My Initial Solution
I dived into the problem headfirst, convinced that I could conquer it effortlessly. My initial solution? Compare the two strings character by character and return True if they were the same length. Simple enough, or so I thought:

def anagram(s: str, t: str) -> bool:
    if len(s) != len(t):
        return False

    data_t = {}
    data_s = {}

    for i in s:
        data_s[i] = i
    for f in t:
        data_t[f] = f

    return data_s == data_t

The "Aha! 😒" Moment

I ran my code, and it seemed to work for some test cases. I felt a sense of accomplishment. But then reality hit me like a ton of bricks. What about the characters' counts? An anagram isn't just about having the same characters; it's about having the same characters in the same quantities.

The Big Mistake

My "brilliant" solution had completely overlooked this crucial aspect. I realized that my code would incorrectly declare strings like "rat" and "car" as anagrams simply because they contained the same characters.

I had rushed into solving the problem without fully understanding its requirements. But there's no shame in making mistakes; the real shame is not learning from them.
The Revised Solution

I knew I had to fix my code. I learned about the importance of character counts in anagrams and came up with a revised solution that properly addressed this aspect.

def anagram(s: str, t: str) -> bool:
    if len(s) != len(t):
        return False

    char_count = {}

    for char in s:
        if char in char_count:
            char_count[char] += 1
        else:
            char_count[char] = 1

    for char in t:
        if char in char_count:
            char_count[char] -= 1
            if char_count[char] < 0:
                return False
        else:
            return False

    return True

Lesson
never underestimate the importance of understanding the problem fully

Go ahead and tell me about the day you knew that you had to work on you programming skills.

Character vs. Word Tokenization in NLP: Unveiling the Trade-Offs in Model Size, Parameters, and Compute

KagemaNjoroge — Thu, 22 Feb 2024 11:09:06 +0000

In Natural Language Processing, the choice of tokenization method can make or break a model. Join me on a journey to understand the profound impact of character-level, word-level tokenization and Sub-word tokenization on model size, number of parameters, and computational complexity.

First Things First, What is Tokenization?

AI operates with numbers, and for deep learning on text, we convert that text into numerical data. Tokenization breaks down textual data into smaller units, or tokens, like words or characters, which can be represented numerically.

Decoding Tokenization

There are various techniques in tokenization, such as:

Word Tokenization: Divides text into words, creating a vocabulary of unique terms.
Character Tokenization: Breaks down text into individual characters, useful for specific tasks like morphological analysis.
Sub word Tokenization: Splits words into smaller units, capturing morphological information effectively. Examples BERT and SentencePiece

Character-Level Tokenization

We essentially create tokens out of individual characters present in the text. This involves compiling a unique set of characters found in the dataset, ranging from alphanumeric characters to ASCII symbols. By breaking down the text into these elemental units, we generate a concise set of tokens, resulting in a smaller number of model parameters. This lean approach is particularly advantageous in scenarios with limited datasets, such as low-resource languages, where it can efficiently capture patterns without overwhelming the model.

# Consider the sentence, "I love Python"
# If we tokenize by characters the result will be ['I', ' ', 'l', 'o', 'v', 'e', ' ', 'P', 'y', 't', 'h', 'n']
sent = "I love Python"
tokens = [i for i in set(sent)] # Use a set to obtain ony unique ones
# if we then represent the sentence above numerically
numerical_representation = {i:ch for i, ch in enumerate(tokens)}
number_of_tokens = len(s)

Word-Level Tokenization:

Word-level tokenization involves breaking down the text into individual words. This process results in a vocabulary composed of unique terms present in the dataset. Unlike character-level tokenization, which deals with individual characters, word-level tokenization operates at a higher linguistic level, capturing the meaning and context of words within the text.

This approach leads to a larger model vocabulary, encompassing the diversity of words used in the dataset. While this richness is beneficial for understanding the semantics of the language, it introduces challenges, particularly when working with extensive datasets. The increased vocabulary size translates to a higher number of model parameters, necessitating careful management to prevent overfitting.

# Consider the sentence, "I love Python"
sent = "I love Python"
tokens = [i for i in set(sent.split())]
numerical_representation = {i:ch for i, ch in enumerate(tokens)}

However, the trade-off lies in the potential for overfitting, especially when dealing with smaller datasets. Striking a balance between a rich vocabulary and avoiding over-parameterization becomes a critical consideration when employing word-level tokenization in natural language processing tasks.

Subword Tokenization

Subword tokenization interpolates between word-based and character-based tokenization.

Instead of treating each whole word as a single building block, subword tokenization breaks down words into smaller, meaningful pieces.
These smaller parts, or subwords, carry meaning on their own and help the computer understand the structure of the word in a more detailed way.
Common words get a slot in the vocabulary, but the tokenizer can fall back to word pieces and individual characters for unknown words.

Let's do a simple experiment to show the impacts of a tokenization method on the model

Colab Link to the code

From the experiment I conducted above the trend is as follows:

As the number of tokens increases:
- The model size also increases
- Model training and inference becomes more compute demanding
- The size of dataset required to achieve high accuracy also increases

Which tokenization method should I use?

Sub-word tokenization is the industry standard!

Consider Byte Pair Encodinge(BPE) techniques such as:

Use character level tokenization or word level tokenization where you have a smaller dataset.