DEV Community: Amartya Jha

Fixing Complex Functions and Making Code Modular and Reusable

Amartya Jha — Wed, 07 Aug 2024 20:27:22 +0000

Learn how to fix complex functions and make your code modular and reusable with these expert tips and best practices for cleaner, more maintainable code.

Introduction

In the world of software development, writing clean, modular, and reusable code is a hallmark of good programming practice.

Why Fixing Complex Functions Matters

Complex functions are often difficult to debug, test, and maintain. They can slow down development, introduce bugs, and make it harder for new team members to understand the codebase. Simplifying these functions not only improves readability but also enhances the overall performance and scalability of the software.

Identifying Complex Functions

Signs of Complexity

Length: Functions that are excessively long.
Multiple Responsibilities: Functions performing multiple tasks.
Nested Loops and Conditionals: Deeply nested loops and conditional statements.
High Cyclomatic Complexity: Numerous paths through the function.
Poor Naming Conventions: Non-descriptive variable and function names.

Tools for Analysis

Code Linters: Tools like ESLint, Pylint, and JSHint can help identify complex code.
Code Metrics Tools: Tools such as SonarQube and CodeClimate provide metrics to measure complexity.

Steps to Fix Complex Functions

Break Down the Function

Single Responsibility Principle

Each function should perform a single task. Breaking down a complex function into smaller, single-purpose functions can make the code more understandable and easier to manage.

Example:

Instead of:

def process_data(data):
    # Read data from source
    # Validate data
    # Transform data
    # Save data to database
    pass

Break it down into:

def read_data(source):
    pass

def validate_data(data):
    pass

def transform_data(data):
    pass

def save_data(data):
    pass

def process_data(source):
    data = read_data(source)
    if validate_data(data):
        transformed_data = transform_data(data)
        save_data(transformed_data)

Refactor Loops and Conditionals

Nested loops and conditionals can often be refactored into simpler constructs or moved into separate functions to improve readability.

Example:

Instead of:

for item in items:
    if item.is_valid():
        process_item(item)

Refactor to:

def process_valid_items(items):
    valid_items = filter(is_valid, items)
    for item in valid_items:
        process_item(item)

Use Descriptive Names

Names should convey the purpose of the variable or function. Avoid abbreviations and ensure the names are meaningful.

Example:

Instead of:

def calc(x, y):
    return x + y

Use:

def calculate_sum(number1, number2):
    return number1 + number2

Making Code Modular

What is Modular Code?

Modular code is structured in a way that separates functionality into independent, interchangeable modules. Each module encapsulates a specific piece of functionality, making the codebase easier to manage, test, and reuse.

Principles of Modular Code

Encapsulation: Keep related data and functions together.
Separation of Concerns: Divide the program into distinct features that overlap as little as possible.
Reusability: Modules should be designed to be reusable in different parts of the application or even in different projects.

Creating Modules

Organize by Feature

Group related functions and data together into modules that represent a specific feature or functionality.

Example:

# user_management.py
def create_user(user_data):
    pass

def delete_user(user_id):
    pass

# product_management.py
def add_product(product_data):
    pass

def remove_product(product_id):
    pass

Use Interfaces

Define clear interfaces for your modules to ensure they can interact with other parts of the codebase in a predictable way.

Example:

# user_management.py
class UserManager:
    def create_user(self, user_data):
        pass

    def delete_user(self, user_id):
        pass

Enhancing Reusability

DRY Principle

The "Don't Repeat Yourself" principle emphasizes the importance of reducing duplication. Reusable code should abstract common functionality into functions or classes that can be used throughout the codebase.

Example:

Instead of duplicating code:

def send_email(to, subject, body):
    # Email sending logic
    pass

def send_notification(to, message):
    # Notification sending logic
    pass

Abstract common logic:

def send_message(to, subject, body, message_type):
    if message_type == 'email':
        # Email sending logic
    elif message_type == 'notification':
        # Notification sending logic
    pass

Use Libraries and Frameworks

Leverage existing libraries and frameworks that provide reusable components to avoid reinventing the wheel.

Example:

import requests

def fetch_data_from_api(url):
    response = requests.get(url)
    return response.json()

Write Generic Functions

Write functions that can handle a variety of inputs and use parameters to control behavior.

Example:

def sort_list(data, reverse=False):
    return sorted(data, reverse=reverse)

Testing and Documentation

Unit Testing

Ensure each module and function is thoroughly tested with unit tests to guarantee they work correctly in isolation.

Example:

import unittest

class TestUserManager(unittest.TestCase):
    def test_create_user(self):
        user_manager = UserManager()
        result = user_manager.create_user({'name': 'John'})
        self.assertTrue(result)

Document Your Code

Comprehensive documentation helps other developers understand how to use your modules and functions. Include comments, docstrings, and external documentation as needed.

Example:

def calculate_sum(number1, number2):
    """
    Calculate the sum of two numbers.

    :param number1: First number
    :param number2: Second number
    :return: Sum of number1 and number2
    """
    return number1 + number2

We use CodeAnt AI to detect and suggest auto-fixing of complex function.

It list complex functions in the entire repository with Cyclomatic Complexity.

And then suggest auto-fixing, which is kinda insightful.

Why we still have tech-debt?

Amartya Jha — Sun, 11 Feb 2024 01:06:26 +0000

When a developer creates a pull request, the reviewer checks it for business logic. However, what often gets overlooked are:

Data structure optimization
Code security checks
Function complexity and reusability
Cleaning up dead and duplicate pieces

Let's break it down and understand all 3 stages.

Stage 1 - Development Editor:

We all add linters; they detect issues related to formatting and some anti-patterns. As developers, we see numerous issues, and even some paid linters stack rank the issues based on severity. However, it fails in multiple aspects because developers don't have time to manually fix these issues. Developers want these issues to be auto-fixed, or else they will silence them. Secondly, these static linters lack context about your codebase, so they may not effectively prioritize high-severity, high-impact issues for you.

Stage 2 - Git Hook (Maybe):

Some organizations have Git hooks in place to prevent certain bad practices from being pushed. The problem here lies in the depth of the rules, as most rules focus on high-level, framework-specific checks. You can think of these as extensions of your linters, with the added capability that you can't push the code if certain rules are not followed.

Stage 3 - Pull Requests (PRs):

Now, you have pushed your changes and created a PR, waiting for the reviewer to merge it. Here, the challenge is that the reviewer also has 10 other PRs to review, so they primarily check for the correctness of the business logic. Having reviewed over 1000 PRs, what I've learned is that no reviewer can ever have 100% context of their codebase. Therefore, we trust that the developer has followed all the checks, fixed anti-patterns, dead and duplicate code, ensured the use of correct data structures, etc. Additionally, awareness of security and compliance issues is not always present with most developers, so we don't review those aspects. Moreover, both the reviewer and developer are under pressure to ship the product faster, with the end goal being deployment in lower environments. If something breaks, we will fix it there.

The reality is that nothing will break because the existing checks primarily focus on unit or end-to-end tests for business logic, which are already reviewed. Consequently, "bad" code gets checked in and committed.

This situation results in technical debt. Why haven't we been able to solve it until now? It's due to a lack of enforcement that doesn't slow down developer velocity.

What do we need then?
We need tools that have a great context of the codebase, prioritize high-severity, high-impact issues, and suggest auto-fixing of these issues without breaking any existing logic. The tool should also seamlessly integrate with the developer journey starting from IDEs to PR checkers.

We are building CodeAnt AI (YC W24) on the same lines, and it is live; feel free to check it out here.

Losing events from AWS lambda?

Amartya Jha — Fri, 09 Feb 2024 03:16:30 +0000

Hello everyone, I have been building venture-backed startups for over 4 years now, and security has always been a challenge. Trust me, identifying impactful bugs and dedicating the team's time to fixing them is another struggle - one that I wanted to fix.

Currently, I am building CodeAnt AI, backed by YC. We have made it a priority to develop this as a tool that tackles real pain points for developers.

Here is an example: We deployed our tool in a company, scanned their codebase, and discovered a missing Deadletter queue in AWS Lambda.

Let's understand the impact of this bug. When AWS Lambda sends events to an SQS queue and encounters failures, it could be due to network partitions or sending too much data for SQS to handle. In such cases, AWS retries the failed events a certain number of times (around 3). If all retries fail, the event is discarded without storage. This means losing the entire event data, with potential consequences, especially if it's a transaction event.

CodeAnt fixed this by suggesting Deadletter queue. AWS Lambda now sends all failed events to this queue. When SQS is back up, the Deadletter queue resends these failed events, ensuring no loss of transaction data in our case.

Would love it if you guys would try CodeAnt AI, and give me some feedback - https://github.com/marketplace/codeant-ai.

We detect and auto-fix bad code, like security vulnerabilities, code antipatterns, complex functions, dead & duplicate code etc.