DEV Community: itzsrikanth

Debugging Java ClassNotFoundException: A Beginner's Guide to Finding Missing Dependencies

itzsrikanth — Sun, 16 Nov 2025 01:42:04 +0000

Introduction

If you've ever encountered a ClassNotFoundException or NoClassDefFoundError in Java, you know how frustrating it can be. Your code compiled fine, but at runtime, Java can't find a class it needs. This guide will teach you a systematic approach to debugging these issues, using a real-world Apache Flink example.

In this tutorial, we'll walk through diagnosing and fixing a missing Hadoop dependency in a Flink application. More importantly, you'll learn a repeatable framework you can apply to any Java dependency issue.

The Problem: Understanding the Error
The Debugging Process
Common Java Dependency Errors
Step-by-Step Framework
Practical Tools and Commands
Advanced Tips
Practice Exercise
Resources

The Problem: Understanding the Error

Let's start with the error message we encountered:

Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configurable
    at java.base/java.lang.ClassLoader.defineClass1(Native Method)
    at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
    ...
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configurable
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
    ...

At first glance, this looks intimidating. But let's break it down systematically.

The Debugging Process

Step 1: Read the Stack Trace Bottom-Up

Key Principle: Always start with the deepest "Caused by" clause. This is your root cause.

Main Exception: ProgramInvocationException
    ↓
Caused by: TableException
    ↓
Caused by: ServiceConfigurationError
    ↓
Caused by: NoClassDefFoundError         ← Near root cause
    ↓
Caused by: ClassNotFoundException       ← ROOT CAUSE! Start here!

In our case, the root cause is:

ClassNotFoundException: org.apache.hadoop.conf.Configurable

What this tells us:

Java can't find the class Configurable
It's in package org.apache.hadoop.conf
This is a Hadoop class

Step 2: Identify the Library

From the fully qualified class name, you can identify which library it belongs to:

org.apache.hadoop.conf.Configurable
└──────┬─────┘ └──┬──┘ └────┬────┘
   Vendor    Module  Class

Common package patterns:

org.apache.flink.*         → Apache Flink
org.apache.hadoop.*        → Apache Hadoop  
org.apache.kafka.*         → Apache Kafka
com.fasterxml.jackson.*    → Jackson JSON library
org.springframework.*      → Spring Framework
com.google.gson.*          → Google Gson

Step 3: Understand the Context

Look at where the error occurred in your stack trace:

at com.example.MyTableFactory.<init>(MyTableFactory.java:127)

This tells us:

The MyTableFactory class is being initialized
It requires Hadoop classes to work
The Hadoop library is missing from the classpath

Step 4: Search for the JAR

Now we need to find where the missing class should be. Let's search systematically:

# Search in your application directory
find . -name "*.jar" -exec grep -l "org/apache/hadoop" {} \;

# For Apache Flink specifically, check both lib and opt directories
ls build/flink/lib/ | grep -i hadoop
ls build/flink/opt/ | grep -i hadoop

Why check both lib/ and opt/?

lib/ → JARs loaded automatically by the framework
opt/ → Optional JARs that must be explicitly enabled

In our case, we found:

$ ls build/flink/opt/ | grep -i hadoop

flink-s3-fs-hadoop-1.16.1.jar       ← Contains Hadoop classes!
flink-azure-fs-hadoop-1.16.1.jar
flink-gs-fs-hadoop-1.16.1.jar

Step 5: Verify the JAR Contains the Missing Class

Before proceeding, let's confirm this JAR actually contains the class we need:

# Method 1: Using unzip
unzip -l flink-s3-fs-hadoop-1.16.1.jar | grep "org/apache/hadoop/conf/Configurable"

# Method 2: Using jar command  
jar -tf flink-s3-fs-hadoop-1.16.1.jar | grep "Configurable"

Output:

717  2025-09-27 03:45   org/apache/hadoop/conf/Configurable.class  ✓ Found it!

Step 6: Fix the Issue

Now that we've identified the problem and the solution, we can fix it:

# Copy the JAR from opt/ to lib/ so it's loaded automatically
cp build/flink/opt/flink-s3-fs-hadoop-1.16.1.jar build/flink/lib/

Alternative solutions depending on your setup:

Add the JAR to your application's classpath
Include it as a runtime dependency in Maven/Gradle
Set the CLASSPATH environment variable
Use the -cp flag when running Java

Common Java Dependency Errors

Understanding these common errors will help you debug faster:

Error	Meaning	Common Causes	Solution
`ClassNotFoundException`	Java can't find the `.class` file at runtime	Missing JAR in classpath	Add the JAR containing the class
`NoClassDefFoundError`	Class was available at compile time but missing at runtime	Missing dependency JAR, or initialization failure	Check runtime classpath; verify static initializers
`NoSuchMethodError`	Method exists but with different signature	Version conflict between JARs	Check for duplicate dependencies with different versions
`LinkageError`	Class loaded by incompatible classloaders	Duplicate JARs in classpath	Remove duplicate JARs; use dependency management
`IncompatibleClassChangeError`	Class structure changed between compile and runtime	Mismatched library versions	Ensure compile and runtime versions match

Step-by-Step Framework

Here's a repeatable process for debugging any ClassNotFoundException:

1. Identify the Missing Class

ClassNotFoundException: com.example.MyClass
                        └──────┬──────┘
                           Package name

Extract:

Package: com.example
Class: MyClass
Likely library: Based on package naming convention

2. Search Your Dependencies

For Maven projects:

# View all dependencies
mvn dependency:tree

# Search for specific dependency
mvn dependency:tree | grep -i "hadoop"

For Gradle projects:

# View all dependencies
./gradlew dependencies

# View specific configuration
./gradlew dependencies --configuration runtimeClasspath

3. Check JAR Contents

Quick check if a JAR contains a class:

# Using jar command (shows all files)
jar -tf myapp.jar | grep "MyClass"

# Using unzip (more detailed)
unzip -l myapp.jar | grep "com/example/MyClass"

# Using zipgrep (searches file contents)
zipgrep "MyClass" myapp.jar

4. Verify Runtime Classpath

Add temporary debugging code:

import java.net.URL;
import java.net.URLClassLoader;

public class ClasspathDebugger {
    public static void main(String[] args) {
        ClassLoader cl = ClassLoader.getSystemClassLoader();
        URL[] urls = ((URLClassLoader)cl).getURLs();

        System.out.println("Current Classpath:");
        for(URL url: urls) {
            System.out.println("  " + url.getFile());
        }
    }
}

Or use JVM flags:

# Shows all classes being loaded and their source
java -verbose:class -jar myapp.jar

# Shows classpath being used
java -XX:+TraceClassPaths -jar myapp.jar

5. Compare Compile vs Runtime Dependencies

This is a common source of issues:

dependencies {
    // Compile-time only (not included in runtime)
    compileOnly 'org.apache.hadoop:hadoop-common:3.3.0'

    // Runtime dependency (included when running)
    runtimeOnly 'org.apache.hadoop:hadoop-common:3.3.0'

    // Both compile and runtime (recommended)
    implementation 'org.apache.hadoop:hadoop-common:3.3.0'
}

Practical Tools and Commands

Tool 1: Finding Classes in JARs

Script to search all JARs for a class:

#!/bin/bash
# find-class.sh - Search for a class in all JARs

CLASS_NAME=$1
SEARCH_DIR=${2:-.}

echo "Searching for $CLASS_NAME in $SEARCH_DIR..."

find "$SEARCH_DIR" -name "*.jar" | while read jar; do
    if jar -tf "$jar" | grep -q "$CLASS_NAME"; then
        echo "Found in: $jar"
    fi
done

Usage:

chmod +x find-class.sh
./find-class.sh "org/apache/hadoop/conf/Configurable" ./lib

Tool 2: Analyze JAR Dependencies

Using jdeps (Java Dependency Analysis Tool):

# Analyze what a JAR depends on
jdeps myapp.jar

# Show missing dependencies
jdeps -s myapp.jar

# Generate dot file for visualization
jdeps -dotoutput . myapp.jar

Tool 3: Maven Helper Commands

# Show dependency tree with conflicts
mvn dependency:tree -Dverbose

# Show effective POM (resolved dependencies)
mvn help:effective-pom

# Analyze dependencies
mvn dependency:analyze

# Copy all dependencies to a directory
mvn dependency:copy-dependencies -DoutputDirectory=./lib

Tool 4: Gradle Helper Commands

# Show all dependencies
./gradlew dependencies

# Show dependency insight (why a dependency is included)
./gradlew dependencyInsight --dependency hadoop-common

# Show build scan for web-based analysis
./gradlew build --scan

Advanced Tips

Tip 1: Use Smart Search Queries

When Googling errors, be specific:

❌ Bad:

"java error"

✅ Good:

"NoClassDefFoundError org.apache.hadoop.conf.Configurable"

✅ Better:

"NoClassDefFoundError hadoop Configurable flink"

✅ Best:

"apache flink hadoop-common missing dependency"

Tip 2: Check for Version Conflicts

Scenario: You have two versions of the same library:

lib/
  ├── hadoop-common-2.7.3.jar
  └── hadoop-common-3.3.0.jar

How to detect:

# Maven
mvn dependency:tree | grep -A 5 "conflicts"

# Gradle
./gradlew dependencies | grep "(*)"  # (*) indicates version conflict

How to fix in Gradle:

configurations.all {
    resolutionStrategy {
        // Force a specific version
        force 'org.apache.hadoop:hadoop-common:3.3.0'

        // Or fail on version conflict
        failOnVersionConflict()
    }
}

Tip 3: Understand Dependency Scopes

Different scopes affect when dependencies are available:

Scope	Compile Time	Runtime	Example Use Case
`compile` (Maven) / `implementation` (Gradle)	✓	✓	Your main dependencies
`provided` (Maven) / `compileOnly` (Gradle)	✓	✗	APIs provided by container
`runtime` (Maven) / `runtimeOnly` (Gradle)	✗	✓	Database drivers
`test` (Maven) / `testImplementation` (Gradle)	✓ (tests)	✓ (tests)	Testing libraries

Tip 4: Create a Dependency Report

Generate a comprehensive report:

# Maven
mvn project-info-reports:dependencies
# Opens in: target/site/dependencies.html

# Gradle
./gradlew htmlDependencyReport
# Opens in: build/reports/project/dependencies/index.html

Tip 5: Use IDE Features

Modern IDEs can help significantly:

IntelliJ IDEA:

Right-click on a class → "Go To" → "Declaration" (shows which JAR it's from)
View → Tool Windows → "Maven/Gradle" (visual dependency tree)
Ctrl+Shift+Alt+U on a class (shows full dependency diagram)

Eclipse:

Ctrl+Shift+T (Open Type) shows JAR location
Project → Properties → Java Build Path (view all JARs)
Dependency Hierarchy plugin

VS Code:

Java Dependency Viewer extension
Maven for Java extension
Gradle for Java extension

Practice Exercise

Here's an exercise to test your understanding. Given this error:

Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
    at com.example.MyApp.main(MyApp.java:15)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.ObjectMapper
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)

Your task:

What library is missing?
What Maven/Gradle dependency would you add?
How would you verify which version to use?
What commands would you run to debug this?

Click to see solution

1. Identify the library:

Package: com.fasterxml.jackson.databind
Library: Jackson Databind (JSON processing library)

2. Add dependency:

Maven:


    com.fasterxml.jackson.core
    jackson-databind
    2.15.2

Gradle:

implementation 'com.fasterxml.jackson.core:jackson-databind:2.15.2'

3. Verify version:

# Check what other libraries use
mvn dependency:tree | grep jackson

# Check Maven Central for latest
# Visit: https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind

4. Debug commands:

# Search for existing Jackson JARs
find . -name "*jackson*.jar"

# Check if class exists in a specific JAR
jar -tf jackson-databind-2.15.2.jar | grep ObjectMapper

# View current dependencies
mvn dependency:tree

Common Pitfalls and Solutions

Pitfall 1: "It Works on My Machine"

Problem: Application works locally but fails in production.

Cause: Different classpaths or dependency versions.

Solution:

# Package dependencies with your application
mvn package -DincludeDependencies=true

# Or create a fat JAR
mvn clean package assembly:single

Pitfall 2: Transitive Dependency Conflicts

Problem: Library A needs version 1.0, Library B needs version 2.0 of the same dependency.

Solution:

// Gradle - Exclude transitive dependency
implementation('com.example:library-a:1.0') {
    exclude group: 'com.conflicting', module: 'dependency'
}

// Then explicitly add the version you need
implementation 'com.conflicting:dependency:2.0'

Pitfall 3: Shaded JARs

Problem: Can't find a class even though the JAR is present.

Cause: Some libraries "shade" (rename) their dependencies to avoid conflicts.

Solution:

# Check for shaded classes
jar -tf myapp.jar | grep "shaded"

# Look for relocated packages
jar -tf myapp.jar | grep -E "shadow|shaded|relocated"

Checklist for Debugging ClassNotFoundException

Use this checklist whenever you encounter a ClassNotFoundException:

[ ] Identify the fully qualified class name from the error
[ ] Determine which library the class belongs to (from package name)
[ ] Check if the JAR is in your dependencies (mvn dependency:tree or ./gradlew dependencies)
[ ] Verify the JAR is in the runtime classpath (not just compile-time)
[ ] Check for version conflicts (grep "(*)" in Gradle output)
[ ] Ensure the class exists in the JAR version you have (jar -tf)
[ ] Look for shaded or relocated classes
[ ] Check if it's a provided/compileOnly dependency that needs to be changed
[ ] Verify no exclusions are removing the dependency
[ ] Test with a clean build (mvn clean install or ./gradlew clean build)

Resources

Official Documentation

Tools

Maven Repository - Find dependencies and versions
JitPack - Build GitHub projects as Maven artifacts
jdeps - Java dependency analysis tool

Learning Resources

Conclusion

Debugging ClassNotFoundException errors doesn't have to be mysterious. By following a systematic approach:

Read the stack trace bottom-up to find the root cause
Identify which library the missing class belongs to
Search for the JAR in your dependencies and filesystem
Verify the JAR contains the class you need
Fix by adding it to the correct classpath location

With practice, you'll be able to diagnose and fix these issues in minutes rather than hours. Remember: every dependency error is a puzzle with a logical solution. Keep calm, follow the process, and you'll find the answer.

Happy debugging! 🐛🔍

About This Guide

This guide was created to help Java beginners develop systematic debugging skills. If you found it helpful, please share it with others who might benefit. Feel free to adapt and expand upon these techniques in your own projects.

Questions or feedback? The best way to learn is through practice. Try the exercise above, and experiment with the tools and commands in your own projects.

CheatSheet: Python for DSA

itzsrikanth — Sat, 08 Nov 2025 10:36:35 +0000

Python DSA Cheatsheet - Complete Reference

Note: I took help of LLMs to arrange the structure and come up with few of the examples in the list below

Numeric Operations
List / Array Operations
String Operations
Dictionary / HashMap Operations
Set Operations
Heap / Priority Queue
Deque Operations
Collections Module
Itertools
Matrix Operations
Bit Manipulation
Binary Search
Common Patterns
Input/Output Tricks

Numeric Operations

Setting min / max

import math

cost = float('inf')  # Infinity
cost = -float('inf') # Negative infinity
cost = math.inf      # Alternative

Division Types

# Floor division
7 // 2  # Output: 3

# Regular division (returns float)
7 / 2   # Output: 3.5

# Ceiling division
import math
math.ceil(7 / 2)  # Output: 4

# Python's floor division with negative numbers
-7 // 2   # Output: -4 (rounds down/toward negative infinity)
int(-7 / 2)  # Output: -3 (truncates toward zero)

Modulo Operations

# Positive modulo
7 % 3   # Output: 1

# Negative modulo (Python always returns positive result)
-7 % 3  # Output: 2

# To get true modulo
((a % b) + b) % b

Power and Root

# Power
2 ** 10  # Output: 1024
pow(2, 10)  # Output: 1024

# Modular exponentiation (for large numbers)
pow(2, 10, 1000)  # Output: 24 (2^10 mod 1000)

# Square root
import math
math.sqrt(16)  # Output: 4.0
16 ** 0.5      # Output: 4.0

GCD and LCM

import math

# GCD
math.gcd(12, 18)  # Output: 6

# Multiple GCD
from functools import reduce
reduce(math.gcd, [12, 18, 24])  # Output: 6

# LCM
def lcm(a, b):
    return abs(a * b) // math.gcd(a, b)

# Multiple LCM
from functools import reduce
reduce(lcm, [12, 18, 24])

List / Array Operations

Initialization

# 1D array with zeros
arr = [0] * 5  # Output: [0, 0, 0, 0, 0]

# 2D array with zeros (CORRECT way)
matrix = [[0] * cols for _ in range(rows)]

# WRONG way (all rows reference same list!)
# matrix = [[0] * cols] * rows  # DON'T USE THIS

# Initialize with specific value
arr = [float('inf')] * 5

# List comprehension
squares = [x**2 for x in range(5)]  # Output: [0, 1, 4, 9, 16]

Insertion

arr = [1, 2, 3]

# At the end - O(1)
arr.append(4)  # [1, 2, 3, 4]

# At specific index - O(n)
arr.insert(1, 5)  # [1, 5, 2, 3, 4]

# Extend with another list
arr.extend([6, 7])  # [1, 5, 2, 3, 4, 6, 7]
arr += [8, 9]       # Alternative

Removal

arr = [1, 2, 3, 4, 5]

# Pop at the end - O(1)
last = arr.pop()  # Returns 5, arr = [1, 2, 3, 4]

# Pop at specific index - O(n)
first = arr.pop(0)  # Returns 1, arr = [2, 3, 4]

# Remove by value (first occurrence) - O(n)
arr.remove(3)  # arr = [2, 4]

# Clear all
arr.clear()  # arr = []

Slicing (IMPORTANT!)

arr = [0, 1, 2, 3, 4, 5]

# Basic slicing [start:end:step]
arr[1:4]      # [1, 2, 3] (end is exclusive)
arr[:3]       # [0, 1, 2] (from start)
arr[3:]       # [3, 4, 5] (to end)
arr[:]        # [0, 1, 2, 3, 4, 5] (copy entire list)

# Step
arr[::2]      # [0, 2, 4] (every 2nd element)
arr[1::2]     # [1, 3, 5] (every 2nd starting from index 1)

# Reverse
arr[::-1]     # [5, 4, 3, 2, 1, 0]
arr[-1::-1]   # Same as above

# Negative indices
arr[-1]       # 5 (last element)
arr[-3:]      # [3, 4, 5] (last 3 elements)
arr[:-2]      # [0, 1, 2, 3] (all except last 2)

Sorting

arr = [3, 1, 4, 1, 5, 9, 2]

# In-place sort - O(n log n)
arr.sort()  # arr is modified
arr.sort(reverse=True)  # Descending order

# Return new sorted list
sorted_arr = sorted(arr)  # arr is unchanged

# Custom sorting with key
students = [("Alice", 20), ("Bob", 18), ("Charlie", 22)]
sorted_students = sorted(students, key=lambda x: x[1])  # Sort by age

# Multi-level sorting
envelopes = [[5,4], [6,4], [6,7], [2,3]]
# Sort by x[0] ASC, then x[1] DESC
envelopes.sort(key=lambda x: (x[0], -x[1]))

# Sort with custom comparator (use functools.cmp_to_key)
from functools import cmp_to_key
def compare(a, b):
    return a - b  # Ascending
arr.sort(key=cmp_to_key(compare))

Binary Search (bisect module)

from bisect import bisect_left, bisect_right, insort

arr = [1, 2, 4, 4, 5, 7]

# bisect_left: leftmost insertion point
bisect_left(arr, 4)   # Output: 2 (index of first 4)
bisect_left(arr, 3)   # Output: 2 (would insert before 4)

# bisect_right (or bisect): rightmost insertion point
bisect_right(arr, 4)  # Output: 4 (after last 4)

# Insert and maintain sorted order
insort(arr, 3)  # arr = [1, 2, 3, 4, 4, 5, 7]

# Check if element exists
def exists(arr, x):
    i = bisect_left(arr, x)
    return i < len(arr) and arr[i] == x

List Comprehension and Filtering

# Basic list comprehension
squares = [x**2 for x in range(10)]

# With condition
evens = [x for x in range(10) if x % 2 == 0]

# Nested comprehension (flatten)
matrix = [[1, 2], [3, 4], [5, 6]]
flat = [val for row in matrix for val in row]  # [1, 2, 3, 4, 5, 6]

# Extract specific column from 2D list
arr = [["a", 1], ["b", 2], ["c", 3]]
second_col = [row[1] for row in arr]  # [1, 2, 3]

# Enumerate with comprehension
indexed = [(i, val) for i, val in enumerate(arr)]

Other Useful Operations

arr = [3, 1, 4, 1, 5, 9, 2, 6]

# Min/Max
min(arr)  # 1
max(arr)  # 9
min(arr, default=0)  # Returns 0 if arr is empty

# Sum
sum(arr)  # 31

# Count occurrences
arr.count(1)  # 2

# Find index of element
arr.index(4)  # 2 (first occurrence)

# Check existence
4 in arr  # True

# Reverse in-place
arr.reverse()

# Length
len(arr)  # 8

# All/Any
all([True, True, False])  # False
any([False, False, True])  # True

# Zip (combine multiple lists)
list(zip([1, 2, 3], ['a', 'b', 'c']))  # [(1, 'a'), (2, 'b'), (3, 'c')]

# Unzip
pairs = [(1, 'a'), (2, 'b'), (3, 'c')]
nums, chars = zip(*pairs)  # nums = (1, 2, 3), chars = ('a', 'b', 'c')

String Operations

Basic Operations

s = "hello world"

# Length
len(s)  # 11

# Access by index
s[0]    # 'h'
s[-1]   # 'd'

# Slicing (same as list)
s[0:5]  # "hello"
s[::-1] # "dlrow olleh" (reverse)

# Concatenation
s1 + " " + s2
"".join([s1, s2])  # More efficient for multiple strings

# Repeat
"abc" * 3  # "abcabcabc"

String Methods

s = "Hello World"

# Case conversion
s.lower()      # "hello world"
s.upper()      # "HELLO WORLD"
s.title()      # "Hello World"
s.capitalize() # "Hello world"
s.swapcase()   # "hELLO wORLD"

# Check properties
s.isalpha()    # False (has space)
s.isdigit()    # False
s.isalnum()    # False
s.isspace()    # False

# Strip whitespace
"  hello  ".strip()   # "hello"
"  hello  ".lstrip()  # "hello  "
"  hello  ".rstrip()  # "  hello"

# Split and Join
"a,b,c".split(',')    # ['a', 'b', 'c']
"a b c".split()       # ['a', 'b', 'c'] (splits on any whitespace)
','.join(['a', 'b'])  # "a,b"

# Replace
s.replace('World', 'Python')  # "Hello Python"

# Find and Index
s.find('o')     # 4 (first occurrence, returns -1 if not found)
s.index('o')    # 4 (raises exception if not found)
s.rfind('o')    # 7 (last occurrence)
s.count('l')    # 3

# Check prefix/suffix
s.startswith('Hello')  # True
s.endswith('World')    # True

String Building (Performance)

# SLOW - string concatenation in loop
result = ""
for c in chars:
    result += c  # Creates new string each time

# FAST - use list and join
result = ''.join(chars)

# For complex building
from io import StringIO
builder = StringIO()
builder.write("hello")
builder.write(" world")
result = builder.getvalue()  # "hello world"

Character Operations

# ASCII conversion
ord('A')    # 65
chr(65)     # 'A'

# Check if character
c = 'a'
c.isalpha()   # True
c.isdigit()   # False
c.islower()   # True
c.isupper()   # False

Dictionary / HashMap Operations

Initialization

# Empty dict
d = {}
d = dict()

# With initial values
d = {'a': 1, 'b': 2}
d = dict(a=1, b=2)
d = dict([('a', 1), ('b', 2)])

# From keys with default value
keys = ['a', 'b', 'c']
d = dict.fromkeys(keys, 0)  # {'a': 0, 'b': 0, 'c': 0}

Access and Modification

d = {'a': 1, 'b': 2}

# Access
d['a']           # 1 (raises KeyError if not exists)
d.get('a')       # 1 (returns None if not exists)
d.get('c', 0)    # 0 (custom default)

# Set
d['c'] = 3       # Add or update

# Delete
del d['a']
value = d.pop('b')         # Remove and return value
value = d.pop('x', None)   # Safe pop with default

# Check existence
'a' in d         # True
'a' not in d     # False

Iteration

d = {'a': 1, 'b': 2, 'c': 3}

# Keys
for key in d:           # Iterate over keys
    print(key)

for key in d.keys():    # Explicit
    print(key)

# Values
for val in d.values():
    print(val)

# Key-value pairs
for key, val in d.items():
    print(key, val)

Useful Operations

# Get all keys/values as list
list(d.keys())    # ['a', 'b', 'c']
list(d.values())  # [1, 2, 3]

# Update dictionary
d1 = {'a': 1, 'b': 2}
d2 = {'b': 3, 'c': 4}
d1.update(d2)     # d1 = {'a': 1, 'b': 3, 'c': 4}

# Merge dictionaries (Python 3.9+)
d3 = d1 | d2

# Copy
d_copy = d.copy()  # Shallow copy

# Clear
d.clear()

# Length
len(d)

# setdefault (get or set if not exists)
d.setdefault('d', 0)  # Returns 0 and sets d['d'] = 0 if not exists

Sorting Dictionary

d = {'b': 2, 'a': 1, 'c': 3}

# Sort by key
sorted_d = dict(sorted(d.items()))

# Sort by value
sorted_d = dict(sorted(d.items(), key=lambda x: x[1]))

# Sort by value (descending)
sorted_d = dict(sorted(d.items(), key=lambda x: x[1], reverse=True))

Set Operations

Initialization

# Empty set (NOT {})
s = set()

# With values
s = {1, 2, 3}
s = set([1, 2, 3])

Basic Operations

s = {1, 2, 3}

# Add
s.add(4)        # {1, 2, 3, 4}

# Remove
s.remove(2)     # Raises KeyError if not exists
s.discard(2)    # No error if not exists
s.pop()         # Remove and return arbitrary element

# Check membership
2 in s          # True
5 not in s      # True

# Length
len(s)          # 3

# Clear
s.clear()

Set Operations

s1 = {1, 2, 3, 4}
s2 = {3, 4, 5, 6}

# Union
s1 | s2                # {1, 2, 3, 4, 5, 6}
s1.union(s2)           # Same

# Intersection
s1 & s2                # {3, 4}
s1.intersection(s2)    # Same

# Difference
s1 - s2                # {1, 2} (in s1 but not s2)
s1.difference(s2)      # Same

# Symmetric difference (XOR)
s1 ^ s2                # {1, 2, 5, 6}
s1.symmetric_difference(s2)  # Same

# Subset/Superset
s1.issubset(s2)        # False
s1.issuperset(s2)      # False
s1.isdisjoint(s2)      # False

Heap / Priority Queue

import heapq

# Min heap by default
heap = []

# Push - O(log n)
heapq.heappush(heap, 3)
heapq.heappush(heap, 1)
heapq.heappush(heap, 4)

# Pop min - O(log n)
min_val = heapq.heappop(heap)  # 1

# Peek min - O(1)
min_val = heap[0]  # Don't pop

# Heapify existing list - O(n)
arr = [3, 1, 4, 1, 5, 9, 2]
heapq.heapify(arr)  # arr is now a heap

# Push and pop in one operation
heapq.heappushpop(heap, 5)  # Push 5, then pop min
heapq.heapreplace(heap, 5)  # Pop min, then push 5

# N largest/smallest elements
heapq.nlargest(3, arr)   # [9, 5, 4]
heapq.nsmallest(3, arr)  # [1, 1, 2]

# Max heap trick (negate values)
max_heap = []
heapq.heappush(max_heap, -3)
heapq.heappush(max_heap, -1)
max_val = -heapq.heappop(max_heap)  # 3

# Heap with custom objects
heap = []
heapq.heappush(heap, (priority, item))  # Use tuple

Deque Operations

from collections import deque

# Initialize
dq = deque()
dq = deque([1, 2, 3])
dq = deque([1, 2, 3], maxlen=5)  # Fixed size

# Append - O(1)
dq.append(4)        # Add to right: [1, 2, 3, 4]
dq.appendleft(0)    # Add to left: [0, 1, 2, 3, 4]

# Pop - O(1)
dq.pop()            # Remove from right, returns 4
dq.popleft()        # Remove from left, returns 0

# Extend - O(k)
dq.extend([5, 6])        # [1, 2, 3, 5, 6]
dq.extendleft([0, -1])   # [-1, 0, 1, 2, 3, 5, 6]

# Access
dq[0]              # First element
dq[-1]             # Last element

# Rotate
dq.rotate(1)       # Rotate right: [6, 1, 2, 3, 5]
dq.rotate(-1)      # Rotate left: [1, 2, 3, 5, 6]

# Other operations
dq.count(2)        # Count occurrences
dq.remove(3)       # Remove first occurrence
dq.reverse()       # Reverse in-place
dq.clear()         # Clear all
len(dq)            # Length

Collections Module

Counter

from collections import Counter

# Initialize
c = Counter([1, 2, 2, 3, 3, 3])  # Counter({3: 3, 2: 2, 1: 1})
c = Counter("hello")              # Counter({'l': 2, 'h': 1, 'e': 1, 'o': 1})

# Most common
c.most_common(2)   # [(3, 3), (2, 2)]

# Access counts
c[2]               # 2
c[5]               # 0 (no KeyError)

# Update
c.update([1, 1, 2])

# Subtract
c.subtract([1, 2])

# Arithmetic operations
c1 = Counter({'a': 3, 'b': 1})
c2 = Counter({'a': 1, 'b': 2})
c1 + c2            # Counter({'a': 4, 'b': 3})
c1 - c2            # Counter({'a': 2}) (keeps only positive)

# Convert to dict
dict(c)

defaultdict

from collections import defaultdict

# With int (default 0)
d = defaultdict(int)
d['a'] += 1        # No KeyError, starts at 0

# With list (default [])
d = defaultdict(list)
d['a'].append(1)   # No KeyError, starts with []

# With set
d = defaultdict(set)
d['a'].add(1)

# With lambda
d = defaultdict(lambda: float('inf'))

# Convert to normal dict
dict(d)

OrderedDict (less needed in Python 3.7+)

from collections import OrderedDict

# Maintains insertion order (dicts do this by default now)
d = OrderedDict()
d['a'] = 1
d['b'] = 2

# Move to end
d.move_to_end('a')  # Move 'a' to end

# Pop last
d.popitem(last=True)   # Pop from end
d.popitem(last=False)  # Pop from beginning

Itertools

from itertools import *

# Combinations
list(combinations([1, 2, 3], 2))  # [(1, 2), (1, 3), (2, 3)]

# Permutations
list(permutations([1, 2, 3], 2))  # [(1, 2), (1, 3), (2, 1), (2, 3), (3, 1), (3, 2)]

# Product (Cartesian product)
list(product([1, 2], ['a', 'b']))  # [(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')]

# Combinations with replacement
list(combinations_with_replacement([1, 2], 2))  # [(1, 1), (1, 2), (2, 2)]

# Chain (flatten)
list(chain([1, 2], [3, 4]))  # [1, 2, 3, 4]

# Accumulate (cumulative sum by default)
list(accumulate([1, 2, 3, 4]))  # [1, 3, 6, 10]
list(accumulate([1, 2, 3, 4], max))  # [1, 2, 3, 4] (running max)

# Groupby
data = [('a', 1), ('a', 2), ('b', 3), ('b', 4)]
for key, group in groupby(data, key=lambda x: x[0]):
    print(key, list(group))
# Output: a [('a', 1), ('a', 2)]
#         b [('b', 3), ('b', 4)]

# Cycle (infinite)
counter = 0
for item in cycle([1, 2, 3]):
    print(item)
    counter += 1
    if counter > 10:
        break

# Repeat
list(repeat(10, 3))  # [10, 10, 10]

# Islice (slice iterator)
list(islice(range(10), 2, 8, 2))  # [2, 4, 6]

# Pairwise (Python 3.10+)
list(pairwise([1, 2, 3, 4]))  # [(1, 2), (2, 3), (3, 4)]

Matrix Operations

# Initialize matrix
rows, cols = 3, 4
matrix = [[0] * cols for _ in range(rows)]

# Fill with specific value
matrix = [[1] * cols for _ in range(rows)]

# Traverse
for i in range(rows):
    for j in range(cols):
        print(matrix[i][j])

# Get dimensions
rows = len(matrix)
cols = len(matrix[0]) if matrix else 0

# Transpose
transposed = [[matrix[j][i] for j in range(rows)] for i in range(cols)]
# Or using zip
transposed = list(map(list, zip(*matrix)))

# Rotate 90 degrees clockwise
rotated = [[matrix[rows-1-j][i] for j in range(rows)] for i in range(cols)]

# Flatten
flat = [val for row in matrix for val in row]

# Diagonal traversal
for i in range(rows):
    print(matrix[i][i])  # Main diagonal

# 4-directional movement
directions = [(0, 1), (1, 0), (0, -1), (-1, 0)]  # right, down, left, up
for dx, dy in directions:
    new_x, new_y = x + dx, y + dy

# 8-directional movement
directions = [(-1,-1), (-1,0), (-1,1), (0,-1), (0,1), (1,-1), (1,0), (1,1)]

Bit Manipulation

# Basic operations
x = 5  # 101 in binary

# AND
x & 1     # Check if odd: 1 (True)
x & (x-1) # Remove rightmost set bit: 4 (100)

# OR
x | 1     # Set rightmost bit: 5

# XOR
x ^ 1     # Flip rightmost bit: 4
a ^ a     # Always 0
a ^ 0     # Always a

# NOT
~x        # Flip all bits: -6

# Left shift (multiply by 2)
x << 1    # 10 (1010)

# Right shift (divide by 2)
x >> 1    # 2 (10)

# Check if power of 2
(x & (x-1)) == 0 and x != 0

# Count set bits
bin(x).count('1')  # 2

# Get bit at position i
(x >> i) & 1

# Set bit at position i
x | (1 << i)

# Clear bit at position i
x & ~(1 << i)

# Toggle bit at position i
x ^ (1 << i)

# Get rightmost set bit
x & -x

# Common tricks
x & 1           # Check if odd
x & (x-1)       # Remove rightmost 1
x | (x+1)       # Set rightmost 0
x & -x          # Get rightmost 1

Binary Search

# Template 1: Find exact value
def binary_search(arr, target):
    left, right = 0, len(arr) - 1
    while left <= right:
        mid = left + (right - left) // 2
        if arr[mid] == target:
            return mid
        elif arr[mid] < target:
            left = mid + 1
        else:
            right = mid - 1
    return -1

# Template 2: Find leftmost position
def binary_search_left(arr, target):
    left, right = 0, len(arr)
    while left < right:
        mid = left + (right - left) // 2
        if arr[mid] < target:
            left = mid + 1
        else:
            right = mid
    return left

# Template 3: Find rightmost position
def binary_search_right(arr, target):
    left, right = 0, len(arr)
    while left < right:
        mid = left + (right - left) // 2
        if arr[mid] <= target:
            left = mid + 1
        else:
            right = mid
    return left - 1

# Using bisect module
from bisect import bisect_left, bisect_right

# Find leftmost >= target
idx = bisect_left(arr, target)

# Find rightmost <= target
idx = bisect_right(arr, target) - 1

Common Patterns

Sliding Window

# Fixed size window
def max_sum_subarray(arr, k):
    window_sum = sum(arr[:k])
    max_sum = window_sum

    for i in range(k, len(arr)):
        window_sum += arr[i] - arr[i-k]
        max_sum = max(max_sum, window_sum)

    return max_sum

# Variable size window
def longest_substring_k_distinct(s, k):
    left = 0
    max_len = 0
    char_count = {}

    for right in range(len(s)):
        char_count[s[right]] = char_count.get(s[right], 0) + 1

        while len(char_count) > k:
            char_count[s[left]] -= 1
            if char_count[s[left]] == 0:
                del char_count[s[left]]
            left += 1

        max_len = max(max_len, right - left + 1)

    return max_len

Two Pointers

# Opposite direction
def two_sum_sorted(arr, target):
    left, right = 0, len(arr) - 1
    while left < right:
        curr_sum = arr[left] + arr[right]
        if curr_sum == target:
            return [left, right]
        elif curr_sum < target:
            left += 1
        else:
            right -= 1
    return [-1, -1]

# Same direction
def remove_duplicates(arr):
    if not arr:
        return 0

    slow = 0
    for fast in range(1, len(arr)):
        if arr[fast] != arr[slow]:
            slow += 1
            arr[slow] = arr[fast]

    return slow + 1

Prefix Sum

# 1D prefix sum
def prefix_sum(arr):
    prefix = [0]
    for num in arr:
        prefix.append(prefix[-1] + num)
    return prefix

# Range sum query
prefix = prefix_sum(arr)
range_sum = prefix[right+1] - prefix[left]

# 2D prefix sum
def build_prefix_2d(matrix):
    rows, cols = len(matrix), len(matrix[0])
    prefix = [[0] * (cols+1) for _ in range(rows+1)]

    for i in range(1, rows+1):
        for j in range(1, cols+1):
            prefix[i][j] = (matrix[i-1][j-1] + 
                           prefix[i-1][j] + 
                           prefix[i][j-1] - 
                           prefix[i-1][j-1])
    return prefix

# Range sum 2D
def range_sum_2d(prefix, r1, c1, r2, c2):
    return (prefix[r2+1][c2+1] - 
            prefix[r1][c2+1] - 
            prefix[r2+1][c1] + 
            prefix[r1][c1])

Fast/Slow Pointers (Cycle Detection)

# Floyd's cycle detection
def has_cycle(head):
    slow = fast = head
    while fast and fast.next:
        slow = slow.next
        fast = fast.next.next
        if slow == fast:
            return True
    return False

# Find cycle start
def detect_cycle(head):
    slow = fast = head
    while fast and fast.next:
        slow = slow.next
        fast = fast.next.next
        if slow == fast:
            slow = head
            while slow != fast:
                slow = slow.next
                fast = fast.next
            return slow
    return None

Monotonic Stack

# Next greater element
def next_greater_elements(arr):
    n = len(arr)
    result = [-1] * n
    stack = []  # Store indices

    for i in range(n):
        while stack and arr[stack[-1]] < arr[i]:
            result[stack.pop()] = arr[i]
        stack.append(i)

    return result

# Previous smaller element
def previous_smaller_elements(arr):
    n = len(arr)
    result = [-1] * n
    stack = []

    for i in range(n):
        while stack and arr[stack[-1]] >= arr[i]:
            stack.pop()
        if stack:
            result[i] = arr[stack[-1]]
        stack.append(i)

    return result

Input/Output Tricks

# Fast input (for competitive programming)
import sys
input = sys.stdin.readline

# Read single integer
n = int(input())

# Read multiple integers from one line
a, b, c = map(int, input().split())

# Read list of integers
arr = list(map(int, input().split()))

# Read multiple lines
lines = sys.stdin.readlines()

# Fast output
print(*arr)  # Prints list elements separated by space

# Format output
print(f"{value:.2f}")  # 2 decimal places
print(f"{value:05d}")  # Pad with zeros to width 5

# Multiple test cases
t = int(input())
for _ in range(t):
    # Solve each test case
    pass

# EOF handling
try:
    while True:
        line = input()
        # Process line
except EOFError:
    pass

Miscellaneous

Lambda Functions

# Basic lambda
square = lambda x: x**2

# Multiple arguments
add = lambda x, y: x + y

# With conditional
max_fn = lambda x, y: x if x > y else y

# In sorting
arr.sort(key=lambda x: (x[0], -x[1]))

# In filter
evens = list(filter(lambda x: x % 2 == 0, arr))

# In map
squares = list(map(lambda x: x**2, arr))

Functools

from functools import reduce, lru_cache, cmp_to_key

# Reduce
product = reduce(lambda x, y: x * y, [1, 2, 3, 4])  # 24

# LRU Cache (memoization)
@lru_cache(maxsize=None)
def fib(n):
    if n <= 1:
        return n
    return fib(n-1) + fib(n-2)

# Custom comparator
def compare(a, b):
    return a - b
arr.sort(key=cmp_to_key(compare))

Try-Except

# Basic
try:
    result = 10 / 0
except ZeroDivisionError:
    print("Cannot divide by zero")

# Multiple exceptions
try:
    # code
except (ValueError, TypeError) as e:
    print(e)

# Catch all
try:
    # code
except Exception as e:
    print(e)

# Finally
try:
    # code
finally:
    # Always executes
    pass

Quick Select Algorithm

itzsrikanth — Wed, 01 Oct 2025 17:53:29 +0000

I would not take full credits for this writing this code by hand, some of it goes to the Leetcode discussion section and "Claude Sonnet 4 Thinking" model. Purpose of this series is to document for my learning and if it proves to be useful for others.

There are two popular parition schemes to find the pivot:

hoare
lomuto

Lomute partition scheme

def partition(self, left: int, right: int):

        # choose rightmost element (not the index) as the pivot
        pivot = self.arr[right]

        # boundary pointer - tracks the end of the "≤ pivot" region
        i = left

        # scanner pointer - explores the array from left to right
        for j in range(left, right):
            if self.arr[j] <= pivot:
                self.arr[i], self.arr[j] = self.arr[j], self.arr[i]
                i += 1
        self.arr[i], self.arr[right] = self.arr[right], self.arr[i]
        return i

The idea behind i and j pointer is that the former needs to hold the place for numbers smaller than pivot point to be place. It is moved ahead only when we are going to attempt a swap. The latter would scan the elements looking for a lower value to swap.

Taking the example from geeksforgeeks:

Array: [10, 4, 5, 8, 6, 11, 26]  pivot = 26
       left=0, right=6

Initial: [10, 4, 5, 8, 6, 11, 26]  i=0
         ^i                    ^pivot

j=0: arr[0]=10 ≤ 26? Yes → swap(i=0,j=0), i=1
     [10, 4, 5, 8, 6, 11, 26]  i=1
         ^i

j=1: arr[1]=4 ≤ 26? Yes → swap(i=1,j=1), i=2
     [10, 4, 5, 8, 6, 11, 26]  i=2
            ^i

j=2: arr[2]=5 ≤ 26? Yes → swap(i=2,j=2), i=3
     [10, 4, 5, 8, 6, 11, 26]  i=3
               ^i

...continuing this pattern...

Final: [10, 4, 5, 8, 6, 11, 26]  i=6
                           ^i

Place pivot: swap(i=6, right=6) → no change
Return i=6

The goal is to rearrange array so that:

all elements ≤ pivot are on the left, and
all elements > pivot are on the right, with pivot in its correct final position.

[ ≤ pivot | > pivot | unprocessed | pivot]
left  i-1   i    j-1  j    right-1  right

main

class Solution:
    def __init__(self, arr: List[int]):
        self.arr = arr

    def partition(self, left: int, right: int):
        # ...

    def quickselect(self, left: int, right: int, k: int) -> int:
        if left == right:
            return self.arr[left]

        pivot_index = self.partition(left, right)

        if k == pivot_index - left + 1:
            return self.arr[pivot_index]
        elif k < pivot_index - left + 1:
            return self.quickselect(left, pivot_index - 1, k)
        else:
            return self.quickselect(pivot_index + 1, right, k - (pivot_index - left + 1))

References

https://www.geeksforgeeks.org/dsa/quickselect-algorithm/

Comment system: API Gateway & DynamoDB Integration in Pulumi

itzsrikanth — Wed, 24 Jul 2024 05:26:22 +0000

Inspired by Alex DeBrie's tweet to use API Gateway with DynamoDB proxy integration, I built a Comment system in Pulumi.

Background

The idea is to create a comment system in AWS by skipping the Lambda function and using API Gateway and DynamoDB integration. This is possible by using the AWS_PROXY integration type in API Gateway. This integration type allows you to pass the request directly to the backend service without any transformation.

We assume to have a site with 100 pages and each page has 10 recent comments. So, we would create a DynamoDB table which has slug as Hash key and createdAt as Range key. The slug would be the page URL and createdAt would be the timestamp when the comment was created. This way, we can query the table to get the 10 most recent comments for a given page.

For simplicity, we are going to skip the user table etc.**

Pulumi Code

Create a DynamoDB table to store comments:

// Create a DynamoDB table to store comments
const commentsTable = new aws.dynamodb.Table("CommentsTable", {
    attributes: [
        { name: "slug", type: "S" },
        { name: "createdAt", type: "N" },
    ],
    hashKey: "slug",
    rangeKey: "createdAt",
    billingMode: "PROVISIONED",
    readCapacity: 2,
    writeCapacity: 1,
});

We need to define attributes for the fields on which we would query the table. In this case, we are querying based on slug and createdAt.

DB Provisioning and cost calculation:

Next, the billingMode is set to PROVISIONED. We can arrive at the readCapacity and writeCapacity based on the expected traffic. For that we need have some rough assumptions. In this case,

we are assuming a site deployed with ISR (Incremental Static Regeneration) and the pages are cached in CDN (Cloudfront) for a day. So, the traffic would be 10 pages x 1 read query per page.
for simplicity, the queries will not be parallel. We can distribute the reads over 60 seconds.
no additional writes are happening during the build process.

100 reads / 60 seconds ≈ 1.67 reads/second

Once we have our stack running, these values can be adjusted based on the actual traffic.

So the cost will be:

2 RCUs * $0.00013 per RCU-hour = $0.00026 per hour
and,
Monthly cost = $0.00026 * 24 * 30 = $0.1872

The cost is based on the us-east-1 region. The cost may vary based on the region.

Similar calculation would go in for writeCapacity. For now, we will take it as 1 WCU.

Create a Rest API Gateway:

RestApi Resource is the top-level container for your API. It defines the basic information about your API, such as its name and description.

// Create an API Gateway Rest API
const api = new aws.apigateway.RestApi("Backend", {
    name: "Backend",
}, {
    dependsOn: commentsTable,
});

Once we have the API Gateway, we need to create a resource and method to handle the POST request. The Resource resource is a logical container for the resources that make up your API. In terms of URL path, it corresponds to a collection of API resources. Multiple resources can be created to represent different parts of your API's URL structure. For our case, we would create a resource for comments under the Backend API Gateway.

// Create a resource for comments
const commentsResource = new aws.apigateway.Resource("CommentsResource", {
    parentId: api.rootResourceId,
    pathPart: "comments",
    restApi: api.id,    // <--- Reference to the Rest API
}, {
    dependsOn: api,     // <--- Make sure the API is created first
});

Similarly, we need to create a method to handle the POST request. The Method resource defines the HTTP method that clients can use to interact with your API. For our case, we would create a POST method under the comments resource.

// Create a method for POST requests to /comments
const commentsMethod = new aws.apigateway.Method("CommentsMethod", {
    restApi: api.id,
    resourceId: commentsResource.id,    // <--- Reference to the comments resource
    httpMethod: "POST",
    authorization: "NONE",
}, {
    dependsOn: commentsResource,
});

Integration with DynamoDB:

Specifying a role is crucial for security and access control while integrating API Gateway with AWS services using the AWS_PROXY integration type. The role specified in the integration is assumed by API Gateway, not the AWS service being called. This role includes permissions that API Gateway needs to invoke actions on the integrated AWS service.with other AWS services.

This approach follows the principle of least privilege, where the role grants only the necessary permissions for API Gateway to perform specific actions. This minimizes the risk of unauthorized access or actions within your AWS environment.

const apigatewayRole = new aws.iam.Role("ApigatewayRole", {
    assumeRolePolicy: {
        Version: "2012-10-17",
        Statement: [{
            Action: "sts:AssumeRole",
            Effect: "Allow",
            Principal: {
                Service: "apigateway.amazonaws.com",
            },
        }],
    },
});

// Create an integration with DynamoDB
const commentsIntegration = new aws.apigateway.Integration("DynamodbIntegration", {
    restApi: api.id,
    resourceId: commentsResource.id,
    httpMethod: commentsMethod.httpMethod,
    type: "AWS",
    integrationHttpMethod: "POST",
    credentials: apigatewayRole.arn,
    uri: `arn:aws:apigateway:${currentRegion}:dynamodb:action/PutItem`,
    requestTemplates: {
        "application/json": pulumi.interpolate`{
            "TableName": "${commentsTable.name}",
            "Item": {
                "commentId": {
                    "S": "$context.requestId"
                },
                "comment": {
                    "S": "$input.body"
                }
            }
        }`,
    },
    passthroughBehavior: "WHEN_NO_MATCH",
}, {
    dependsOn: [commentsMethod, apigatewayRole],
});

The Integration resource defines how API Gateway integrates with other AWS services. In this case, we are integrating with DynamoDB. The uri property specifies the ARN of the service action to be called. The requestTemplates property specifies the mapping templates to transform the request body into the format expected by the backend service.

Deploy the stack:

// Deploy the API
const deployment = new aws.apigateway.Deployment("ApiDeployment", {
    restApi: api.id,
    stageName: "v1",
}, {
    dependsOn: commentsIntegration,
});

Since we would require the recent comments, we need to use ScanIndexForward: false in the query along with Limit: 10.

Checkout the repo for full code.

My Next.Js configuration

itzsrikanth — Sat, 13 Jul 2024 05:59:25 +0000

Generally while creating a content-heavy Next.Js project, I prefer to have it initialized with a few params like below:

npx create-next-app --ts \
--tailwind \
--eslint \
--app --src-dir \
--import-alias \
--use-yarn \
-e app-dir-mdx <app_name>

PostCSS: Processor and Plugin module

itzsrikanth — Mon, 10 Jan 2022 01:44:21 +0000

Processor
LazyResult
Result
Plugins
Warning
Other Entities

Processor

Creates a Processor instance. This instance can be used on multiple Stylesheets. postcss default export function returns a Processor instance. The main purpose of this module is to process the plugins, which is called Normalization here, add to the plugins data member for storage and return a LazyResult promise.

Data members

plugins - `Array<Plugin|Transformer|TransformCallback>`

stores a list of plugins added to this Processor instance.

Methods

use

takes in a plugin to append to the list of plugins and returns instance of Processor.

process

Parses CSS (can be passed in as various types) to return LazyResult promise.

normalize

Takes in Array< plugin > are argument. For each of the plugin, the following steps are done:

Normalization involves a preprocessing step where
- if postcss === true, it is executed immediately as function without argument.
- if postcss contains any other truthy value, then the object is destructured to set postcss as top level value.
Next step involves added the plugins to normalized list.
- if plugin is object
  - and plugin.plugins is Array, the plugins are merged to the normalized plugin list.
  - and plugin.postcssPlugin is truthy, it is appended to normalized plugin list.
- type 'function' directs becomes eligible to be added in normalized list.
Other types are naturally eligible to enter the error conditions.

LazyResult

constructor

Takes in CSS either as string or already parsed Node|Root, to pass Root to new instance of Result. Along with Result, helpers and plugins are also added as data members to this.

Result

This is a very simple data structure to store fields (like a POJO):

processor
Array<Warning> as messages
root
css
map
opts Main processing logic is present in other modules. Here, both content and toString returns the same Result.css.

Plugins

Extends from Processors. If they are async, then Promise must be returned.

prepare

takes in Result and returns Processors

Warning

A simple structure that is used to get debug info i.e. source and location of the error and the plugin that is causing this. It has got a self-explanatory toString method.

Other Entities

AcceptedPlugin

Processor takes an Array of AcceptedPlugin which can be of type:

Plugin
PluginCreator<any>
OldPlugin<any>
TransformCallback
{postcss: TransformCallback|Processor}
Processor - The current Processor instance will copy this Processor's plugins

PostCSS: Input and other modules source code walk-through

itzsrikanth — Sun, 09 Jan 2022 02:40:59 +0000

Terminal Highlight

Purpose of this module is to enable syntax highlight while printing the stringified CSS (after transformation) to the terminal. picocolors is used for this (similar to chalk npm package). For each token, a specific color is defined as a map.

getTokenType

When the type is word, it is very generic and we need to classify them as hash or class. Using Tokenizer, forward lookup is done to detect bracket or parentheses.

terminalHighlight

Get Processor instance from Tokenizer initialization and until EOF is reached, wrap all the tokens using picocolors by splitting them by newline character.

Input

This takes in CSS as String type as first argument. The main purpose of this entity is to store CSS, origin, sourcemap and to provide associated functionalities listed below. The steps to initialize are:

Error handling
BOM check and handling
When from is specified, it is check if value is a network path or absolute path. Else it is resolved using path.
When from is not specified, id is generated using nanoid
Source map added if available, generated from compilation step before PostCSS kicks in.

fromOffset

Purpose of this method is to return line and column number when offset is provided.

CSS string is split by \n and stored as Array<string>.
The no of characters in each line is counted incrementally (including the newline), say, if there are 10 lines with each line having 20 characters. The corresponding array created is:

lineToIndex = [20, 40, 60, ... 200]

This is stored as cache with Symbol('fromOffsetCache'). Using this the line number and column number is determined in O(log n) complexity similar to searching sorted Array.

error

Returns CssSyntaxError
Parameters are:

error message (string)
any one of:
- both line and column as {offset?:number; line:number; col:number}
- line (number) and column (null)
opts ({plugins: any})
If offset is present, FilePosition is determined using fromOffset().

Steps:

generate the start and end line-column numbers from parameters (which is FilePosition) and pass to CssSyntaxError to get result.
Add {...FilePosition, source, url} to result

origin

return false if
- not having map, or
- map not having source
get sourcemap consumer to get the original FilePosition data before previous stage compilation
Add url, source (sourceContent) from consumer to FilePosition

mapResolve

Takes in file path and checks if it has protocol like file:// or http://. Else, it resolves path from map source.

from (getter)

Returns file or nanoid generated id

toJSON

Adds the following keys:

hasBOM
css
file
id
map and remove the map.consumerCache

PostCSS: Architecture overview

itzsrikanth — Sun, 09 Jan 2022 02:40:32 +0000

I am a big fan of PostCSS and this is my attempt to understand this package to become an active contributor. I am planning to write a series involving architecture, code walk-through for this amazing package and its top plugins.

Before diving into very low level details, reading this architecture is a must. Almost all of the low level detailed info are available in API page with multiple theoretical example, generated from static typing jsDoc. The intention of this series is to provide them in digestible and pragmatic way. For that many references from PostCSS plugins will be drawn. This will get extended by diving deep into the codebase as well. As mentioned in the arch doc, the AST conversion is split into. two parts: tokenization and parsing for performance reasons. Lets get started with the basics:

Everything is a Node here. The statement is too generic but what it means is the CSS is converted into a tree structure. Which means that many tokens in stylesheet are converted into Node to form the hierarchical structure. To illustrate this, lets take an example:

.active {
  transform: scale(1.1);
}

Here, this stylesheet is stored as a series of characters. They are read character by character and converted into tokens. The tokens are then parsed by the parser and converted into the AST structure, which sets the stage for all the plugins to work their magic. So, this is a 2-stage process.

Lets understand the above code structure after tokenization:

.active
{
transform
:
scale(1.1)
;
}

It is converted into 7 tokens. These tokens are consumed by Parser using nextToken() in Tokenizer.

Web Accessibility learned, the hard way...

itzsrikanth — Tue, 04 Jan 2022 08:17:55 +0000

This post is going to be my ever updating journal to record many learnings which I learned by implementation, from peers, some hints from blogs, sites etc. Main intent is to record everything that is not available in stackoverflow and links for everything available in stackoverflow for reference on a11y.

Screen readers used:

Voiceover (iOSv15)
Android Talkback

Cookbook recipes

List item element receiving focus, announcing "bullet"

CSS can help fix this issue by saying list-style-type: none

Link vs Button on pages

When we have a user action causing redirection/navigation to another URL, the role should be link. But in cases where the user actions is causing pop-ups etc, it should be button.

Using flexbox to prevent screen-reader focus

I have not come across any articles explaining this, but when we have the children as flexbox, the screen-reader just focuses on parent container. If you have aria-label on the parent to flexbox, it reads it. Else falls back to reading the contents combined.

Removing `tabindex` for talkback to prevent focus

In Android, when we have tabindex set, it becomes eligible for receiving focus, irrespective of its value. So, its text contents are read. Removal of that attribute prevents the focus.

Announcing status or toast updates

using HTML attributes

This method involves creating a live-region and setting politeness for announcing the alerts.

using JS

This itself involves having multiple approaches. One of them is to create a singleton placeholder and push content inside this region. The attributes like aria-hidden, aria-live etc needs to be set and this can be unset after some fixed/dynamic timeout duration.

Focus elements on page load in voiceover in Javascript

This is the best I have come across to achieve this. Tried using various focusInterval with setTimeout instead of setInterval but faced certain drawbacks each time.
When 30ms is used with setTimeout, focus was missing for some cases. With 50ms, the default focus announces an element followed by shift in focus to read the current one.

Understanding screen reader behaviours

`tabindex` interpretation

In the codepen above, various scenarios of tabindex are added on both parent and child. In voiceover the tabindex attributes are not considered much and screen-reader focuses mainly on presence of text/content rather than the attribute. Whereas for talkback, based on how we specify the tabindex the behaviour changes drastically. Example:

When we have tabindex only on parent: the focus does not come to the child element since the parent already has the attribute.
When both parent and child has the attribute: the focus comes to both of them and inner content read repetition happens.
When there in attribute only on child: it is equivalent of no attribute because the child being a text node, it will naturally receive the focus.
No attribute anywhere: same as case 3

Generic list of dev features required to bootstrap a great frontend project

itzsrikanth — Sun, 02 Jan 2022 07:04:14 +0000

Bootstrapping a new project from scratch or creating a generic boilerplate code is not an easy task when seen from a long time duration standpoint. The perspective and requirements might change rapidly if not analyzed minutely. I am listing here few of the implementations that I have included in many of my official and pet-projects:

CSS implementations

There are a variety of solutions available for adding stylings to your project including the new way of CSS-in-JS. But each has their own pros and cons. They must chosen based on the type of project. I will include more elaboration on my next post but in short, as an example, the requirements of a website is very different from that of a WebApp or a hybrid mobile application.

In websites, the metrics like TTFB, FCP, CLS etc plays a vital role (source) in not only in enhancing the user experience but also is a big contributor towards the terms like SEO, marketing expenditure, churn rate and ultimately in overall revenue. So, JS executions might take a lot of time to reflect the styles when compared to CSS counterparts. We might not benefit from CSS-in-JS solutions here unless it is extracted as separate stylesheet using some plugins like MiniCssExtractPlugin.

The same metrics are important for WebApp as well but no web-crawler is going to penalize your SEO value or others. But once your app warms up, you need to have reactive performance and a great frame rate, silky smooth navigations etc similar to a native app.

CSS Modules

This brings in best of both worlds and could be very subjective. But I am happy that I do not have to spend effort in thinking of a long class name based on BEM, SMACSS etc.

JS implementations

A global loader for all API/Ajax calls

It should be initialized as a singleton and a counter should be provisioned in such a way that when no of pending calls is zero, it is hidden. In other cases it should be visible

Common Modals / Drawer widgets

Project configuration

Linters

Code linting
Commit message linting like commitizen

In case of multi-package project usage of tools

like:

lerna
rush

Managing environment configs using

dotenv

Compiling a Node.js module into a single file

This is not a mandatory process, but for some cases this can help avoid the npm modules installation step by converting all the node_modules dependencies into a single JS file.

Project automation packages

New to Java, my journey so far...

itzsrikanth — Sun, 21 Nov 2021 16:06:13 +0000

Introduction

Talked to few senior backend engineers before my dive into Java. One valuable suggestion was to learn Class, Abstract class and Interface. Some of good articles are:

Dev blog posts that helped me in frontend journey

itzsrikanth — Sun, 17 Oct 2021 10:29:57 +0000

There any many useful resources and some awesome guides like roadmap.sh which gives good perspective of where we stand. Apart from those there are few other blogs that I would like to highlight, like:

Theodo blog about JS library tree shaking and related module concept.
Josh Comeau's blog on z-index stacking introduced me to his CSS courses.
michaelmang
Guide to help choose frameworks | kruschecompany.com
usehooks.com
Understanding the core JavaScript | dmitrysoshnikov.com
https://www.patterns.dev/

React Native

React Native - Beyond Basics | kadikraman.github.io

Webpack

Node v8 deep-dive webpack 5 fix | engineering.tines.com

Mobile browsers

MDN | Detect tab switches and browser app sleep/awake cycle

Build tools

Broccoli blog gives overall architecture and comparison with gulp and grunt.
Compiling to JavaScript, and Debugging with Source Maps, along with Source Map Revision 3 Proposal
New insights on modern frameworks, 2017

DEV Community: itzsrikanth

Debugging Java ClassNotFoundException: A Beginner's Guide to Finding Missing Dependencies

Introduction

Table of Contents

The Problem: Understanding the Error

The Debugging Process

Step 1: Read the Stack Trace Bottom-Up

Step 2: Identify the Library

Step 3: Understand the Context

Step 4: Search for the JAR

Step 5: Verify the JAR Contains the Missing Class

Step 6: Fix the Issue

Common Java Dependency Errors

Step-by-Step Framework

1. Identify the Missing Class

2. Search Your Dependencies

3. Check JAR Contents

4. Verify Runtime Classpath

5. Compare Compile vs Runtime Dependencies

Practical Tools and Commands

Tool 1: Finding Classes in JARs

Tool 2: Analyze JAR Dependencies

Tool 3: Maven Helper Commands

Tool 4: Gradle Helper Commands

Advanced Tips

Tip 1: Use Smart Search Queries

Tip 2: Check for Version Conflicts

Tip 3: Understand Dependency Scopes

Tip 4: Create a Dependency Report

Tip 5: Use IDE Features

Practice Exercise

Common Pitfalls and Solutions

Pitfall 1: "It Works on My Machine"

Pitfall 2: Transitive Dependency Conflicts

Pitfall 3: Shaded JARs

Checklist for Debugging ClassNotFoundException

Resources

Official Documentation

Tools

Learning Resources

Conclusion

About This Guide

CheatSheet: Python for DSA

Python DSA Cheatsheet - Complete Reference

Table of Contents

Numeric Operations

Setting min / max

Division Types

Modulo Operations

Power and Root

GCD and LCM

List / Array Operations

Initialization

Insertion

Removal

Slicing (IMPORTANT!)

Sorting

Binary Search (bisect module)

List Comprehension and Filtering

Other Useful Operations

String Operations

Basic Operations

String Methods

String Building (Performance)

Character Operations

Dictionary / HashMap Operations

Initialization

Access and Modification

Iteration

Useful Operations

Sorting Dictionary

Set Operations

Initialization

Basic Operations

Set Operations

Heap / Priority Queue

Deque Operations

Collections Module

Counter

defaultdict

main

plugins - `Array<Plugin|Transformer|TransformCallback>`

Removing `tabindex` for talkback to prevent focus

`tabindex` interpretation