DEV Community

itzsrikanth
itzsrikanth

Posted on

Debugging Java ClassNotFoundException: A Beginner's Guide to Finding Missing Dependencies

#java #tutorial #beginners

Introduction

If you've ever encountered a ClassNotFoundException or NoClassDefFoundError in Java, you know how frustrating it can be. Your code compiled fine, but at runtime, Java can't find a class it needs. This guide will teach you a systematic approach to debugging these issues, using a real-world Apache Flink example.

In this tutorial, we'll walk through diagnosing and fixing a missing Hadoop dependency in a Flink application. More importantly, you'll learn a repeatable framework you can apply to any Java dependency issue.

Table of Contents

The Problem: Understanding the Error

Let's start with the error message we encountered:

Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configurable
    at java.base/java.lang.ClassLoader.defineClass1(Native Method)
    at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
    ...
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configurable
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
    ...
Enter fullscreen mode Exit fullscreen mode

At first glance, this looks intimidating. But let's break it down systematically.

The Debugging Process

Step 1: Read the Stack Trace Bottom-Up

Key Principle: Always start with the deepest "Caused by" clause. This is your root cause.

Main Exception: ProgramInvocationException
    ↓
Caused by: TableException
    ↓
Caused by: ServiceConfigurationError
    ↓
Caused by: NoClassDefFoundError         ← Near root cause
    ↓
Caused by: ClassNotFoundException       ← ROOT CAUSE! Start here!
Enter fullscreen mode Exit fullscreen mode

In our case, the root cause is:

ClassNotFoundException: org.apache.hadoop.conf.Configurable
Enter fullscreen mode Exit fullscreen mode

What this tells us:

  • Java can't find the class Configurable
  • It's in package org.apache.hadoop.conf
  • This is a Hadoop class

Step 2: Identify the Library

From the fully qualified class name, you can identify which library it belongs to:

org.apache.hadoop.conf.Configurable
└──────┬─────┘ └──┬──┘ └────┬────┘
   Vendor    Module  Class
Enter fullscreen mode Exit fullscreen mode

Common package patterns:

org.apache.flink.*         → Apache Flink
org.apache.hadoop.*        → Apache Hadoop  
org.apache.kafka.*         → Apache Kafka
com.fasterxml.jackson.*    → Jackson JSON library
org.springframework.*      → Spring Framework
com.google.gson.*          → Google Gson
Enter fullscreen mode Exit fullscreen mode

Step 3: Understand the Context

Look at where the error occurred in your stack trace:

at com.example.MyTableFactory.<init>(MyTableFactory.java:127)
Enter fullscreen mode Exit fullscreen mode

This tells us:

  • The MyTableFactory class is being initialized
  • It requires Hadoop classes to work
  • The Hadoop library is missing from the classpath

Step 4: Search for the JAR

Now we need to find where the missing class should be. Let's search systematically:

# Search in your application directory
find . -name "*.jar" -exec grep -l "org/apache/hadoop" {} \;

# For Apache Flink specifically, check both lib and opt directories
ls build/flink/lib/ | grep -i hadoop
ls build/flink/opt/ | grep -i hadoop
Enter fullscreen mode Exit fullscreen mode

Why check both lib/ and opt/?

  • lib/ → JARs loaded automatically by the framework
  • opt/ → Optional JARs that must be explicitly enabled

In our case, we found:

$ ls build/flink/opt/ | grep -i hadoop

flink-s3-fs-hadoop-1.16.1.jar       ← Contains Hadoop classes!
flink-azure-fs-hadoop-1.16.1.jar
flink-gs-fs-hadoop-1.16.1.jar
Enter fullscreen mode Exit fullscreen mode

Step 5: Verify the JAR Contains the Missing Class

Before proceeding, let's confirm this JAR actually contains the class we need:

# Method 1: Using unzip
unzip -l flink-s3-fs-hadoop-1.16.1.jar | grep "org/apache/hadoop/conf/Configurable"

# Method 2: Using jar command  
jar -tf flink-s3-fs-hadoop-1.16.1.jar | grep "Configurable"
Enter fullscreen mode Exit fullscreen mode

Output:

717  2025-09-27 03:45   org/apache/hadoop/conf/Configurable.class  ✓ Found it!
Enter fullscreen mode Exit fullscreen mode

Step 6: Fix the Issue

Now that we've identified the problem and the solution, we can fix it:

# Copy the JAR from opt/ to lib/ so it's loaded automatically
cp build/flink/opt/flink-s3-fs-hadoop-1.16.1.jar build/flink/lib/
Enter fullscreen mode Exit fullscreen mode

Alternative solutions depending on your setup:

  • Add the JAR to your application's classpath
  • Include it as a runtime dependency in Maven/Gradle
  • Set the CLASSPATH environment variable
  • Use the -cp flag when running Java

Common Java Dependency Errors

Understanding these common errors will help you debug faster:

Error Meaning Common Causes Solution
ClassNotFoundException Java can't find the .class file at runtime Missing JAR in classpath Add the JAR containing the class
NoClassDefFoundError Class was available at compile time but missing at runtime Missing dependency JAR, or initialization failure Check runtime classpath; verify static initializers
NoSuchMethodError Method exists but with different signature Version conflict between JARs Check for duplicate dependencies with different versions
LinkageError Class loaded by incompatible classloaders Duplicate JARs in classpath Remove duplicate JARs; use dependency management
IncompatibleClassChangeError Class structure changed between compile and runtime Mismatched library versions Ensure compile and runtime versions match

Step-by-Step Framework

Here's a repeatable process for debugging any ClassNotFoundException:

1. Identify the Missing Class 

ClassNotFoundException: com.example.MyClass
                        └──────┬──────┘
                           Package name
Enter fullscreen mode Exit fullscreen mode

Extract:

  • Package: com.example
  • Class: MyClass
  • Likely library: Based on package naming convention

2. Search Your Dependencies

For Maven projects:

# View all dependencies
mvn dependency:tree

# Search for specific dependency
mvn dependency:tree | grep -i "hadoop"
Enter fullscreen mode Exit fullscreen mode

For Gradle projects:

# View all dependencies
./gradlew dependencies

# View specific configuration
./gradlew dependencies --configuration runtimeClasspath
Enter fullscreen mode Exit fullscreen mode

3. Check JAR Contents

Quick check if a JAR contains a class:

# Using jar command (shows all files)
jar -tf myapp.jar | grep "MyClass"

# Using unzip (more detailed)
unzip -l myapp.jar | grep "com/example/MyClass"

# Using zipgrep (searches file contents)
zipgrep "MyClass" myapp.jar
Enter fullscreen mode Exit fullscreen mode

4. Verify Runtime Classpath

Add temporary debugging code:

import java.net.URL;
import java.net.URLClassLoader;

public class ClasspathDebugger {
    public static void main(String[] args) {
        ClassLoader cl = ClassLoader.getSystemClassLoader();
        URL[] urls = ((URLClassLoader)cl).getURLs();

        System.out.println("Current Classpath:");
        for(URL url: urls) {
            System.out.println("  " + url.getFile());
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Or use JVM flags:

# Shows all classes being loaded and their source
java -verbose:class -jar myapp.jar

# Shows classpath being used
java -XX:+TraceClassPaths -jar myapp.jar
Enter fullscreen mode Exit fullscreen mode

5. Compare Compile vs Runtime Dependencies

This is a common source of issues:

dependencies {
    // Compile-time only (not included in runtime)
    compileOnly 'org.apache.hadoop:hadoop-common:3.3.0'

    // Runtime dependency (included when running)
    runtimeOnly 'org.apache.hadoop:hadoop-common:3.3.0'

    // Both compile and runtime (recommended)
    implementation 'org.apache.hadoop:hadoop-common:3.3.0'
}
Enter fullscreen mode Exit fullscreen mode

Practical Tools and Commands

Tool 1: Finding Classes in JARs

Script to search all JARs for a class:

#!/bin/bash
# find-class.sh - Search for a class in all JARs

CLASS_NAME=$1
SEARCH_DIR=${2:-.}

echo "Searching for $CLASS_NAME in $SEARCH_DIR..."

find "$SEARCH_DIR" -name "*.jar" | while read jar; do
    if jar -tf "$jar" | grep -q "$CLASS_NAME"; then
        echo "Found in: $jar"
    fi
done
Enter fullscreen mode Exit fullscreen mode

Usage:

chmod +x find-class.sh
./find-class.sh "org/apache/hadoop/conf/Configurable" ./lib
Enter fullscreen mode Exit fullscreen mode

Tool 2: Analyze JAR Dependencies

Using jdeps (Java Dependency Analysis Tool):

# Analyze what a JAR depends on
jdeps myapp.jar

# Show missing dependencies
jdeps -s myapp.jar

# Generate dot file for visualization
jdeps -dotoutput . myapp.jar
Enter fullscreen mode Exit fullscreen mode

Tool 3: Maven Helper Commands 

# Show dependency tree with conflicts
mvn dependency:tree -Dverbose

# Show effective POM (resolved dependencies)
mvn help:effective-pom

# Analyze dependencies
mvn dependency:analyze

# Copy all dependencies to a directory
mvn dependency:copy-dependencies -DoutputDirectory=./lib
Enter fullscreen mode Exit fullscreen mode

Tool 4: Gradle Helper Commands 

# Show all dependencies
./gradlew dependencies

# Show dependency insight (why a dependency is included)
./gradlew dependencyInsight --dependency hadoop-common

# Show build scan for web-based analysis
./gradlew build --scan
Enter fullscreen mode Exit fullscreen mode

Advanced Tips

Tip 1: Use Smart Search Queries

When Googling errors, be specific:

❌ Bad:

"java error"
Enter fullscreen mode Exit fullscreen mode

✅ Good:

"NoClassDefFoundError org.apache.hadoop.conf.Configurable"
Enter fullscreen mode Exit fullscreen mode

✅ Better:

"NoClassDefFoundError hadoop Configurable flink"
Enter fullscreen mode Exit fullscreen mode

✅ Best:

"apache flink hadoop-common missing dependency"
Enter fullscreen mode Exit fullscreen mode

Tip 2: Check for Version Conflicts

Scenario: You have two versions of the same library:

lib/
  ├── hadoop-common-2.7.3.jar
  └── hadoop-common-3.3.0.jar
Enter fullscreen mode Exit fullscreen mode

How to detect:

# Maven
mvn dependency:tree | grep -A 5 "conflicts"

# Gradle
./gradlew dependencies | grep "(*)"  # (*) indicates version conflict
Enter fullscreen mode Exit fullscreen mode

How to fix in Gradle:

configurations.all {
    resolutionStrategy {
        // Force a specific version
        force 'org.apache.hadoop:hadoop-common:3.3.0'

        // Or fail on version conflict
        failOnVersionConflict()
    }
}
Enter fullscreen mode Exit fullscreen mode

Tip 3: Understand Dependency Scopes

Different scopes affect when dependencies are available:

Scope Compile Time Runtime Example Use Case
compile (Maven) / implementation (Gradle) ✓ ✓ Your main dependencies
provided (Maven) / compileOnly (Gradle) ✓ ✗ APIs provided by container
runtime (Maven) / runtimeOnly (Gradle) ✗ ✓ Database drivers
test (Maven) / testImplementation (Gradle) ✓ (tests) ✓ (tests) Testing libraries

Tip 4: Create a Dependency Report

Generate a comprehensive report:

# Maven
mvn project-info-reports:dependencies
# Opens in: target/site/dependencies.html

# Gradle
./gradlew htmlDependencyReport
# Opens in: build/reports/project/dependencies/index.html
Enter fullscreen mode Exit fullscreen mode

Tip 5: Use IDE Features

Modern IDEs can help significantly:

IntelliJ IDEA:

  • Right-click on a class → "Go To" → "Declaration" (shows which JAR it's from)
  • View → Tool Windows → "Maven/Gradle" (visual dependency tree)
  • Ctrl+Shift+Alt+U on a class (shows full dependency diagram)

Eclipse:

  • Ctrl+Shift+T (Open Type) shows JAR location
  • Project → Properties → Java Build Path (view all JARs)
  • Dependency Hierarchy plugin

VS Code:

  • Java Dependency Viewer extension
  • Maven for Java extension
  • Gradle for Java extension

Practice Exercise

Here's an exercise to test your understanding. Given this error:

Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
    at com.example.MyApp.main(MyApp.java:15)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.ObjectMapper
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
Enter fullscreen mode Exit fullscreen mode

Your task:

  1. What library is missing?
  2. What Maven/Gradle dependency would you add?
  3. How would you verify which version to use?
  4. What commands would you run to debug this?

Click to see solution

1. Identify the library:

  • Package: com.fasterxml.jackson.databind
  • Library: Jackson Databind (JSON processing library)

2. Add dependency:

Maven:


    com.fasterxml.jackson.core
    jackson-databind
    2.15.2
Enter fullscreen mode Exit fullscreen mode

Gradle:

implementation 'com.fasterxml.jackson.core:jackson-databind:2.15.2'
Enter fullscreen mode Exit fullscreen mode

3. Verify version:

# Check what other libraries use
mvn dependency:tree | grep jackson

# Check Maven Central for latest
# Visit: https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind
Enter fullscreen mode Exit fullscreen mode

4. Debug commands:

# Search for existing Jackson JARs
find . -name "*jackson*.jar"

# Check if class exists in a specific JAR
jar -tf jackson-databind-2.15.2.jar | grep ObjectMapper

# View current dependencies
mvn dependency:tree
Enter fullscreen mode Exit fullscreen mode

Common Pitfalls and Solutions

Pitfall 1: "It Works on My Machine"

Problem: Application works locally but fails in production.

Cause: Different classpaths or dependency versions.

Solution:

# Package dependencies with your application
mvn package -DincludeDependencies=true

# Or create a fat JAR
mvn clean package assembly:single
Enter fullscreen mode Exit fullscreen mode

Pitfall 2: Transitive Dependency Conflicts

Problem: Library A needs version 1.0, Library B needs version 2.0 of the same dependency.

Solution:

// Gradle - Exclude transitive dependency
implementation('com.example:library-a:1.0') {
    exclude group: 'com.conflicting', module: 'dependency'
}

// Then explicitly add the version you need
implementation 'com.conflicting:dependency:2.0'
Enter fullscreen mode Exit fullscreen mode

Pitfall 3: Shaded JARs

Problem: Can't find a class even though the JAR is present.

Cause: Some libraries "shade" (rename) their dependencies to avoid conflicts.

Solution:

# Check for shaded classes
jar -tf myapp.jar | grep "shaded"

# Look for relocated packages
jar -tf myapp.jar | grep -E "shadow|shaded|relocated"
Enter fullscreen mode Exit fullscreen mode

Checklist for Debugging ClassNotFoundException

Use this checklist whenever you encounter a ClassNotFoundException:

  • [ ] Identify the fully qualified class name from the error
  • [ ] Determine which library the class belongs to (from package name)
  • [ ] Check if the JAR is in your dependencies (mvn dependency:tree or ./gradlew dependencies)
  • [ ] Verify the JAR is in the runtime classpath (not just compile-time)
  • [ ] Check for version conflicts (grep "(*)" in Gradle output)
  • [ ] Ensure the class exists in the JAR version you have (jar -tf)
  • [ ] Look for shaded or relocated classes
  • [ ] Check if it's a provided/compileOnly dependency that needs to be changed
  • [ ] Verify no exclusions are removing the dependency
  • [ ] Test with a clean build (mvn clean install or ./gradlew clean build)

Resources

Official Documentation

Tools

  • Maven Repository - Find dependencies and versions
  • JitPack - Build GitHub projects as Maven artifacts
  • jdeps - Java dependency analysis tool

Learning Resources

Conclusion

Debugging ClassNotFoundException errors doesn't have to be mysterious. By following a systematic approach:

  1. Read the stack trace bottom-up to find the root cause
  2. Identify which library the missing class belongs to
  3. Search for the JAR in your dependencies and filesystem
  4. Verify the JAR contains the class you need
  5. Fix by adding it to the correct classpath location

With practice, you'll be able to diagnose and fix these issues in minutes rather than hours. Remember: every dependency error is a puzzle with a logical solution. Keep calm, follow the process, and you'll find the answer.

Happy debugging! 🐛🔍

About This Guide

This guide was created to help Java beginners develop systematic debugging skills. If you found it helpful, please share it with others who might benefit. Feel free to adapt and expand upon these techniques in your own projects.

Questions or feedback? The best way to learn is through practice. Try the exercise above, and experiment with the tools and commands in your own projects.

Top comments (0)