DEV Community: Nitesh Thakur

Gauss Elimination:

Nitesh Thakur — Fri, 05 Jun 2026 16:01:59 +0000

Solving Linear Equations with Gauss Elimination: A Complete Guide with C Code

Have you ever wondered how engineering software solves systems of 50 equations with 50 unknowns in milliseconds? The answer at the heart of most linear solvers is the Gauss Elimination Method — one of the most important algorithms in numerical computing.

Today I'll break it down from scratch: the concept, the algorithm, a working C implementation, and a real example with output.

The Problem We're Solving

Suppose you have a system of linear equations like this:

2x₁ +  x₂ − x₃ =  8
−3x₁ − x₂ + 2x₃ = −11
−2x₁ + x₂ + 2x₃ = −3

You need to find the values of x₁, x₂, and x₃ that satisfy all three equations simultaneously.

For 2 or 3 variables, you could solve by hand using substitution. But what about 10 variables? Or 100? That's where Gauss Elimination comes in.

The Core Idea

Gauss Elimination works in two phases:

Phase 1: Forward Elimination

Transform the system so that each equation has one fewer variable than the one above it — an upper triangular form.

Original:               After elimination:
[ 2   1  -1 |  8 ]     [ 2   1  -1 |  8  ]
[-3  -1   2 | -11]  →  [ 0  0.5 0.5|  1  ]
[-2   1   2 | -3 ]     [ 0   0   1 | -1  ]

Phase 2: Back Substitution

Once the matrix is triangular, solve from the bottom up:

Last row gives x₃ directly
Second-to-last gives x₂ using known x₃
And so on upward

The Algorithm (Step by Step)

INPUT:  n equations, augmented matrix [A|b]
OUTPUT: solution x[0], x[1], ..., x[n-1]

FORWARD ELIMINATION:
For k = 0 to n-2:
    If A[k][k] == 0 → zero pivot, STOP
    For i = k+1 to n-1:
        factor = A[i][k] / A[k][k]
        For j = k to n:
            A[i][j] = A[i][j] - factor × A[k][j]

BACK SUBSTITUTION:
x[n-1] = A[n-1][n] / A[n-1][n-1]
For i = n-2 down to 0:
    sum = A[i][n] - Σ (A[i][j] × x[j])
    x[i] = sum / A[i][i]

The key operation is computing a factor (the multiplier) for each row below the pivot, then using it to zero out the entry in the pivot column.

C Implementation

#include <stdio.h>
#include <math.h>

#define MAX 10

int main() {
    float a[MAX][MAX+1], x[MAX];
    int n, i, j, k;
    float factor, sum;

    printf("Gauss Elimination Method\n");
    printf("========================\n");
    printf("Enter number of equations: ");
    scanf("%d", &n);

    /* Input the augmented matrix */
    printf("Enter augmented matrix [A|b] row by row:\n");
    for (i = 0; i < n; i++) {
        printf("Row %d: ", i+1);
        for (j = 0; j <= n; j++)
            scanf("%f", &a[i][j]);
    }

    /* ---- Forward Elimination ---- */
    for (k = 0; k < n-1; k++) {
        if (fabs(a[k][k]) < 1e-9) {
            printf("Zero pivot encountered. Method fails!\n");
            return 1;
        }
        for (i = k+1; i < n; i++) {
            factor = a[i][k] / a[k][k];
            for (j = k; j <= n; j++)
                a[i][j] -= factor * a[k][j];
        }
    }

    /* ---- Back Substitution ---- */
    x[n-1] = a[n-1][n] / a[n-1][n-1];
    for (i = n-2; i >= 0; i--) {
        sum = a[i][n];
        for (j = i+1; j < n; j++)
            sum -= a[i][j] * x[j];
        x[i] = sum / a[i][i];
    }

    /* ---- Display solution ---- */
    printf("\nSolution:\n");
    for (i = 0; i < n; i++)
        printf("x[%d] = %.4f\n", i+1, x[i]);

    return 0;
}

Running the Example

Compile and run:

gcc gauss_elimination.c -o gauss_elimination -lm
./gauss_elimination

Enter the augmented matrix for our system:

Enter number of equations: 3
Row 1: 2  1  -1  8
Row 2: -3  -1  2  -11
Row 3: -2  1  2  -3

Output:

Solution:
x[1] =  2.0000
x[2] =  3.0000
x[3] = -1.0000

Let's verify: 2(2) + (3) − (−1) = 4 + 3 + 1 = 8 ✓

What is the Augmented Matrix?

Instead of writing the equations separately, we combine the coefficients and constants into one matrix:

2x₁ + x₂ - x₃ = 8   becomes row → [ 2   1  -1 |  8 ]

The vertical bar separates the left-hand side coefficients from the right-hand side constants. This compact form makes it easy to apply row operations systematically.

Why Check for Zero Pivot?

The factor is computed as A[i][k] / A[k][k]. If A[k][k] is zero, we get a division-by-zero error. This is called a zero pivot and means the method fails in its basic form.

The fix is partial pivoting — before each elimination step, swap the current row with the row below that has the largest absolute value in the pivot column. This both avoids division by zero and reduces numerical error.

Complexity

The algorithm runs in O(n³) time — three nested loops. This is fine for small-to-medium systems (up to a few hundred equations). For very large systems (thousands of variables), iterative methods or sparse solvers are preferred.

Advantages

Produces an exact solution (not iterative approximation)
Works for any n×n non-singular system
Foundation for LU decomposition, which is used in almost all professional linear algebra libraries

Limitations

Fails with zero pivot without partial pivoting
Numerical instability can accumulate for large n
O(n³) makes it slow for very large systems

Key Takeaway

Gauss Elimination is the foundational algorithm behind solving linear systems. Every time you use NumPy's linalg.solve(), MATLAB's backslash operator, or any finite element solver — there's a descendant of Gauss Elimination working under the hood.

Understanding it at this level gives you a solid foundation for numerical analysis, scientific computing, and engineering simulations.

Newton-Raphson Method

Nitesh Thakur — Fri, 05 Jun 2026 15:00:02 +0000

Understanding the Newton-Raphson Method: Fast Root Finding with C

If you've ever wondered how calculators find square roots, or how engineering software solves complex equations in milliseconds — the answer often involves a technique called the Newton-Raphson Method.

Today I'll walk you through the concept, the algorithm, and a working C implementation — all from scratch.

What Problem Does It Solve?

Given a function f(x), we want to find a value of x where f(x) = 0 — called a root.

For example: what is the root of x³ - 2x - 5 = 0?

We can't solve this algebraically with a simple formula. This is where numerical methods come in.

The Core Idea

Imagine you're standing on a curve and you want to find where it crosses zero. Newton-Raphson says:

"Draw a tangent line at your current point. Where does that tangent line hit zero? That's your next guess."

Mathematically, the formula is:

x₁ = x₀ - f(x₀) / f'(x₀)

Where:

x₀ is your current guess
f(x₀) is the function value at that guess
f'(x₀) is the derivative (slope) at that point
x₁ is your improved guess

You repeat this process until the change between guesses is smaller than your desired tolerance.

The Algorithm (Step by Step)

INPUT:  f(x), f'(x), initial guess x₀, tolerance ε, max iterations N

Step 1: Set iteration counter i = 0
Step 2: WHILE i < N:
   2.1  Calculate f(xᵢ) and f'(xᵢ)
   2.2  IF |f'(xᵢ)| < ε → derivative too small, STOP
   2.3  xᵢ₊₁ = xᵢ - f(xᵢ) / f'(xᵢ)
   2.4  error = |xᵢ₊₁ - xᵢ|
   2.5  IF error < ε → ROOT FOUND, STOP
   2.6  Set xᵢ = xᵢ₊₁, increment i
Step 3: IF i = N → "Max iterations reached"

C Implementation

#include <stdio.h>
#include <math.h>
#include <stdlib.h>

#define MAX_ITERATIONS 100
#define EPSILON 0.00001

// Function: x^3 - 2x - 5
double f(double x) {
    return x*x*x - 2*x - 5;
}

// Derivative: 3x^2 - 2
double f_prime(double x) {
    return 3*x*x - 2;
}

int main() {
    double x0, x1, error;
    int iteration = 0;

    printf("Newton-Raphson Method\n");
    printf("Enter initial guess: ");
    scanf("%lf", &x0);

    printf("\nIteration\t x0\t\t f(x0)\t\t x1\t\t Error\n");
    printf("----------------------------------------------------------------\n");

    do {
        if (fabs(f_prime(x0)) < EPSILON) {
            printf("Derivative too small. Method fails!\n");
            exit(1);
        }

        x1    = x0 - f(x0) / f_prime(x0);
        error = fabs(x1 - x0);

        printf("%d\t\t %.6f\t %.6f\t %.6f\t %.6f\n",
               iteration + 1, x0, f(x0), x1, error);

        x0 = x1;
        iteration++;

        if (iteration > MAX_ITERATIONS) {
            printf("Max iterations reached.\n");
            break;
        }

    } while (error > EPSILON);

    printf("\nRoot = %.6f\n", x1);
    printf("f(root) = %.6f\n", f(x1));
    printf("Iterations: %d\n", iteration);

    return 0;
}

Sample Output

Starting with initial guess x₀ = 2:

Iteration    x0          f(x0)       x1          Error
----------------------------------------------------------------
1            2.000000    -1.000000   2.100000    0.100000
2            2.100000     0.061000   2.094568    0.005432
3            2.094568     0.000202   2.094551    0.000017
4            2.094551     0.000000   2.094551    0.000000

Root = 2.094551
f(root) = 0.000000
Iterations: 4

Only 4 iterations to find the root with 6 decimal places of accuracy. That's the power of Newton-Raphson.

Why Is It So Fast?

Newton-Raphson has quadratic convergence — meaning the number of correct decimal places roughly doubles with every iteration. Compare this to bisection method which adds about one correct bit per iteration (linear convergence).

When It Can Fail

f'(x) = 0 at some point — division by zero, method breaks
Poor initial guess — may overshoot and diverge
Functions with multiple roots — may converge to a different root than expected

Key Takeaway

Newton-Raphson is one of the most elegant algorithms in numerical analysis. It combines simplicity, speed, and practicality — and it's a must-know for any CS or engineering student.

Try changing the function f(x) and f_prime(x) in the code above to solve your own equations!

Full source code and README available on GitHub: https://github.com/nt3005156/Numerical-Method-All

Tags: #numericalMethods #cprogramming #algorithms #computerScience #mathematics