From Lab to Code #4: Exploring Neural Network Efficiency with Python and PyTorch

A larger neural network can often achieve better accuracy.

But does that always make it the better model?

When working with neural networks, it is easy to focus on a single metric: accuracy. We train a model, evaluate it on a test dataset, and compare the percentage of correct predictions.

However, this leaves out an important engineering question:

How much computation do we need to achieve that accuracy?

A small model might be slightly less accurate but require significantly fewer operations. Depending on the application, that trade-off can matter more than achieving the highest possible score.

For the fourth project in my From Lab to Code series, I wanted to explore this relationship by building an interactive environment for experimenting with convolutional neural networks.

The result is Neural Network Efficiency Lab, a Python and PyTorch application that compares three independently trained CNN models, visualizes their predictions, and shows how architectural decisions affect computational cost.

Rather than building another digit-recognition demo, the goal was to investigate a more interesting question:

How small can a neural network be while still delivering useful classification accuracy?

The Idea: Accuracy Is Only Half the Story

For this experiment, I used the MNIST handwritten-digit dataset.

MNIST contains grayscale images of handwritten digits from 0 to 9. Each image has a resolution of 28 × 28 pixels.

It is a relatively simple classification problem, which makes it useful for comparing small neural networks without requiring a large training infrastructure.

I designed three convolutional neural networks with increasing capacity:

• Micro CNN: A very small network designed for a restricted computation budget.
• Compact CNN: A moderately sized network with more convolutional channels.
• Wide CNN: A larger network intended to investigate the accuracy gained from additional computation.

All three use the same basic architecture, dataset, and training procedure.

The primary difference is the number of convolutional channels.

This makes it easier to observe how increasing model capacity affects both accuracy and computational requirements.

The project also includes an interactive interface where users can select trained models, draw digits, inspect predictions, visualize feature maps, and experiment with hypothetical CNN architectures.

---

1. Understanding the CNN Architecture

A convolutional neural network processes an image by applying learned filters to local regions.

Unlike a fully connected layer, where every output depends on every input, a convolutional layer reuses small filters across different image positions.

These filters can learn patterns such as edges, strokes, or more complex arrangements of local features.

For this project, I implemented a deliberately small architecture:

Input Image
   1 × 28 × 28
        |
        v
Convolution 1
        |
       ReLU
        |
     MaxPool
        |
        v
Convolution 2
        |
       ReLU
        |
     MaxPool
        |
        v
      Flatten
        |
        v
   Linear Layer
        |
        v
    10 Logits

The two convolutional layers extract features, while max pooling reduces the spatial resolution.

Finally, a fully connected layer maps the extracted representation to ten output values, one for each digit.

The implementation uses PyTorch:

class DigitCNN(nn.Module):
    def __init__(self, config):
        super().__init__()

        k = config.kernel

        self.conv1 = nn.Conv2d(
            1, config.c1, k, padding=k // 2
        )
        self.pool1 = nn.MaxPool2d(2)

        self.conv2 = nn.Conv2d(
            config.c1, config.c2, k, padding=k // 2
        )
        self.pool2 = nn.MaxPool2d(2)

        self.head = nn.Linear(
            config.c2 * 7 * 7, 10
        )

    def forward(self, x):
        x = self.pool1(
            torch.relu(self.conv1(x))
        )
        x = self.pool2(
            torch.relu(self.conv2(x))
        )

        return self.head(torch.flatten(x, 1))

The output consists of ten logits, which are unnormalized classification scores.

During training, these logits are passed to PyTorch's CrossEntropyLoss.

During inference, softmax converts them into a probability distribution over the ten classes:

P(class i) = exp(logit_i) / Σ exp(logit_j)

The predicted digit is the class with the highest probability.

These probabilities are useful for inspecting the model's predictions, although they should not automatically be interpreted as perfectly calibrated confidence estimates.

Why Change the Number of Channels?

The number of convolutional channels determines how many filters a layer learns.

A layer with more channels can represent a larger collection of features.

However, more channels also mean additional learned parameters and arithmetic operations.

This creates an interesting trade-off.

A larger network may learn more useful features and improve classification accuracy, but the improvement might be relatively small compared with the additional computation.

To investigate this, I kept the architecture structure consistent while changing its channel widths.

---

2. Measuring Computational Cost with MACs

Accuracy is relatively easy to understand.

We count the number of correct predictions and divide by the total number of evaluated examples:

Accuracy = Correct Predictions / Total Predictions

But how should we measure the computational cost of a neural network?

One useful approximation is the number of multiply-accumulate operations, or MACs.

A multiply-accumulate operation combines multiplication and addition:

accumulator += weight × input

Neural networks perform large numbers of these operations in convolutional and fully connected layers.

Counting MACs provides a hardware-independent estimate of some of the arithmetic work required for a forward pass.

Calculating Convolutional MACs

For a standard convolutional layer, the MAC count depends on the output dimensions, input channels, output channels, and kernel size.

A simplified formula is:

MACs = H_out × W_out × C_out × C_in × K²

where:

• H_out and W_out are the output feature-map dimensions.
• C_in is the number of input channels.
• C_out is the number of output channels.
• K is the width of a square convolution kernel.

In our architecture, the input has one channel and a resolution of 28 × 28.

The first convolution preserves the spatial resolution through padding.

After the first pooling operation, the image representation becomes 14 × 14.

After the second pooling operation, it becomes 7 × 7.

For convolutional channel counts C1 and C2, and kernel size K, the estimated costs are:

Conv1 = 28 × 28 × C1 × K²

Conv2 = 14 × 14 × C1 × C2 × K²

Dense = 7 × 7 × C2 × 10

Total MACs = Conv1 + Conv2 + Dense

I implemented this directly in Python:

def mac_breakdown(cfg):
    k2 = cfg.kernel ** 2

    conv1 = 28 * 28 * cfg.c1 * k2
    conv2 = 14 * 14 * cfg.c1 * cfg.c2 * k2
    linear = 7 * 7 * cfg.c2 * 10

    return {
        "conv1": conv1,
        "conv2": conv2,
        "linear": linear,
        "total": conv1 + conv2 + linear,
    }

This makes it possible to calculate the theoretical arithmetic cost of a model without executing it.

An Important Limitation

MACs are not the same as inference latency.

The calculation does not include operations such as ReLU, pooling, softmax, memory access, or data movement.

It also does not account for hardware-specific optimizations or parallel execution.

Consequently, a model with fewer MACs is not guaranteed to be faster on every processor or GPU.

For this project, MACs provide a transparent way to compare architectural complexity, not a benchmark of actual deployment performance.

---

3. Three Models, One Classification Task

With the architecture and cost calculation in place, I created three CNN configurations.

Model	Conv Channels	Parameters	MACs / Image
Micro CNN	3 → 6	3,148	55,860
Compact CNN	8 → 16	9,098	290,080
Wide CNN	16 → 32	20,490	1,031,744

All three use 3 × 3 convolution kernels.

The difference in computational cost is substantial.

The Wide CNN requires approximately 18.5 times as many MACs as the Micro CNN.

But does it achieve 18.5 times better classification performance?

Of course, accuracy cannot scale in that way. Once a model is already performing well, each additional improvement becomes harder to achieve.

The more useful question is how much accuracy we gain in exchange for that increased cost.

To answer it, the models need to be trained and evaluated under comparable conditions.

---

4. Training and Evaluating the Models

For a fair comparison, I used the same training procedure for all three architectures.

The official MNIST dataset was divided into:

Split	Images	Purpose
Training	54,000	Learn model parameters
Validation	6,000	Select the best training checkpoint
Test	10,000	Final evaluation

The validation set was created from the official MNIST training split.

The official test set was kept separate and used only for the final evaluation.

This distinction matters because selecting a model based on its test accuracy would introduce information leakage into the evaluation process.

Training Configuration

The models were trained using:

• Optimizer: Adam
• Learning rate: 0.001
• Batch size: 256
• Loss function: Cross-entropy
• Training duration: 4 epochs
• Random seed: 2026

For each model, I selected the checkpoint with the highest validation accuracy and then evaluated it on the official 10,000-image test set.

The implementation uses the standard PyTorch training sequence:

optimizer.zero_grad(set_to_none=True)

logits = model(images)

loss = criterion(logits, labels)

loss.backward()

optimizer.step()

Cross-entropy provides a differentiable objective, while backpropagation calculates gradients for the trainable parameters.

Adam uses these gradients to update the network weights.

The key benefit of keeping the training procedure consistent is that we can focus on how changes to the architecture affect the observed results.

However, the experiment uses only one training run per architecture. It is not a statistical comparison across multiple random seeds.

---

5. Experimental Results: More Computation, Smaller Gains

After training, the three models achieved the following results:

Model	Test Accuracy	MACs / Image
Micro CNN	96.67%	55,860
Compact CNN	97.92%	290,080
Wide CNN	98.63%	1,031,744

All accuracy values come from the held-out official MNIST test set.

These results reveal the central trade-off behind the project.

Micro vs. Compact

Moving from Micro to Compact increases the cost from 55,860 to 290,080 MACs.

That is approximately 5.2 times the computation.

The test accuracy improves from 96.67% to 97.92% — a gain of 1.25 percentage points.

Micro vs. Wide

Moving from Micro to Wide requires approximately 18.5 times the computation.

Yet accuracy improves by only 1.96 percentage points, from 96.67% to 98.63%.

This does not mean the larger model is useless.

For some applications, reducing the error rate can be extremely valuable, even if the improvement in accuracy appears small.

But it does show why accuracy should not be considered in isolation.

The best model depends on the available resources and the requirements of the application.

A Constrained-Compute Experiment

To make the trade-off more concrete, I introduced an illustrative budget of 75,000 MACs per image.

Under this constraint:

• Micro CNN fits within the budget.
• Compact CNN exceeds the budget.
• Wide CNN exceeds the budget by a much larger margin.

The Micro CNN achieves 96.67% test accuracy while requiring only 55,860 MACs.

In this particular experiment, it provides a strong result under the selected arithmetic budget.

The 75,000-MAC threshold is an educational design constraint rather than a limit derived from real hardware.

The experiment demonstrates how a computational budget can influence architecture selection.

It does not establish which network would be fastest on a particular CPU, GPU, or embedded device.

---

6. Making the Experiments Interactive

After training the models, I wanted to make the results easier to explore.

A table of numbers can show that one network is larger than another.

But an interactive application can help explain what those differences mean.

For the frontend, I created a browser interface connected to a local Python backend using FastAPI.

The backend loads the saved PyTorch model checkpoints and performs inference.

The browser displays the predictions, model metrics, and visualizations.

Comparing the Models

The first part of the application compares Micro, Compact, and Wide CNN.

Users can inspect each model's:

• Test accuracy.
• Number of parameters.
• Estimated MAC count.
• Architecture configuration.

An accuracy-versus-computation visualization makes the trade-off easier to see.

Instead of considering accuracy and model size separately, users can observe how much additional computation is associated with each improvement.

Drawing a Digit

The application also includes an interactive drawing canvas.

Users can draw a handwritten digit using their mouse and send it to one of the trained models.

The image is converted into a 28 × 28 grayscale representation and normalized using the same convention as the training data.

The selected model produces ten logits.

Softmax converts them into class probabilities, allowing the interface to display the predicted digit and its probability distribution.

The application also includes twenty attributed MNIST test examples for trying the models on actual dataset samples.

An interesting limitation appears when using the drawing canvas.

Handwritten digits produced with a mouse can look quite different from images in MNIST.

Differences in stroke width, positioning, and drawing style can affect the prediction.

A high accuracy on the MNIST test dataset therefore does not guarantee equally high accuracy on arbitrary user drawings.

This is a small example of distribution shift: a model is evaluated on inputs that differ from those it encountered during training.

Looking Inside the CNN

I also wanted to show something beyond the final classification result.

What does a convolutional layer actually produce?

To investigate this, the application extracts intermediate feature maps from the first two convolutional layers.

For a selected input, users can inspect visualizations of the learned activations.

The model returns both its final classification logits and intermediate features:

f1 = torch.relu(self.conv1(x))
x = self.pool1(f1)

f2 = torch.relu(self.conv2(x))
x = self.pool2(f2)

logits = self.head(torch.flatten(x, 1))

return logits, (f1, f2)

The application displays a selection of feature channels from each layer.

These visualizations provide a way to observe how the input is transformed as it passes through the network.

However, the displayed feature maps are normalized individually for visualization.

This means their brightness should not be interpreted as a directly comparable activation magnitude across channels.

---

7. Exploring Architectures Without Retraining

One of the more useful features of the project is the Architecture Sandbox.

Here, users can change the number of convolutional channels and the kernel size.

The application immediately recalculates:

• Number of trainable parameters.
• Estimated MACs for each layer.
• Total arithmetic cost.
• Whether the hypothetical architecture fits the selected MAC budget.

This makes the relationship between architecture and computation directly observable.

For example, increasing the output channels of the second convolution affects the number of operations required by that layer.

Increasing the kernel size also increases computation because each output value depends on more input positions.

But there is an important separation between the architecture calculator and the trained model comparison.

Changing the architecture does not automatically tell us its accuracy.

An untrained network has no measured classification performance.

To evaluate a new architecture, we would need to train it, select an appropriate checkpoint using validation data, and measure its performance on held-out test data.

Therefore, the Architecture Sandbox only reports properties that can be calculated analytically.

It does not invent an accuracy score for hypothetical models.

This distinction was important to preserve because the project is intended to demonstrate actual engineering trade-offs, not simulated performance claims.

---

8. What the Experiment Does and Does Not Prove

The results suggest that a small CNN can achieve strong MNIST classification accuracy with a relatively low arithmetic cost.

However, several limitations are worth considering.

First, the models were trained using a single random seed.

A broader comparison would require repeated training runs to estimate performance variation.

Second, the experiment uses only three configurations of the same basic architecture.

It does not establish the optimal CNN for MNIST or the best architecture under a particular computation budget.

Third, MACs estimate only part of the computational work.

Actual inference speed depends on hardware, software libraries, memory behavior, and other implementation details.

Finally, MNIST is a relatively simple classification benchmark.

The relationship between model capacity and accuracy may look very different for larger images, more complex datasets, or real-world deployment conditions.

The purpose of this project is therefore not to introduce a new neural network architecture or claim state-of-the-art performance.

It is to build a transparent, reproducible experiment in which the trade-offs between network size, arithmetic cost, and classification performance can be explored.

---

What I Learned

This project reinforced a few important ideas about machine learning engineering.

First, accuracy alone is not enough to evaluate a model.

Two models can have similar accuracy while requiring very different amounts of computation.

Depending on the application, the smaller model may be the more practical choice.

Second, the relationship between model capacity and accuracy is not linear.

In this experiment, increasing computational cost substantially produced progressively smaller accuracy gains.

The largest model performed best on the test set, but the smallest model was much more computationally economical.

Third, mathematical cost estimates are useful, but they must be interpreted correctly.

MAC counts make it possible to compare arithmetic complexity without hardware-dependent measurements.

However, they should not be confused with actual runtime performance.

Fourth, intermediate visualizations make neural networks easier to understand.

Inspecting convolutional feature maps helps connect the mathematical architecture to the representations produced during inference.

And finally, a clear separation between measured results and hypothetical configurations is essential.

We can calculate the theoretical cost of a network architecture immediately.

But we cannot know how accurately that architecture will classify images until it has been properly trained and evaluated.

That difference is central to designing reliable machine learning experiments.

Final Thoughts

For this fourth From Lab to Code project, I wanted to investigate a question that extends beyond handwritten-digit recognition:

How do we balance model accuracy against computational requirements?

By training three CNN architectures under comparable conditions, measuring their classification performance, and calculating their arithmetic costs, I built a small environment for exploring this trade-off.

The interactive application adds another perspective by allowing users to compare actual models, classify their own drawings, inspect intermediate feature maps, and experiment with architectural changes.

The most interesting result was that the smallest network achieved 96.67% test accuracy using only 55,860 estimated MACs per image.

A significantly larger network improved accuracy to 98.63%, but required approximately 18.5 times more MACs.

Neither model is universally better.

The appropriate choice depends on the accuracy requirements, the computational constraints, and the environment in which the model will be used.

That is the engineering idea I wanted to make visible.

And it follows the same motivation behind From Lab to Code:

Build something small, understand the underlying principles, make the results observable, and share the implementation.

The full source code, trained model checkpoints, reproducible experiment results, tests, and setup instructions are available on GitHub:

Explore Neural Network Efficiency Lab on GitHub

The implementation and interface were developed independently. The project uses the public MNIST dataset with separate attribution for the included samples; no university course code or instructional materials are redistributed.

Thanks for reading!