Introduction
A neural network is an information-processing model inspired by the structure of the human nervous system. Just as neurons in the brain are interconnected and process signals collectively, neural networks are made up of interconnected computational units that transform information in a parallel, non-linear way. This ability to model non-linearity makes neural networks especially useful for detecting patterns and extracting meaning from complex data.
In modern analytics and machine learning, neural networks are considered one of the most powerful modeling techniques. At the same time, they are often treated as a black box—users may clearly see inputs and outputs but struggle to understand the internal weight-adjustment process. This article explains the basic mechanism of neural networks, shows how to implement them in R, and demonstrates visualization and cross-validation so readers can gain both conceptual and practical knowledge.
Table of Contents
- The Basics of Neural Network
- Fitting Neural Network in R
- Cross Validation of a Neural Network
- The Basics of Neural Network A neural network is built around layers of units connected by weighted links. An activation function at each node determines how inputs are transformed before passing forward. The input layer accepts raw data, hidden layers process and refine the information, and the output layer generates predictions. A major strength of neural networks is that they are adaptive: they learn from training data by adjusting connection weights to minimize prediction error. This makes them suitable for situations where relationships are non-linear or too complex for traditional statistical models. The simplest neural network is the perceptron. It combines weighted inputs, applies an activation function, and produces an output. While perceptrons can solve linearly separable problems, they cannot handle complex, non-linear ones. To overcome this, multilayer neural networks include hidden layers that allow the model to capture non-linear interactions.
Several learning rules can be applied when training a neural network:
Least Mean Square (LMS)
Gradient Descent (most common)
Newton’s Method
Conjugate Gradient
Training is typically done with backpropagation, where the prediction error at the output layer is propagated backward. Each weight is updated in proportion to its contribution to the total error. Through many iterations, the network gradually converges to a solution that generalizes well to new data.
Figure 1. A simple neural network structure.
Fitting Neural Network in R
Let’s now implement a neural network in R using the Cereal dataset from Carnegie Mellon University (link). The goal is to predict cereal ratings using nutritional attributes such as calories, protein, fat, sodium, and fiber.
We begin by splitting the dataset into training (60%) and testing (40%) sets. A random seed ensures reproducibility.
Read data
data <- read.csv("cereals.csv", header = TRUE)
Create training and test sets
set.seed(80)
samplesize <- 0.60 * nrow(data)
index <- sample(seq_len(nrow(data)), size = samplesize)
datatrain <- data[index, ]
datatest <- data[-index, ]
Before fitting a neural network, we must scale the data. Without scaling, variables with larger ranges (e.g., sodium) may dominate those with smaller ranges (e.g., protein). We use min–max normalization, which rescales values to a [0,1] range while preserving their distribution.
Min–max scaling
maxs <- apply(data, 2, max)
mins <- apply(data, 2, min)
scaled <- as.data.frame(scale(data, center = mins, scale = maxs - mins))
Now we fit the model using the neuralnet package.
Install and load package
install.packages("neuralnet")
library(neuralnet)
Create scaled training and test sets
trainNN <- scaled[index, ]
testNN <- scaled[-index, ]
Fit neural network with 3 hidden neurons
set.seed(2)
NN <- neuralnet(rating ~ calories + protein + fat + sodium + fiber,
data = trainNN, hidden = 3, linear.output = TRUE)
Visualize the network
plot(NN)
Figure 2. Visualization of the neural network with weighted connections and bias nodes.
To evaluate performance, we make predictions on the test set, rescale them back to the original scale, and compute RMSE (Root Mean Square Error).
Prediction on test data
predict_testNN <- compute(NN, testNN[, c(1:5)])
predict_testNN <- predict_testNN$net.result * (max(data$rating) - min(data$rating)) + min(data$rating)
Plot predicted vs. actual ratings
plot(datatest$rating, predict_testNN, col = "blue", pch = 16,
ylab = "Predicted Rating", xlab = "Actual Rating")
abline(0,1)
RMSE
RMSE.NN <- sqrt(mean((datatest$rating - predict_testNN)^2))
In this run, the RMSE is approximately 6.05, indicating the average prediction error in rating units.
Figure 3. Predicted rating vs. actual rating.
Cross Validation of a Neural Network
Evaluating a neural network on just one train-test split can give misleading results. To better understand its generalization, we apply cross-validation.
The simplest form is the holdout method, which we used above. But this depends heavily on how data is split. A more robust method is k-fold cross-validation, where the dataset is divided into k subsets. Each subset acts as the test set once while the others form the training set. This reduces variance in performance metrics. At the extreme, when k = n (the number of rows), we get leave-one-out cross-validation (LOOCV).
In the example below, we vary the size of the training set from 10 to 65 and repeat sampling 100 times for each size. For each sample, we train the network, test it, and compute RMSE. This gives a robust picture of how training size affects accuracy.
Install and load packages
install.packages(c("boot", "plyr"))
library(boot)
library(plyr)
set.seed(50)
k <- 100
RMSE.NN <- NULL
List <- list()
Nested loop for repeated sampling
for (j in 10:65) {
for (i in 1:k) {
index <- sample(1:nrow(data), j)
trainNN <- scaled[index, ]
testNN <- scaled[-index, ]
datatest <- data[-index, ]
NN <- neuralnet(rating ~ calories + protein + fat + sodium + fiber,
trainNN, hidden = 3, linear.output = TRUE)
predict_testNN <- compute(NN, testNN[, c(1:5)])
predict_testNN <- predict_testNN$net.result * (max(data$rating) - min(data$rating)) + min(data$rating)
RMSE.NN[i] <- sqrt(mean((datatest$rating - predict_testNN)^2))
}
List[[j]] <- RMSE.NN
}
Matrix.RMSE <- do.call(cbind, List)
We can visualize the error distribution using boxplots:
Boxplot for training set size = 65
boxplot(Matrix.RMSE[,56], ylab = "RMSE",
main = "RMSE BoxPlot (Training Set = 65)")
Figure 4. Boxplot of RMSE distribution for training size 65.
To see the effect of training size on performance, we plot the median RMSE for each training size.
Median RMSE for each training set size
install.packages("matrixStats")
library(matrixStats)
med <- colMedians(Matrix.RMSE)
X <- seq(10,65)
plot(med ~ X, type = "l", xlab = "Training Set Size",
ylab = "Median RMSE",
main = "Variation of RMSE with Training Set Size")
Figure 5. Variation of RMSE with training set size.
The plot shows that the median RMSE decreases as the training set grows larger, reinforcing the importance of sufficient training data for accuracy.
End Notes
This article explained the fundamentals of neural networks, demonstrated their implementation in R, and explored model evaluation. Neural networks, inspired by biological systems, transmit information through interconnected layers, with weights optimized via backpropagation.
We used the cereal dataset to predict product ratings from nutritional attributes. After fitting a neural network using the neuralnet package, we evaluated performance using RMSE and visualized predictions against actual outcomes. Finally, cross-validation was applied to assess robustness, showing that accuracy improves as the training set increases.
The key lessons are:
Neural networks are powerful for non-linear prediction tasks.
Data scaling and preprocessing are crucial before training.
Cross-validation is essential for checking generalization.
With these foundations, practitioners can extend neural network modeling in R to larger datasets and more complex architectures, including deeper networks implemented with packages like keras.
Our mission is to help organizations unlock the full potential of their data. Over the past two decades, we’ve partnered with Fortune 500 companies and mid-sized firms alike to address complex analytics challenges and deliver measurable results. Our expertise spans across Tableau consultants, Power BI Consultants, and AI Consulting, empowering businesses to transform raw information into strategic insights that drive growth and efficiency.
Top comments (0)