DEV Community: Rijal Ghodi

Epoch, Batch, and Iteration in Machine Learning

Rijal Ghodi — Fri, 22 Nov 2024 10:26:44 +0000

Background

In the journey of learning Artificial Neural Networks (ANNs), I often encounter confusion surrounding the terms: Epoch, Batch, and Iteration. Sometimes I mix them up when using them. Let’s tackle these concepts together!

Definition

Epoch:
- Definition: An epoch refers to one complete pass of the model through all the available training data.
- Example: If you have a pack of 52 playing cards, an epoch is when you have completely learned all the cards.
Batch:
- Definition: A batch is a subset of the training data used in one iteration of updating model parameters.
- Example: Continuing with the playing card analogy, if you divide the cards into 4 stacks and learn the cards stack by stack, you will have 4 batches, each containing 13 cards.
Iteration:
- Definition: An iteration refers to one step of processing where the model processes one batch of data and update the model parameters.
- Example: Continuing with the playing card analogy, you would need to complete 4 iterations to fully finish the epoch.

Relationship

In each epoch, you go through several batches, and in each batch, you perform multiple iterations.
For example, if you have 100 examples and use a batch size of 20, you’d have 5 iterations per epoch

Lets see the usage of these terms in typical ANN training:

Let's say you have 1000 training data points representing house area versus price. You want to build an Artificial Neural Network (ANN) to predict price from a given unseen house area. To achieve this, you need to determine the parameters (typically weight and bias) that establish the relationship between house area and price.
Because handling 1000 training data points at once is resource-intensive, you divide them into 20 batches. Each batch consists of 50 training data points (1000 / 20 = 50), so the batch size is 50.
First, initialize the model with random parameters.
Next, use the first batch to calculate the loss and update the model parameters accordingly.
Then, with the updated parameters from the previous batch, proceed to calculate the loss and update the model parameters using the second batch.
Continue this process iteratively until you process the final batch.
Congratulations! After completing 20 iterations to finish one epoch, you obtain the optimized model parameters that effectively represent the relationship between house area and price.

Closing

Having a clear understanding of these concepts will help you correctly use these terms in the context of training your Artificial Neural Network. Epochs, batches, and iterations work together to form an efficient and effective training process for your model. I hope this explanation helps clear up any confusion you had about these terms!

First Principle of Machine Learning

Rijal Ghodi — Fri, 22 Nov 2024 10:11:12 +0000

Machine learning is very popular today. Despite the fancy terms and complex jargon often associated with it, machine learning is simply a tool — much like a sword and axe. It is a tool that help human to recognize patterns in the world.

The Goal of Machine Learning

The goal of machine learning is to discover pattern in data.

Specifically, the goal is to create a mathematical function (or pattern) that can accurately map one set of data to another.

For example, imagine we have a list of house sizes and their corresponding prices. We have the data, but we don't know the exact function that links the size of a house to its price. This is where machine learning can help.

Scatter plot of house size vs price. Our goal is to find a function that link both.

The Model

To begin, we need to create a “model.” A model is a mathematical function that we’ll adjust to fit our dataset as closely as possible. This model can be simple, like a straight line (linear function), or more complex, like a curve (quadratic function) or other structures.

The Parameters

No matter which model we pick, it will always have parameters. Parameters are numbers that determine how the function behaves and looks.

For example, let’s look at a simple linear function: $y=wx+b$

This function has two parameters: $w$ (the slope) and $b$ (the y-intercept). These two numbers control the shape and position of the line. Think of these parameters like tuning knobs on a radio, each one adjusting the radio behaviour.

The Training

Our goal in machine learning can be restated as: to find the best set of parameters that will make the model fit the dataset. This process of searching the best parameters is called “training” the model.

Loss Function

Training starts with initialization. We assign random values to the parameters, which of course results in a “wrong” function. This means the function predict the wrong output for the given input. Don’t worry; we’ll use these mistake to gradually improve the model over time.

To learn from a mistake, we need to measure the mistake! We need to figure out how far off the model’s predictions are from the actual output. In machine learning, this mistake is called “loss”.

First attempt of training. we initialize the parameters with random values, which results in a model that is initially wrong (as shown by the orange line). The model’s predictions differ from the actual values, known as the loss. Our goal is to minimize this loss.

Gradient Descent

By nature, the lower a loss, the better a model. We need to make a model with lowest loss. So, our goal of machine learning can be restated as this: to find a model’s parameters that minimize the loss.

To minimize the loss, imagine you’re adjusting two knobs on a radio to find your favorite station. Initially, you might turn both knobs randomly. You notice that one knob is quicker at finding the station, while the other is slower. As you experiment, you start fine-tuning each dial based on how effective it is.

Just like tuning a radio to find a favorite station, adjusting the model’s parameters involves checking how changes affect the loss. This adjustment process is guided by something called the gradient, which indicates how quickly the loss changes as we adjust the parameters.

Optimizer

Intuitively, if the gradient is higher, we need to make bigger adjustments to the parameters. If the gradient is lower, we should make smaller, more gradual changes. There are various optimizers to help with this tuning, but the basic idea remains the same.

Repeat

Repeat this process: define new parameters-> produce a wrong model -> calculate the loss -> determine the gradient -> adjust the parameters accordingly -> define new parameters-> … Keep repeating these steps until finding the optimal parameters that minimize the loss.

After several times of adjusting the parameters, the model’s predictions finally align with the actual data (notice how the orange line fits the blue dots).

Conclusion

Congratulations! You’ve grasped the concept of machine learning without needing to dive into math and code. In summary, machine learning aims to create the best function to map an input dataset to an output dataset. The process is straightforward: we start with a model that’s initially wrong, learn from its mistakes, and use that knowledge to improve the model over time. To me, this approach offers a valuable philosophical lesson about learning and growth.

Test

Rijal Ghodi — Sun, 09 Jun 2024 05:43:31 +0000

$x+y = 5$

$$
\int_{a}^{b} f(x) \, dx
$$

How to Create Component Library Like Material UI or Mantine UI?

Rijal Ghodi — Wed, 12 Jul 2023 02:08:40 +0000

Creating a robust and reusable component library is a fundamental aspect of modern web development. If you've ever wondered how to develop component libraries like Material UI or Mantine UI from scratch, you're in the right place. In this article, we will delve into the essential steps for creating your own component library. So, let's embark on this journey!

📜 Table of Content

📜 Table of Content
🛠️ Techs & Pre-Knowledge
🔭 Project Description
👟 Init NPM Project
⚙️ Configure Typescript
🧱 Create Components
⛑️ Build

🛠️ Techs & Pre-Knowledge

In this tutorial, we will be utilizing the following technologies:

React.js
TypeScript
NPM
esbuild
Storybook
GitHub

Throughout this tutorial, we assume that you already possess a basic understanding of the following concepts. If you are unfamiliar with any of them, please refer to articles covering the topics below:

Basic React.js
Basic TypeScript
Basic GitHub
Basic NPM

🔭 Project Description

So, in this project, we will create a component library called cool-components that includes customizable components. To keep things simple, we will focus on creating just one component: the Button! Users will be able to adjust the color, variant, and size of the Button. Curious about how? Let's dive in!

Note: If you would like to see the final result of this project, please visit the following link.

👟 Init NPM Project

⚙️ Configure Typescript

🧱 Create Components

⛑️ Build

🚀 Publish into NPM

🤖 (Optional) Automate Publish Using Github

📚 (Optional) Add StoryBook

My First Post

Rijal Ghodi — Thu, 06 Jul 2023 10:32:37 +0000

Hello! How are you today? I'm glad to hear that you're doing fine.

Here, I aim to share my learning journey in Web Development, Mobile Development, Machine Learning, and other computer science-related topics. I also want to showcase code snippets within the text, like this inline code, as well as code blocks like:

This code block

It's truly amazing to be able to present code in such a visually appealing manner!

In conclusion, I look forward to sharing my knowledge and experiences in various areas of computer science and providing engaging content for you to explore.