DEV Community: Sean Atukorala

Solution for "error[E0425]: cannot find function `every` in module `time`" when using Iced Rust GUI library

Sean Atukorala — Mon, 26 Dec 2022 22:22:09 +0000

Problem:
Running into the following error when trying to use the Iced GUI library for a Rust program that replicates the stopwatch Iced GUI example:

$ cargo build
error[E0425]: cannot find function `every` in module `time`
   --> src\main.rs:203:23
    |
203 |                 time::every(Duration::from_millis(10)).map(Message::Tick)
    |                       ^^^^^ not found in `time`

Solution:
The problem was that the smol feature was not included in the features option for the iced dependency listed in the Cargo.toml file.

Try replacing your existing iced dependency line with the following line:

# Cargo.toml file

[dependencies]
iced = {version = "0.6", features = ["smol"]}

Then run cargo build and see if the error is resolved!

Conclusion

Thanks for reading this blog post!

If you have any questions or concerns please feel free to post a comment in this post and I will get back to you if I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn and subscribe to my YouTube channel.

How to build a Turing Machine Using Pixi.js

Sean Atukorala — Sun, 27 Nov 2022 18:56:11 +0000

Ever wondered how to create a Turing machine using a JavaScript game library? If so, keep reading to learn how to build a Turing machine using the popular JavaScript/HTML rendering library: PixiJS!

What is a Turing Machine?

First of all, let's briefly go over what a Turing Machine is: A Turing machine is a theoretical machine originally conceived by the mathematician Alan Turing. The Turing machine can supposedly simulate any computer algorithm regardless of the level of complexity.

Although the original idea for the machine was purely abstract and hypothetical here is an image of a physical representation of this machine:

Figure 2: A physical representation of a Turing machine

Keep in mind that it is not exactly in accordance with the theoretical concept as it is not possible to have a tape of infinite length.

On a more practical level, a Turing machine can read from and write to an infinite tape(although no infinite tapes exist in real life). It manipulates symbols on this theoretically infinite tape based on a predefined set of rules also known as instructions.

The machine has a head which is positioned over a single square on the aforementioned infinite tape and performs one of the following three instructions:

Read the value of the square that the head is currently pointing towards
Write a value based on the predefined instruction set to the square that the head is currently pointing towards
Move the tape either to the left or right by one square in order to do one of this three actions on a neighboring square

Development:

In this tutorial we will be building a Turing Machine application that replicates a palindrome program.

{/* */}

For this tutorial it would be easy if you could follow along with the sample application in this GitHub link: Palindrome Turing Machine Starter Code

First visit this GitHub link and clone the repo. Then open the folder in a coding editor(I'll be using VSCode for this tutorial):

Figure 3: Image of the cloned repo in VSCode

For the sake of brevity we'll only cover the most important functions in the main.js file of this program.

Developing the GUI:

So let's start out by covering some rendering functions:

function RenderTapePointer() {
    // center the sprite's anchor point
    tape_pointer.anchor.set(0.5);
    // move the sprite to the center of the screen
    tape_pointer.x = app.screen.width / 2 - 45;
    tape_pointer.y = (app.screen.height / 2) + 150;
    tape_pointer.width = 30;
    tape_pointer.height = 30;

    app.stage.addChild(tape_pointer);
}

function RenderTapeWPixi() {
    firstTapeSquare = new PIXI.Text("1", { fontName: "foo" });
    secondTapeSquare = new PIXI.Text("0", { fontName: "foo" });
    thirdTapeSquare = new PIXI.Text("0", { fontName: "foo" });
    fourthTapeSquare = new PIXI.Text("1", { fontName: "foo" });
    fifthTapeSquare = new PIXI.Text("0".toString(), { fontName: "foo" });
    sixthTapeSquare = new PIXI.Text("0", { fontName: "foo" });
    seventhTapeSquare = new PIXI.Text("1", { fontName: "foo" });

    tapeSquaresArr = [firstTapeSquare, secondTapeSquare, thirdTapeSquare, fourthTapeSquare, fifthTapeSquare, sixthTapeSquare, seventhTapeSquare];

    const style = new PIXI.TextStyle();

    for (var i = 0; i < 7; i++) {
        // Apply the font to our text
        style.x = app.screen.width / 2;
        style.y = app.screen.height / 2;
        console.log(app.screen.height / 2);
        tapeSquaresArr[i].x = (app.screen.width - 100) / 2 + (i * 25);
        tapeSquaresArr[i].y = 400;

        // Update the font style
        style.fill = 'black';

        // Update text
        tapeSquaresArr[i].updateText();

        app.stage.addChild(tapeSquaresArr[i])
    }
}

function RenderStateText() {
    stateText.x = 50;
    stateText.y = 400;

    stateText.width = 100;
    stateText.height = 50;

    app.stage.addChild(stateText);

    stateValText.x = 170;
    stateValText.y = 400;
    stateValText.width = 50;
    stateValText.height = 50;

    app.stage.addChild(stateValText);
}

Here is a brief explanation for the functions above:

RenderTapePointer(): This function's main job is to render the tape pointer that will indicate the current tape square that the program is computing. All image objects in the Pixi.js rendering library are called Sprites. Notice the app.stage.addChild(tape_pointer) line which is responsible for showing the Sprite on-screen by adding it to the Pixi.js Stage object. Here is a screenshot of the tape pointer:

Figure 3: Tape pointer pointing to the current tape square

RenderTapeWPixi(): This function renders the actual tape that will be used by the Turing Machine. Since the Turing Machine is setup to run a Palindrome program, having a finite length of tape is feasible. Here is a screenshot of the tape containing some predefined values:

Figure 4: Tape containing seven binary values

RenderStateText(): This function renders the text that indicates the current state of the Turing Machine. The current state of the machine is important because it influences the selecting of the new symbol to be placed in the currently selected tape square. For example: If the current state is 0 and the current symbol is 1 then the new symbol that will be written into the current tape square is _(This is based on the state rules that we will cover soon). Here is a screenshot of the rendered state text:

Figure 5: Current state text rendered to the stage

Next we'll cover the functions that render the buttons and title for the application:

var startBtn = PIXI.Sprite.from("./assets/start-icon.png")
startBtn.buttonMode = true;
startBtn.anchor.set(0.5);
startBtn.position.x = 200;
startBtn.position.y = 150;
startBtn.width = 50;
startBtn.height = 50;
startBtn.interactive = true;
startBtn.on('mousedown', startPalindromeProgram);
app.stage.addChild(startBtn);

var restartBtn = PIXI.Sprite.from("./assets/restart-icon-v2.png")
restartBtn.buttonMode = true;
restartBtn.anchor.set(0.5);
restartBtn.position.x = 200;
restartBtn.position.y = 150;
restartBtn.width = 50;
restartBtn.height = 50;
restartBtn.interactive = true;
restartBtn.on('mousedown', restartPalindromeProgram);
app.stage.addChild(restartBtn);
restartBtn.visible = false;

var stopBtn = PIXI.Sprite.from("./assets/stop-icon.png")
stopBtn.buttonMode = true;
stopBtn.anchor.set(0.5);
stopBtn.position.x = 400;
stopBtn.position.y = 150;
stopBtn.width = 50;
stopBtn.height = 50;
stopBtn.interactive = true;
stopBtn.on('mousedown', stopProgram);
app.stage.addChild(stopBtn);

var resumeBtn = PIXI.Sprite.from("./assets/resume-icon.png")
resumeBtn.buttonMode = true;
resumeBtn.anchor.set(0.5);
resumeBtn.position.x = 600;
resumeBtn.position.y = 150;
resumeBtn.width = 50;
resumeBtn.height = 50;
resumeBtn.interactive = true;
resumeBtn.on('mousedown', resumeProgram);
app.stage.addChild(resumeBtn);

var moveTapeInterval = null;

function stopProgram() {
    moveTapeInterval.pause();
    toggleGameSpeedSliderVisible(true);
}

function resumeProgram() {
    moveTapeInterval.resume(gameSpeed);
    toggleGameSpeedSliderVisible(false);
}

function restartPalindromeProgram() {
    ResetGame();
    StartTape();
    restartBtn.visible = false;
    restartBtnLabel.visible = false;
}

...

const title = new PIXI.Text('Palindrome Program Turing Machine', {
    fill: '#272d37',
    fontFamily: 'Roboto',
    fontSize: 20,
    align: 'center',
});
title.roundPixels = true;
title.x = stageWidth / 2;
title.y = 40;
title.anchor.set(0.5, 0);
app.stage.addChild(title);

Here is a brief explanation of the code above:

startBtn: This button controls the start of the Palindrome program. Notice the eventlistener 'mousedown' which makes starting the Palindrome program possible
restartBtn: This button allows the user to restart the Palindrome program once it has reached the end state. It is not shown in the initial startup of the application
stopBtn: This button pauses the Palindrome program. It is useful for adjusting the game speed which can only be changed when the game is paused.
resumeBtn: This button allows the user to resume the game after being paused. Clicking it also makes the game speed slider disappear in order to prevent the user from changing the game speed while the program is running

Here are screenshots showing all four buttons:

Figure 6: The start, pause, resume and restart buttons on display

Lastly the title variable is an instance of PIXI.Text(), which is used to create a title for the application on the top of the game screen.

Developing the internal logic:

Moving onto the internal logic for the application, let's start out by defining the state rules for the Palindrome program:

var palindromePrgStateRules = [
    // [<current_state>, <current_symbol>, <new_symbol>, <direction>, <new_state>]
    ['0', '0', '_', 'r', '1o'],
    ['0', '1', '_', 'r', '1i'],
    ['0', '_', '_', '*', 'accept'],
    ['0', '1', '_', 'r', '1o'],
    ['1i', '_', '_', 'l', '2i'],
    ['1i', '*', '*', 'r', '1i'],
    ['1o', '*', '*', 'r', '1o'],
    ['1o', '_', '_', 'l', '2o'],
    ['2o', '0', '_', 'l', '3'],
    ['2o', '_', '_', '*', 'accept'],
    ['2o', '*', '*', '*', 'reject'],
    ['2i', '1', '_', 'l', '3'],
    ['2i', '_', '_', '*', 'accept'],
    ['2i', '*', '*', '*', 'reject'],
    ['3', '_', '_', '*', 'accept'],
    ['3', '*', '*', 'l', '4'],
    ['4', '*', '*', 'l', '4'],
    ['4', '_', '_', 'r', '0'],
    ['accept', '*', ':', 'r', 'accept2'],
    ['accept2', '*', ')', '*', 'halt-accept'],
    ['reject', '_', ':', 'r', 'reject2'],
    ['reject', '*', '_', 'l', 'reject'],
    ['reject2', '*', ')', '*', 'halt-reject'],
];
var comparisonValues = {
    "0": ["0", "*"],
    "1": ["1", "*"],
    "*": ["0", "1", "*"],
    " ": ["_"],
    "_": ["_"],
    ")": [")"]
}

The palindromePrgStateRules 2d array is an array containing array elements, each with 5 strings in them to represent the following rules:

The first element represents the current state value of the program
The second element represents the current symbol that the tape pointer is pointing to
The third element represents the new symbol that should take the place of the current symbol(second element)
The fourth element represents the direction that the tape pointer should take for its next move. 'l' means move to the left, 'r' means move to the right.
The fifth and final element represents the new state value that should take the place of the current state value(first element)

Finally, the comparisonValues object serves as a comparison reference for the current symbol that the tape pointer is pointing to and the values that are considered equivalent comparisons to it. For example: if the current symbol is 1 then the compatible equivalents to it are '1' and '*'(based on the line "1": ["1", "*"],). This object is very useful in deciding the appropriate next symbol to assign for a given square that the tape pointer is pointing to.

Next, let's cover the functions responsible for starting the Palindrome program:

function startPalindromeProgram() {
    renderTapePointer();
    renderTapeWPixi();
    renderStateText();
    startTape();
    toggleStartBtnVisible(false);
    toggleGameSpeedSliderVisible(false);
    startBtnLabel.visible = false;
}

...

function startTape() {
    moveTapeInterval = new IntervalTimer(moveTapeLogic, gameSpeed);
    moveTapeInterval.start();
}

function moveTapeLogic() {
    var moveTapeDirection = ProcessInput();
    var counter = 0;
    console.log("moveTapeDirection: " + moveTapeDirection);
    switch (moveTapeDirection) {
        case "r":
            tape_pointer.x += 25;
            currentHeadIndex++;
            break;
        case "l":
            tape_pointer.x -= 25;
            currentHeadIndex--;
            break;
        case "halt-reject":
            clearInterval(moveTapeInterval);
            moveTapeInterval.clear();
            toggleRestartButton(true);
            break;
        default:
            break;
    }
    counter++;
}

Here is a brief explanation for the two functions above:

startPalindromeProgram(): This the main starting function for the whole program. It calls the rendering functions to render the necessary elements on screen, hides the start button and game slider, and finally calls the startTape() function.
startTape(): This function starts the program by using the IntervalTime class to setup an interval. You might be wondering the purpose of using the IntervalTimer class instead of the more convenient setTimeout() function. Well here it is: IntervalTimer allows for the ability to pause, resume and clear the currently running interval by calling those respective methods.
moveTapeLogic(): This function can be considered the controller for this application because it identifies the output received by calling the ProcessInput() function. The output signifies the direction that the tape pointer should move next to, and this logic is handled by the switch statement in this function. The way the tape pointer's position is represented programmatically is via the currentHeadIndex variable. When the tape pointer is moved one square to the right this variable is incremented by one, and when the tape pointer is moved to the left currentHeadIndex is decremented by one. The ending state is reached when a output of halt-reject is received from ProcessInput(). The setInterval() function underlying moveTapeInterval is cleared, which brings the program to an end.

Next we'll cover the main processing function for this application:

function ProcessInput() {
    var i = 0;
    var foundMatch = false;
    for (i = 0; i < palindromePrgStateRules.length; i++) {

        var textComparison = null;
        var currentTapeVal = null;
        if (currentHeadIndex >= tapeSquaresArr.length) {
            textComparison = compareTapeValues(" ", palindromePrgStateRules[i][1]);
            currentTapeVal = " ";
        } else {
            textComparison = compareTapeValues(tapeSquaresArr[currentHeadIndex].text, palindromePrgStateRules[i][1]);
            currentTapeVal = tapeSquaresArr[currentHeadIndex].text;
        }

        if (textComparison && currentState == palindromePrgStateRules[i][0]) {
            var newTapeSymbol = setNewTapeSymbol(currentTapeVal, palindromePrgStateRules[i][2]);

            try {
                tapeSquaresArr[currentHeadIndex].text = newTapeSymbol;
                tapeSquaresArr[currentHeadIndex].updateText();
            } catch (err) {
                console.log("current head index is empty");
            }

            currentState = palindromePrgStateRules[i][4];
            stateValText.text = palindromePrgStateRules[i][4];
            foundMatch = true;
            break;
        }
    }
    if (foundMatch) {
        return palindromePrgStateRules[i][3];
    } else {
        return "halt-reject";
    }
}

The ProcessInput() function performs the following steps in determining the next action to take when presented with the current input symbol. That is, the current tape symbol that the tape pointer is pointing to.

It loops over the palindromePrgStateRules array in order to find a matching element for the current state of the game. For example: if the current state value is 2i and the current tape symbol is 1 then the element ['2i', '1', '_', 'l', '3'] matches this criteria.
Following the example in the previous step: the new symbol for the tape square is set to _, the tape pointer is moved one position to the left, and the state value is change from 2i to 3.
In the event that a matching element in the palindromePrgStateRules array is not found to match the current state of the program, the halt-reject value is returned by the function and the program is bought to the end state.

Let's quickly look at the function for comparing the current input symbol to a given input symbol in the palindromePrgStateRules array:

function compareTapeValues(currentHeadVal, stateCurrentSymbolVal) {
    if ((currentState == "accept" || currentState == "accept2") && currentHeadVal == "_") {
        return true;
    }
    if (comparisonValues[currentHeadVal.toString()].includes(stateCurrentSymbolVal.toString())) {
        return true;
    } else {
        return false;
    }
}

The above function uses the comparisonValues object that we defined earlier in determining whether currentHeadVal and stateCurrentSymbolVal match. Keep in mind that the symbol: * that is found in some elements of palindromePrgStateRules means that the currentHeadVal gets a free pass; it will match up to any given currentHeadVal value. Another condition that is considered a match is when the current state value is set to accept and currentHeadVal is _.

Now let's visit the function responsible for restarting the program if the user wants to run it once more:

function resetGame() {
    // Reset tape numbers to defaults
    const tapeNumDefaults = ["1", "0", "0", "1", "0", "0", "1"];

    for (var i = 0; i < tapeSquaresArr.length; i++) {
        // Apply the font to our text
        tapeSquaresArr[i].text = tapeNumDefaults[i];

        // Update text
        tapeSquaresArr[i].updateText();
    }

    // Reset tape pointer to default position
    tape_pointer.x = app.screen.width / 2 - 45;
    tape_pointer.y = (app.screen.height / 2) + 150;

    // Reset current head index and current state variables to 0
    currentHeadIndex = 0;
    currentState = '0';

    toggleStartBtnVisible(false);
    toggleGameSpeedSliderVisible(false);
}

The resetGame() function resets the main state variables in the program(currentHeadIndex, currentState) to their default values along with changing the tape square symbols back to their default values and moving the tape pointer sprite back to its starting position. Finally, the start button and the game slider is hidden. The restart button will be displayed in place of the start button, which gives the user the option to restart the Palindrome program.

That brings the development section of this blog post to a close.

If you made it this far then congratulations! You have a pretty good idea of how this application functions.

Program Demo:

Here is a gif showing a demonstration of this program:

Game demonstration link on Imgur

Conclusion

Thanks for reading this blog post!

If you have any questions or concerns feel free to post a comment in this post and I will get back to you if I find the time.

If you found this article helpful feel free share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn and subscribe to my YouTube channel.

Cross-validation in Machine Learning Explained

Sean Atukorala — Fri, 11 Nov 2022 20:13:50 +0000

Cross-validation is a technique for evaluating machine learning models by training several machine learning models on subsets of the available input data and evaluating them on the complementary subset of the data.

Basically, cross-validation is the practice of judging the performance and accuracy of a given machine learning model by using various different divisions of training and testing data, sometimes across multiple rounds of validation, and then finally aggregating the results to derive a highly accurate evaluation of the given machine learning model.

In some types of cross-validation such as K Fold cross-validation, the subsets used for training purposes and the subset used for validating purposes are changed over multiple attempts in order to generate a more accurate conclusion about the performance of the given machine learning model.

While the main benefit of cross-validation is to test the ability of a machine learning model to predict new data, it is also very effective in detecting overfitting and selection bias

Cross-validation allows us to compare the results of various machine learning models and get a sense of how well each one of them will work in practice.

Figure 2: General process of training models

General process for Cross-validation:

Although there are various different types of cross-validation all of them adhere to the following workflow listed below:

Divide the dataset into two different sections: the training section and the testing section
Train the model using the training dataset
Validate the model using the testing dataset
Repeat steps 2 and 3. The exact number of repetitions depends on the specific type of cross-validation that you choose to use

Figure 3: General process for Cross-validation

Different types of Cross-validation:

There are numerous types of cross-validation. Here are brief descriptions of the most common ones:

K-fold cross-validation: This type of cross-validation divides the dataset into k number of groups and then proceeds with the general process of cross-validation.
Hold-out cross-validation: Hold-out cross-validation divides the entire dataset randomly into a training set and a validation set. This type of cross-validation has the advantage of faster execution times since the dataset is split into only two sets, the training and validation sets, the model is build just one time
Stratified k-fold cross-validation: This is considered identical to K-fold cross-validation with the addition that it seeks to make sure that each group has the same proportion of observations/samples with a given categorical value. This type of cross-validation is highly useful when dealing with imbalanced datasets. Keep in mind that Stratified K Fold cross-validation is not suitable to handle time series data since the samples are selected in random order
Leave-p-out cross-validation: In this type of cross-validation there are P number of data points used as test set data while the remainder of the data points are used for training purposes. Its advantage is that it uses the whole dataset for both training and testing and therefore provides much less biased results, but it is not suitable for use with large datasets as it can be very time-consuming and computationally expensive to do so
Monte Carlo(shuffle-split) cross-validation: This type of cross-validation splits the dataset randomly into training and testing sets. The number of cross-validation iterations are not fixed but are decided by analysis. In contrast to K Fold cross-validation where you use one fold as the test set and the remaining folds as the training set, shuffle-split uses a specific training and test set from iteration n for each round n(More on this in these Stack OverFlow answers: link1, link2). Shuffle-split is a good cross-validation technique for large datasets but is not suitable for imbalanced datasets
Time series(rolling) cross-validation: In this type of cross-validation the test set consists of only a single observation/sample. In contrast, the training set consists of the cumulative observations/samples that are encountered before the observation/sample that is used as the test set. When initially starting the cross-validation process, the test set is not a reliable predictor because the model is based on a small training set. Therefore, in order to leverage the full power of time series cross-validation: only when the test set is based on later observations/samples should validation for the model's accuracy be conducted.

Advantages and Disadvantages of Cross-validation:

Overall, I believe that there are more advantages to cross-validation than disadvantages.

Some pros of cross-validation include:

Minimizes overfitting issues: This is accomplished by splitting the dataset into multiple sections and training the model on constantly differing sections. Doing so increases the robustness of the model and therefore reduces overfitting as well
Increases efficiency of data usage: This is because almost all the observations/samples in the dataset are used for both training and testing purposes
Model accuracy tends to increases: This is due to all the data in the dataset(or almost all of it) being utilized to train the model

Here are some disadvantages of cross-validation:

Expensive computational cost: Due to the increased demand of using multiple folds of the dataset for training and testing purposes, computational cost inevitably increases.
High training time: Due to having to train a model on multiple training sets, using cross-validation can incur higher than average training times

Conclusion

That's it for this blog post on Cross-validation.

Thanks for reading this blog post!

If you have any questions or concerns please feel free to post a comment in this post and I will get back to you if I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn and subscribe to my YouTube channel.

K-Nearest Algorithm in Machine Learning Explained

Sean Atukorala — Sat, 22 Oct 2022 20:46:49 +0000

Ever wondered what the K-NN Algorithm is? If so, keep reading to find out!

First let's go over the two types of machine learning algorithms in machine learning:

Supervised: This type of machine learning uses labeled datasets to train models in order to classify data and predict outcomes accurately. This type of learning is useful in predicting outputs or outcomes for new and unseen data on the basis of prior experiences. Some practical examples include classifying images, words or file types. Supervised machine learning is useful when you know what you're looking for in a given dataset
Unsupervised: This is a type of machine learning in which machine learning models learn patterns through untagged and unlabeled data. The model analyses the data in order to find previously unknown patterns in the data. It can also be useful in finding features, which are independant variables that serve as input for a system, which is very useful in conducting categorization. Some practical examples include data exploration and social media recommendation systems, target marketing campaigns. Unsupervised machine learning is useful when you don't know what you're looking for in a given dataset

Basic Overview of K-Nearest Neighbors algorithms

K-Nearest Neighbors algorithms are considered to be one of the simplest Machine Learning algorithms that falls into the Supervised algorithms category.

K-Nearest Neighbors algorithms operate on the basic assumption that closely similar things exist in closer proximity to each other that more dissimilar things. With this initial assumption, the K-NN algorithm will place new unseen data into a category that is most similar to the available categories.

The K-NN algorithm is considered a lazy learning type of algorithm because it doesn't immediately learn from the training data assigned to it. Instead, it merely stores the training data and at the time of needing to classify new and unseen data, it places that data into a category that is the most similar to the new data.

The K-NN algorithm is known as an ad-hoc classifier that is used to classify test data based on distance metrics(such as Euclidean, Chebyshev, Cosine, etc)

K-NN algorithms can be used for both types of Supervised Machine Learning categories: Regression and Classification. Despite this flexibility, it is mostly used for classification ML problems.

Here are some pros and cons of the K-Nearest Neighbors algorithms:

Pros:

It is relatively easy and straight forward to setup
It handles noisy training data pretty well, meaning it predicts accurately despite there being noisy data in the training set. Noise in this context refers to inaccurate data as a result of human error in the data collection process for the dataset being used
It is efficient and effective at using large training sets in the prediction process

Cons:

The value of K always needs to be calculated prior to making any predictions. This can get quite tedious and complex at times
Since the distance between all the training set data points in relation to each other have to be calculated the computational cost can get quite high at times

Basic flow of KNN

The following list of steps provides a general overview of how the KNN algorithm works:

Load and split the data into train and test sets
Pick a value for K. Look for the section below for a more in-depth explanation on how to do so
In order to generate a prediction, do the following steps for all iterations from 1 to the total number of training data points:
1. First, calculate the distance between the test data and each row of training data. The distance metric to be used can be any of the following: Euclidean, Chebyshev, Cosine, Manhattan, Minkowski, Jaccard, etc
2. Sort the calculated distances in ascending order based on distance values
3. Get the number of rows equal to the value of k from the top of the sorted array. Ex: if k = 4, then get the top 4 rows from the sorted array
4. Get the most frequent category of the rows selected in step 3
5. Return the most frequent category as the predicted category for the current training data point

Picking a value for K

The value of K in the K-NN algorithm indicates the count of the nearest neighbors.

The value of K should optimally be a number that is not too small and not too large. Choosing a value of K that is too small leads to unstable decision boundaries and overfitting problems. Meanwhile, having a value of k that is too large can lead to underfitting problems.
It is also important to note that the value of k is non-parametric, meaning it does not rely on any assumptions about the shape or parameters of the dataset from which the training data was drawn.

First of all, keep in mind that there is no definite way to pick the best value for K. The most practical and widely-used strategy to use trail and error to find an optimal K value. To do this, you have to pretend that part of your training data is "unknown".

Alternatively, you can use this general rule of thumb for choosing the best value of k using this following formula:

k = sqrt(N)/2

Where N stands for the number of rows in the training set.

It also might help to keep the value of k to an odd number so as to not create any stalemate/tie situations, arising from choosing a class/category.

Conclusion

Thanks for reading this blog post!

If you have any questions or concerns please feel free to post a comment in this post and I will get back to you if I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn and subscribe to my YouTube channel.

Logistic Regression in Machine Learning Explained

Sean Atukorala — Wed, 05 Oct 2022 18:55:47 +0000

What is Logistic Regression?

Logistic regression is a type of machine learning model. From the two types of machine learning models(regression and classification), it is a classification algorithm even though the name would indicate it to be a regression model.

It derives its name from the logistic function, which is a type of exponential function. This function is characterized by a S-shaped curve(also known as a sigmoid curve).

Accordingly, logistic regression models produce S-shaped curves when plotted on graphs, like so:

Figure 2: Logistic Regression Model when graphed

Here are important points about Logistic Regression:

Logistic Regression is intended for classification ML problems that have a single binary output variable. Machine learning problems with any other type of output variable/(s) should not use Logistic Regression for predictive modelling purposes
Logistic Regression algorithms are similar to Linear Regression algorithms in that they both are predictive analysis algorithms but with Logistic Regression using the more complex Sigmoid function instead of relatively simple linear funtions
Whereas linear regression produces numeric output values, logistic regression produces binary output values(0 or 1)
Although logistic regression is technically a classification algorithm, it does predict probabilities in a regression algorithm sense. Its just that the probability is then converted into a binary value(0 or 1)
Logistic regression uses another machine learning algorithm: Maximum-likelihood estimation in order to estimate the coefficients(Beta values b) for logistic regression from the training data available
Logistic regression and linear regression share the same method of data preparation

Example of Logistic Regression In R

Here is a code snippet in the R programming language that demostrates the use of the Logistic Regression Model:

library(ggplot2)
library(cowplot)

url <- "http://archive.ics.uci.edu/ml/machine-learning-databases/heart-disease/processed.cleveland.data"

data <- read.csv(url, header=FALSE)

...

logistic <- glm(hd ~ ., data = data, family = "binomial")

...

predicted.data <- data.frame(
  probability.of.hd=logistic$fitted.values,
  hd=data$hd)

predicted.data <- predicted.data[
  order(predicted.data$probability.of.hd, decreasing=FALSE),]
predicted.data$rank <- 1:nrow(predicted.data)

ggplot(data=predicted.data, aes(x=rank, y=probability.of.hd)) +
  geom_point(aes(color=hd)) +
  xlab("Index") +
  ylab("Predicted probability of getting heart disease")

The above code snippet is taken from StatQuest with Josh Starmer and you can find the full version of this code in the this GitHub link

Now, here is the graph that is produced by running the full version of the code snippet above:

Figure 3: Logistic Regression graphed using the R programming language

Here are some points of noteworthiness for the above graph:

The range of the Y-axis is always between 0 and 1 for Logistic Regression graphs. This is due to the dependant variable(Predicted probability of getting heart disease) being discrete
The turquoise colored data points represent patients already having heart disease. The light red colored data points represent patients without any past history of heart disease
The x-axis value 'Rank' is representative of the likelihood of each of the 297 patients in the data.frame having heart disease, with a higher rank indicating a high likelihood of getting heart disease

That's it for this blog post on Logistic Regression in Machine Learning. I hope it was helpful!

Conclusion

Thanks for reading this blog post!

If you have any questions or concerns please feel free to post a comment in this post and I will get back to you if I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn and subscribe to my YouTube channel.

[Solved] Package xkbfile was not found in the pkg-config search path

Sean Atukorala — Tue, 27 Sep 2022 18:27:08 +0000

Problem:

Running into the following error when trying to build monaco-editor from source(on the yarn command step)on an Ubuntu VM:

Package xkbfile was not found in the pkg-config search path.
Perhaps you should add the directory containing `xkbfile.pc'
to the PKG_CONFIG_PATH environment variable
No package 'xkbfile' found
gyp: Call to '${PKG_CONFIG:-pkg-config} x11 xkbfile --cflags | sed s/-I//g' returned exit status 0 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/home/sean/.nvm/versions/node/v16.17.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:261:16)
gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Linux 5.15.0-48-generic
gyp ERR! command "/home/sean/.nvm/versions/node/v16.17.0/bin/node" "/home/sean/.nvm/versions/node/v16.17.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/sean/Documents/JavaScriptApplications/vscode/node_modules/native-keymap

Solution:

This error should be resolved by installing the missing package like so:

sudo apt-get install libxkbfile-dev

Conclusion

Thanks for reading this blog post!

If you have any questions or concerns please feel free to post a comment in this post and I will get back to you if I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn and subscribe to my YouTube channel.

[Solved] Package x11 was not found in the pkg-config search path

Sean Atukorala — Tue, 27 Sep 2022 18:26:07 +0000

Problem:

Running into the following error when trying to build monaco-editor from source(on the yarn command step) on an Ubuntu VM:

Package x11 was not found in the pkg-config search path.
Perhaps you should add the directory containing `x11.pc'
to the PKG_CONFIG_PATH environment variable
No package 'x11' found
Package xkbfile was not found in the pkg-config search path.
Perhaps you should add the directory containing `xkbfile.pc'
to the PKG_CONFIG_PATH environment variable
No package 'xkbfile' found
gyp: Call to '${PKG_CONFIG:-pkg-config} x11 xkbfile --cflags | sed s/-I//g' returned exit status 0 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/home/sean/.nvm/versions/node/v16.17.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:261:16)
gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Linux 5.15.0-48-generic
gyp ERR! command "/home/sean/.nvm/versions/node/v16.17.0/bin/node" "/home/sean/.nvm/versions/node/v16.17.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/sean/Documents/JavaScriptApplications/vscode/node_modules/native-keymap

Solution:

This error should be resolved by installing the missing package like so:

sudo apt-get install libx11-dev

Conclusion

Thanks for reading this blog post!

If you have any questions or concerns please feel free to post a comment in this post and I will get back to you if I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn and subscribe to my YouTube channel.

[Solved] Package libsecret-1 was not found in the pkg-config search path

Sean Atukorala — Tue, 27 Sep 2022 18:25:07 +0000

Problem:

Running into the following error when trying to build monaco-editor from source(on the yarn command step) on an Ubuntu VM:

Package libsecret-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsecret-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libsecret-1' found
gyp: Call to 'pkg-config --cflags libsecret-1' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/home/sean/.nvm/versions/node/v16.17.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:261:16)
gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
gyp ERR! System Linux 5.15.0-48-generic
gyp ERR! command "/home/sean/.nvm/versions/node/v16.17.0/bin/node" "/home/sean/.nvm/versions/node/v16.17.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/sean/Documents/JavaScriptApplications/vscode/node_modules/keytar

Solution:

This error should be resolved by installing the missing package like so:

sudo apt-get install libsecret-1-dev

Conclusion

Thanks for reading this blog post!

If you have any questions or concerns please feel free to post a comment in this post and I will get back to you if I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn and subscribe to my YouTube channel.

Linear Regression in Machine Learning Explained

Sean Atukorala — Sat, 24 Sep 2022 19:04:43 +0000

Ever wonder what the terms Regression and Linear Regression mean in a machine learning context? If so, keep reading to find out!

What is Regression in Machine Learning?

Regression is primarily a technique for analyzing the relationship between independent variables/features and a dependent variable/outcome. Regression is a method most frequently used to solve supervised machine learning problems.

Regression modeling, which as the name implies uses regression, consists of building a mapping function such that input variables map to a continuous output variable. The input variables are independent and the continuous output variable is dependent in this case. This type of modelling is mainly useful in two ways: (1) To predict the outcome of new and unseen input data (2) To forecast and predict gaps in missing data.

What does Linear Regression mean in Machine Learning?

Linear regression is a type of machine learning model. It is most easily explained as a linear equation that combines a specific set of input values, which are independent variables, to the predicted output value, which is a dependent variable.

Although linear regression is originally a concept derived from the field of statistics, it is used extensively in machine learning for the purposes of understanding the relationship between independent input variables and a dependent output variable.

There are many different types of linear regression, the most notable of which are simple linear regression(where there is only a single input variable) and multiple linear regression(where there are multiple input variables).

The main points of differentiation between different types of linear regression are concerning the number of independent variables and the type of relationship between the independent and dependent variables.

Linear regression can be defined as a simple linear equation due to the fact that ultimately, it is a line drawn across a set of data points that is designed to model those data points in the most accurately possible way.

Here is what a linear equation looks like:

y = ax + c

Where a represents the slope of the line and the c represents its y-intercept.

Example of Linear Regression:

In order to better understand linear regression, let's go over an example. Here is a graph showing the heights of fathers in comparison to their sons:

Figure 1: Comparison of fathers' heights to their sons

Here is a code snippet using the R programming language that will calculate the linear regression line for the graph above:

library(tidyverse)
library(HistData)
library(caret)

# Obtain the dataset data and rename the `childHeight` column to `son`
galton_heights <- GaltonFamilies %>%
  filter(childNum == 1 & gender == "male") %>%
  select(father, childHeight) %>%
  rename(son = childHeight)

y <- galton_heights$son
test_index <- createDataPartition(y, times = 1, p = 0.5, list = FALSE)

# Divide the original dataset into train and test sets
train_set <- galton_heights %>% slice(-test_index)
test_set <- galton_heights %>% slice(test_index)


# Fit linear regression model
fit <- lm(son ~ father, data = train_set)

# Plot the graph and the linear regression line
plot(galton_heights, pch = 16, col = "blue")
abline(fit, col="red", lwd=5)

And here is the linear regression line that the above code produces:

Figure 2: Linear regression line in graph

Ok, that's it for this blog post on what Linear Regression is. I hope you found it useful!

Conclusion

Thanks for reading this blog post!

If you have any questions or concerns please feel free to post a comment in this post and I will get back to you if I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn and subscribe to my YouTube channel.

Basic Overview of Summary Functions using Pandas

Sean Atukorala — Sat, 10 Sep 2022 20:41:14 +0000

Hello everyone, in this blog post I'll cover some Summary Functions that the Pandas Python Data Analysis Library offers. Let's get started!

List of Summary Functions

Describe Function

The describe() function is used to obtain a informative statistical summary of a given Pandas DataFrame. The data that is displayed if the DataFrame contains numerical columns include the following(which is shown only the numerical columns):

count - Amount of not-null values
mean - Average value of the column values
std - Standard deviation of the column values
25% - Shows the value of the 25th percentile
50% - Shows the value of the 50th percentile
75% - Shows the value of the 75th percentile
max- Maximum value contained in the column values

Note: The percentile value of the column data indicate how many of the values that are less than a given percentile. A percentile is a value on a scale of 100 that indicates the percent of a dataset that is equal to or below it

Info Function

The Info() function is used to display information about the DataFrame that it is used on. The information provided by it include:

Number of columns in the DataFrame
Column labels
Column data types
Memory usage
Range index
Number of cells in each column

Note: The info() function does not have a return value

Value Counts Function

The value_counts() function returns a series containing counts of unique values.

The output object will be ordered in descending fashion. This means the first element is the most frequently-occurring element.

Note: this function excludes NA values by default

Conclusion

Well that's it for this post! Thanks for following along in this article and if you have any questions or concerns please feel free to post a comment in this post and I will get back to you when I find the time.

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn, subscribe to my YouTube channel.

How to detect outliers using Pandas, Matplotlib and Python

Sean Atukorala — Sat, 03 Sep 2022 18:19:54 +0000

Hello everyone, in this tutorial we’ll cover how to detect and remove outliers in a dataset using the Matplotlib and Pandas Machine Learning libraries.

Here is the GitHub repo containing the full version of code shown here along with the train.csv data file(from which the data for the below tutorial is used): House Prices Advanced GitHub Repo

First, let’s display all the datapoints in the scatterplot where the two variable being plotted are OverallQual and SalePrice. This can be done using the following code snippet:

plot, ax = plt.subplots(1, 1, figsize = (12, 5))
sns.scatterplot(data = train_data, x = "OverallQual", y = "SalePrice", c = ["blue"], ax = ax)

Here is the resulting graph that is created:

Figure 1: Initial Scatter Plot Graph

Next, let’s work towards identifying the outliers in the above graph.

plot, ax = plt.subplots(1, 2, figsize = (12, 5))
outliers = (train_data["OverallQual"] == 10) & (train_data["SalePrice"] <= 250000)
sns.scatterplot(data = train_data, x = "OverallQual", y = "SalePrice", c = ["red" if is_outlier else "blue" for is_outlier in outliers], ax = ax[0])
train_data.drop(train_data[(train_data["OverallQual"] == 10) & (train_data["SalePrice"] <= 250000)].index, inplace = True)
sns.scatterplot(data = train_data, x = "OverallQual", y = "SalePrice", ax = ax[1], c = ["blue"])
plt.show()

Here’s an explanation for the code above:

plt.subplots(1, 2, figsize = (12, 5)): We're creating two subplots inside one Matplotlib graph. The first parameter is the number of rows and the second defines the number of columns
outliers = ...: Here we are identifying the outliers in the dataset using some specified criteria, namely: (train_data["OverallQual"] == 10) and (train_data["SalePrice"] <= 250000)
First sns.scatterplot: Here we are defining the first scatterplot that will display the original data with the outliers highlighted in red. The red highlighting is caused by the conditional statement in the c option("red" if is_outlier else "blue" for is_outlier in outliers)
Next we remove the outliers from our dataset via the pandas.DataFrame.drop() method(more on this method in the official docs)
Second sns.scatterplot: Here we are defining the second scatterplot that will display the data with the outliers removed
plt.show(): This method displays the graph that we just specified

And here’s the resulting graph from running the code snippet above:

Figure 2: First Graph showing the outlier in red, Second Graph shows the data with the outlier removed

And that’s one way to remove outliers from a dataset using Matplotlib, Pandas and Python! Thanks for following along.

Conclusion

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn, subscribe to my YouTube channel.

Basic Machine Learning and Statistics Concepts Explained

Sean Atukorala — Sun, 21 Aug 2022 20:41:44 +0000

Hi guys, this blog post is for clarifying and explaining some common machine learning and statistics concepts that can be very helpful to anyone new to the field of data science and machine learning.

Without further ado, let's get into it!

Some Basic Machine Learning Concepts

Machine Learning model: A Machine Learning model is a program that can make predictions or decisions after being trained on a newly created dataset
Training set: A training set is a subset to train a model
Test set: A test set is a subset to test the trained model The training and test sets are two separate collections of data that are used to measure the accuracy of the Machine Learning model that you want to train
DataFrame: DataFrame is a data structure that is part of the Pandas library
Supervised machine learning: This type of machine learning is where some input data is provided to a mapping function in order to obtain a specific output
Unsupervised machine learning: This type of machine learning is characterized by having no correct output for a provided input. Accordingly, no correct answer exists. The goal of this type of machine learning is to analyze data in its entirety and discover facts about the underlying structure
Semi-Supervised learning: This type of machine learning is a combination of supervised and unsupervised machine learning. Semi-supervised learning is most useful when the dataset at hand is a mixture of labeled and unlabeled data points.

Some Basic Statistics Concepts

Here are some statistics concepts that are useful to know:

Mean: This is the average value in a set of data points
Median: This is the middle number in a set of data points
Mode: This is the most frequent number in a set of data points
Standard Deviation: This is a measure that tells you how dispersed the data points in a data set are in relation to the mean value(average value)
Interquartile Range: This is a measure of spread in statistics. Interquartile range is also known as the "middle fifty" and is calculated using the formula:

IQR(Interquartile Range) = Q3(Third quartile) - Q1(First quartile)

Skewness: This is a measure of asymmetry of the probability distribution of a dataset about its mean
Variance: This is a measure of the variation among values in a dataset
Covariance: This is a measure of how two variables in a data set will change together. A positive covariance indicates a positive relation between the two variables and visa versa for the negative.
Normal Distribution: This is a probability distribution function that looks like a bell. Here is an image of a normal distribution:

Figure 1: Normal Distribution Graph

P-value: This is a statistical measurement tool used to measure the probability of obtaining a desired result or outcome from statistical hypothesis testing

Conclusion

If you found this article helpful please share it and make sure to follow me on Twitter and GitHub, connect with me on LinkedIn, subscribe to my YouTube channel.