DEV Community: John Ojo

[Boost]

John Ojo — Wed, 10 Sep 2025 04:07:33 +0000

John Ojo

Aug 2 '25

Understanding Linear Regression Inside-Out: Practical Implementation with scikit-learn and TensorFlow

#machinelearning #python #ai #linearregression

Comments

4 min read

[Boost]

John Ojo — Sat, 06 Sep 2025 16:21:44 +0000

Understanding Linear Regression Inside-Out: Gradients, Loss, and Learning from Scratch

John Ojo ・ Jul 19

#machinelearning #linearregression #python #ai

Understanding Linear Regression Inside-Out: Practical Implementation with scikit-learn and TensorFlow

John Ojo — Sat, 02 Aug 2025 15:38:14 +0000

Welcome back to Part 2 of the series:
“Understanding Linear Regression Inside-Out"

In Part 1, we explored the foundations of linear regression by building it from scratch using NumPy. We delved into concepts like prediction, loss gradient, and gradient descent, gaining an inside-out understanding of how linear regression models learn.

While implementing these algorithms manually is essential for learning, real-world machine learning applications require more efficient, scalable, and production-ready tools. In this second part, we shift our focus to practical implementations employing two widely used machine learning libraries: scikit-learn and TensorFlow.

We'll use the same dataset with identical data loading logic, so we'll skip that section (refer to Part 1's Data Processing section if needed). Although scikit-learn doesn't require normalization, we'll maintain the same preprocessing steps.

Let’s dive in.

The source code on GitHub : Linear Regression Using Library

Prerequisites

jupyter notebook setup
A local Python env setup with packages: numpy, matplotlib, sklearn, and tensorflow

scikit-learn Implementation

scikit-learn's LinearRegression model is straightforward to use. You instantiate it, fit it to your training data, and then call predict() on your test data. During the fitting process, the model learns the best parameters (weights and bias), the same goal we achieved with gradient descent in Part 1.

# Train Linear Regression model using sklearn
# Initialize and train the Linear Regression model
sklearn_model = LinearRegression()
sklearn_model.fit(X_train_norm, y_train)

# Make predictions on validation and test sets
val_predictions = sklearn_model.predict(X_val_norm)
y_predict_sklearn = sklearn_model.predict(X_test_norm)

# Calculate mean squared error for both validation and test sets
val_loss = mean_squared_error(y_val, val_predictions)
test_loss = mean_squared_error(y_test, y_predict_sklearn)

That's all you need to train a Linear Regression model using scikit-learn. From there, you can evaluate performance like mean squared error and plot actual vs predicted value comparisons.

print(f"Validation MSE: {val_loss:.4f}")
print(f"Test MSE: {test_loss:.4f}")

# Create a plot comparing actual vs predicted sales
utils.plot_predictions(y_test, y_predict_sklearn, 'Predicted vs Actual Sales', 'Actual Sales', 'Predicted Sales')

# Print the first 25 actual and predicted values for comparison
for i in range(25):
    print("Print actual vs predicted values")
    print(f"Actual: {y_test[i]}, Predicted: {y_predict_sklearn[i]:.1f}")

If you compare this plot to the one from Part 1, you'll see we get the same results

Tensorflow Implementation

TensorFlow takes a neural network approach. We'll create a simple one-layer neural network, use summary() to view the model architecture, and then model.fit() will learn the best parameters (weights and bias), just like our gradient descent implementation in Part 1, and then call predict() on your test data to make new predictions

# Train Linear Regression model using TensorFlow
# Define a simple Sequential model with a Dense layer (1 unit for linear regression)
tf_model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(X_train_norm.shape[1],)),
    tf.keras.layers.Dense(1)  # Linear regression (no activation)
])

# Display model architecture
tf_model.summary()

# Configure the model training parameters
tf_model.compile(
    optimizer=tf.keras.optimizers.SGD(learning_rate=0.01),
    loss='mse', # Mean Squared Error loss
    metrics=['mae'] # Track Mean Absolute Error during training
)

# Train the model
history = tf_model.fit(
    X_train_norm, y_train,
    validation_data=(X_val_norm, y_val),
    epochs=200,
    batch_size=32
)

# Evaluate on test data
test_loss, test_mae = tf_model.evaluate(X_test_norm, y_test, verbose=0)

# Generate predictions on test data
y_predict_tf = tf_model.predict(X_test_norm).flatten() # Flatten converts 2D array to 1D for easier comparison

That's all you need to train a Linear Regression model using Tensorflow. You can also evaluate performance like mean squared error and plot actual vs predicted value comparisons.

print(f"Test Loss (MSE): {test_loss:.4f}")
print(f"Test Mean Absolute Error: {test_mae:.4f}")

# Create a plot comparing actual vs predicted sales
utils.plot_predictions(y_test, y_predict_tf, 'Predicted vs Actual Sales', 'Actual Sales', 'Predicted Sales')

# Print the first 25 actual and predicted values for comparison
for i in range(25):
    print("Print actual vs predicted values")
    print(f"Actual: {y_test[i]}, Predicted: {y_predict_tf[i]:.1f}")

Comparing this output to our results from Part 1 and those from scikit-learn, you'll see they are virtually identical.

Conclusion

In this second part of the series, we transitioned from building linear regression from scratch to using powerful machine learning libraries, scikit-learn and TensorFlow, to implement the same model more efficiently.

What’s particularly rewarding is how both approaches — manual and library-based — converged to similar results. This reinforces the foundational concepts covered in Part 1, while also demonstrating the real-world value of leveraging mature frameworks for faster development and scalability.

Understanding what's happening under the hood, while also knowing how to use the tools available, gives you a balanced and confident footing in your machine learning journey.

Thanks for following along! If you found this helpful or have questions, feel free to reach out. You can also buy me a coffee — happy learning!

Understanding Linear Regression Inside-Out: Gradients, Loss, and Learning from Scratch

John Ojo — Sat, 19 Jul 2025 13:17:25 +0000

This article is beginner-friendly for those looking to break into Artificial Intelligence/Machine Learning, as well as for those with experience who want to understand what goes on behind the scenes of most ML algorithms.
This will be a two-part series. In this part, I will explain the fundamentals of linear regression models and how they work behind the scenes to train and make predictions. In the next section, I will demonstrate how to utilize scikit-learn and TensorFlow to achieve this.
Understanding the fundamentals is always valuable—when things aren’t working as expected, a solid foundation helps us troubleshoot more effectively.

The source code on GitHub : Basic Linear Regression

Prerequisites

jupyter notebook setup
A local Python env setup with packages: numpy, matplotlib, sklearn, and tensorflow

Introduction

Supervised machine learning trains models using labeled data, where each example consists of both input features and the corresponding target variable. After training, the model can make predictions on new data points.

Linear regression is one of the simplest yet most foundational algorithms in machine learning. It models the relationship between input features and a continuous target variable, forming the basis for more complex models used in modern predictive analytics. Linear regression is particularly useful for predicting continuous values like weather forecasts, sales figures, or stock prices.

Libraries like scikit-learn and TensorFlow make training a linear regression model straightforward using just a few lines of code. However, this convenience can become a black box that obscures how the model learns, which is why, in this article, we’ll demystify linear regression by building it completely from scratch — using only NumPy for mathematical operations with just a touch of TensorFlow for data normalization.

We’ll work with the classic advertising dataset, where the goal is to predict sales based on advertising spend across TV, radio, and newspapers. Along the way, we’ll cover:

Loading and preprocessing real-world data;
Implementing the prediction function, loss calculation, and gradient descent manually;
Evaluating the model on a separate test set; and
Visualizing model performance with clear plots.

At the end, you’ll not only understand what linear regression does, but also why it works and how to implement it step by step. Whether you're new to machine learning or looking to strengthen your foundations, this hands-on project will sharpen your skills.

Stick with me—this will be a bit long, as we’re covering everything from loading and processing the data, to calculating loss, implementing gradient descent, making predictions, and doing it all manually.

Let’s dive in.

Data Processing

In this project, we’ll use the advertising dataset, a well-known dataset in the machine learning community. It’s often used for regression problems, particularly for demonstrating how different types of advertising affect product sales.

The dataset contains 200 rows, and includes the following columns:

tv Advertising budget spent on TV
radio Advertising budget spent on radio
newspaper Advertising budget spent on newspapers
Sales Product sales

Here’s a quick look at the first few rows:

id   tv     radio  newspaper  sales
1   230.1   37.8    69.2      22.1
2    44.5   39.3    45.1      10.4
3    17.2   45.9    69.3       9.3
4   151.5   41.3    58.5      18.5

In our model:
The input features (X) will be: tv, radio, and newspaper
The target variable (y) will be: sales

This setup is ideal for linear regression because we want to predict a continuous value (sales) based on numerical inputs.

Before training, we’ll split the data into training, validation, and test sets, and normalize the features to ensure they are all on a similar scale. That way, our gradient descent optimizer can converge more efficiently.

Let’s get started by loading the needed packages.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
import tensorflow as tf
import utils as utils

NumPy: a popular library for scientific computing
Matplotlib: a popular library for plotting data
scikit-learn: just for splitting the dataset
TensorFlow: only for normalizing the input features
utils.py: contains helper functions needed to load the CSV data, extract the features and target from a dataset, and plot relevant graphs (loss, predictions).

Next is loading and preprocessing the data

raw_data = utils.load_csv_data('<local-path>/data/Advertising.csv')

feature_keys = ['tv', 'radio', 'newspaper']
target_key = 'sales'

X, y = utils.prepare_data(raw_data, feature_keys, target_key)

print("X Shape: ", X.shape)
print("X length: ", len(X))
print("X first 5 features: ", X[:5])
print("X type: ", type(X))

print("y Shape: ", y.shape)
print("y length: ", len(y))
print("y first 5 features: ", y[:5])
print("y type: ", type(y))

Make sure to update the local-path. The utils.load_csv_data() function loads the dataset from the CSV file. After loading the raw data, we’ll convert the relevant columns into NumPy arrays for processing. utils.prepare_data() takes the raw data, the feature keys, and the target key, then returns a tuple of the features and target NumPy arrays. We print the shape, length, type, and the first 5 items in the features and target arrays. This enables us to get a sense of the data we are working with.

Next, we can plot our features vs the target graph to visualize our data

# Plot the first 5 features vs target
print("Plot first 5 X vs y")
utils.plot_features_vs_target(X[:5], y[:5], feature_keys, target_key)

# Plot the entire features vs target
print("Plot entire X vs y")
utils.plot_features_vs_target(X, y, feature_keys, target_key)

Next, we process the data and get it ready for training.
To simulate a realistic training workflow, we split the data into three parts:

75% for training
12.5% for validation (model tuning to ensure the model is not overfitting on the training data)
12.5% for final testing (unseen data)

To help gradient descent converge efficiently, we normalize the features using TensorFlow’s Normalization layer, trained on the training set only to prevent data leakage.

# Step 1: Split the dataset into training, validation, and test sets
# First split: 75% training, 25% temporary set
X_train, X_temp, y_train, y_temp = train_test_split(X, y, test_size=0.25, random_state=55)
# Second split: Divide the temporary set into validation and test sets (50% each, which is 12.5% of the original data each)
X_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.50, random_state=55)

# Step 2: Normalize using TensorFlow (adapt on train only)
# Create a normalization layer that will standardize the features
normalizer = tf.keras.layers.Normalization(axis=-1)
# Fit the normalizer only on training data to avoid data leakage
normalizer.adapt(X_train)  # Only fit on training data

# Step 3: Transform all datasets using the fitted normalizer
# Apply normalization to training data and convert to numpy array
X_train_norm = normalizer(X_train).numpy()
# Apply same normalization to validation data
X_val_norm = normalizer(X_val).numpy() 
# Apply same normalization to test data
X_test_norm = normalizer(X_test).numpy()

# Print shapes of all datasets to verify the splitting worked correctly
print("X_train_norm Shape: ", X_train_norm.shape)
print("y_train Shape: ", y_train.shape)
print("X_val_norm Shape: ", X_val_norm.shape)
print("y_val Shape: ", y_val.shape)
print("X_test_norm Shape: ", X_test_norm.shape)
print("y_test Shape: ", y_test.shape)

At this point, our data is clean, split, and ready to be used for training a linear regression model from scratch. Next up, we’ll implement the linear regression components one by one: prediction, loss function, gradient calculation, weight and bias update using gradient descent, and training. Apologies in advance as I will not go deep into the mathematical equations used, but there are resources available online if you need to explore this further. Let’s get into it.

Prediction Definition

Predictions are how you predict an output f(x) given input features, weights, and a bias. In linear regression, the predicted output is computed as:

f_{w,b}(x^{(i)}) = wx^{(i)} + b / 𝑊⋅𝑋 + 𝑏

where:

W is the weight vector (one weight per feature, since we have three features, then the weight would be a vector of three values)
X is the input feature vector
b is the bias (intercept term)

The training process aims to find the appropriate weights and bias that, when applied to new input values, produce accurate predictions. Essentially, training is about learning these optimal parameters (weights and bias) for future use on new data points.

The process of finding optimal weights and bias involves:

Initializing the parameters (weights and bias) with random values
Making predictions using the current parameters
Calculating the prediction error using a loss function and storing the result
Updating the weights and bias to new values using gradient descent
Repeating steps 2-4 for multiple training epochs
Monitoring the loss over time—decreasing values indicate the model is learning and converging.

This process yields trained parameters (weights and bias) that can make accurate predictions on new data.

def predict(
    X: np.ndarray, 
    W: np.ndarray, 
    b: float
) -> np.ndarray:
    """
    Predict target values using linear regression.

    Args:
        X (np.ndarray): Feature matrix of shape (n_samples, n_features)
        W (np.ndarray): Weight vector of shape (n_features,)
        b (float): Bias term

    Returns:
        np.ndarray: Predicted values of shape (n_samples,)
    """

    # Calculate predictions using the linear regression formula: f(x) = X·W + b
    # - np.dot(X, W) computes the matrix multiplication between features and weights
    # - Adding b applies the bias term to each prediction
    f_x = np.dot(X, W) + b

    return f_x

f_x = np.dot(X, W) + b uses Vectorization to compute f(x).

Loss

Next, we would compute the loss. Loss refers to the difference between the predicted values and the actual (true) values of the target variable. It quantifies how well or poorly the model is performing.
The loss function quantifies how far our predictions are from the actual values. Low loss values indicate good model performance, while high loss values suggest the model needs improvement.
We’ll use mean squared error (MSE) as the loss function. It’s defined as:

\frac{1}{2m} \sum\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) - y^{(i)})^2

def mean_squared_loss(
    X: np.ndarray, 
    y: np.ndarray, 
    W: np.ndarray, 
    b: float
) -> float:
    """
    Compute the mean squared error loss.

    Args:
        X (np.ndarray): Feature matrix of shape (m, n)
        y (np.ndarray): Target vector of shape (m,)
        W (np.ndarray): Weight vector of shape (n,)
        b (float): Bias term

    Returns:
        float: Mean squared error loss
    """

    # Get the number of training examples
    m = X.shape[0]

    # Calculate predictions using the predict function
    predictions = predict(X, W, b) 

    # Compute the squared differences between predictions and actual values
    squared_errors = (predictions - y) ** 2

    # Calculate the mean squared error loss with the 1/2m factor
    loss = np.sum(squared_errors) / (2 * m)

    return loss

Gradient and Gradient Descent

Next, we will compute a gradient. This calculates how much we need to change the weights and bias to reduce prediction error, helping the model learn during training.
The gradient is defined as:

\frac{\partial J(w,b)}{\partial w} = \frac{1}{m} \sum\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) - y^{(i)})x^{(i)}

\frac{\partial J(w,b)}{\partial b} = \frac{1}{m} \sum\limits_{i = 0}^{m-1} (f_{w,b}(x^{(i)}) - y^{(i)})

def compute_gradient(
    X: np.ndarray, 
    y: np.ndarray, 
    W: np.ndarray, 
    b: float
) -> tuple[np.ndarray, float]:
    """
    Compute the gradient of the cost function with respect to parameters W and b.

    Args:
        X: Input features, shape (m, n) where m is number of examples and n is number of features
        y: Target values, shape (m,)
        W: Weight parameters, shape (n,)
        b: Bias parameter

    Returns:
        tuple: Gradients with respect to W and b
            - d_dw: Gradient with respect to W, shape (n,)
            - d_db: Gradient with respect to b, scalar
    """

    # Get the number of examples (m) and features (n)
    m, n = X.shape

    # Calculate model predictions using current parameters
    predictions = predict(X, W, b) 
    # Compute the error (difference between predictions and actual values)
    error = predictions - y

    # Calculate gradient for weights by taking dot product of X transpose and error, then normalize by m
    # This is the partial derivative of the cost function with respect to W
    d_dw = np.dot(X.T, error) / m

    # Calculate gradient for bias by summing all errors and normalizing by m
    # This is the partial derivative of the cost function with respect to b
    d_db = np.sum(error) / m

    return d_dw, d_db

After this, we create the gradient descent function that enables us to perform one step of gradient descent to update model parameters.
The formula used to update the model's parameters is as follows:

\alpha \frac{\partial J(w,b)}{\partial w}

\alpha \frac{\partial J(w,b)}{\partial b}

def gradient_descent(
    X: np.ndarray, 
    y: np.ndarray, 
    W: np.ndarray, 
    b: float, 
    learning_rate: float
)-> tuple[np.ndarray, float]:
    """
    Perform one step of gradient descent to update model parameters.

    Args:
        X: Input features, shape (m, n) where m is number of examples and n is number of features
        y: Target values, shape (m,) or (m, 1)
        W: Current weight parameters, shape (n,) or (n, 1)
        b: Current bias parameter
        learning_rate: Step size for the gradient descent update

    Returns:
        tuple: Updated weights W and bias b after one step of gradient descent
    """

    # Calculate gradients for weights and bias using current parameters
    dW, db = compute_gradient(X, y, W, b)

    # Update weights by subtracting the learning rate multiplied by the gradient
    W =  W - (learning_rate * dW)
    # Update bias by subtracting the learning rate multiplied by the gradient
    b = b - (learning_rate * db)

    return W, b

Model Training

Now that we have implemented the core functionalities needed, i.e., prediction, loss, gradient computation, and training parameter updates, we can train our linear regression model using gradient descent.
The training process involves:

Calculating the loss using the current weights and bias (this predicts the output and then computes the MSE)
Updating the model parameters (weights and bias) using gradient descent
Repeating the process for the specified number of epochs (training steps)

We store the loss over time so that we can plot it. If the loss decreases over the epochs, we can say the model is converging and learning. If it keeps increasing, then the model is performing poorly, and we need to investigate the cause, since the loss should decrease during proper training.

def train(
    X_train: np.ndarray, 
    y_train: np.ndarray, 
    X_val: np.ndarray,
    y_val: np.ndarray,
    W: np.ndarray, 
    b: float, 
    learning_rate: float, 
    epochs: int
) -> tuple[np.ndarray, float, list[float], list[float]]:
    """
    Train a linear model using gradient descent optimization.

    Args:
        X_train: Training features, shape (n_samples, n_features)
        y_train: Training target values, shape (n_samples,)
        X_val: Validation features, shape (n_samples, n_features)
        y_val: Validation target values, shape (n_samples,)
        W: Initial weight matrix, shape (n_features,)
        b: Initial bias term
        learning_rate: Step size for gradient descent updates
        epochs: Number of training iterations

    Returns:
        tuple: Updated weights W, bias b, train loss history, and validation loss history after training
    """

    # Initialize empty lists to store loss values during training
    train_loss_history = []
    val_loss_history = []

    # Iterate through the specified number of training epochs
    for epoch in range(epochs):
        # Calculate and store the mean squared loss on training data
        train_loss = mean_squared_loss(X_train, y_train, W, b)
        train_loss_history.append(train_loss)

        # Calculate and store the mean squared loss on validation data
        val_loss = mean_squared_loss(X_val, y_val, W, b)
        val_loss_history.append(val_loss)      

        # Print progress every 100 epochs
        if epoch % 100 == 0:
            print(f"Epoch {epoch}: Train Loss = {train_loss:.4f}, Val Loss = {val_loss:.4f}")

        # Update model parameters (weights and bias) using gradient descent
        W, b = gradient_descent(X_train, y_train, W, b, learning_rate)

    # Return the trained model parameters and loss histories
    return W, b, train_loss_history, val_loss_history

Now that we have our train function in place, our next step is to run it. We will initialize the weights and bias to zero, use a learning rate of 0.01, and run this process 1000 times.

# Initialize weights and bias
W = np.zeros(X_train.shape[1])  # shape (n_features,)
b = 0.0
learning_rate = 0.01  # Step size for gradient descent
epochs = 1000  # Number of training iterations

# Train the linear regression model
W_trained, b_trained, train_losses, val_losses = train(X_train_norm, y_train, X_val_norm, y_val, W, b, learning_rate, epochs)

# Print the final trained parameters
print(f"Training parameters Weight: {W_trained}, bias {b_trained}")

# Visualize the training and validation loss over epochs
# This helps to monitor model convergence and potential overfitting
utils.plot_loss_curve(train_losses, val_losses)

As you can see, the final weights are [ 3.78713104 2.88855073 -0.14126015], and the bias is 13.92606568567291.
Note: In our case, the model converged after 200 steps, with no further significant change in the loss. We could modify the training process to implement early stopping based on a patience threshold—I'll leave that as an exercise for you.

Testing The Trained Model

We can now use these learned parameters to make predictions on our test dataset.

# Make predictions on the test set using trained weights and bias
y_predict = predict(X_test_norm, W_trained, b_trained)

# Calculate the mean squared loss on the test set
y_predict_loss = mean_squared_loss(X_test_norm, y_test, W_trained, b_trained)   

# Print the test loss
print(f"Test Loss: {y_predict_loss}")

# Create a plot comparing actual vs predicted sales
utils.plot_predictions(y_test, y_predict, 'Predicted vs Actual Sales', 'Actual Sales', 'Predicted Sales')

# Print the first 25 actual and predicted values for comparison
for i in range(25):
    print("Print actual va predicted values")
    print(f"Actual: {y_test[i]}, Predicted: {y_predict[i]:.1f}")

The test loss is 1.129492604788525, which shows that our model did very well in predicting the outcome.

Conclusion

In this tutorial, we built a complete linear regression model from scratch without relying on high-level machine learning libraries. We walked through each fundamental step from loading and preprocessing a real-world dataset to implementing the prediction and loss functions, computing gradients, and training with gradient descent.

Along the way, we used the advertising dataset to model the relationship between advertising spend and product sales. By manually implementing each component, we gained a clearer understanding of how linear regression works under the hood.

This hands-on approach reinforces not just the math behind machine learning, but also the practical workflow required to evaluate and test a simple model.

You typically won't need to implement this from scratch as we did, since packages like scikit-learn and TensorFlow handle the heavy lifting, unless you're developing entirely new ML algorithms. However, as I mentioned earlier, understanding the fundamentals helps you troubleshoot when models are not performing as expected or failing to converge.

In part two of this series, I'll show you how to accomplish the same task using scikit-learn and TensorFlow with just a few lines of code. This approach abstracts away the underlying mathematical complexity of linear regression, however the understanding you’ve gained from this article will give you a real appreciation for the work these libraries do behind the scenes. There's still plenty more to explore! Topics like regularization, feature engineering, and others offer fascinating extensions to what we've covered.

Thanks for following along! If you found this helpful or have questions, feel free to reach out. You can also buy me a coffee — happy learning!

Consuming GraphQL API In ASP.NET Core (Part 2)

John Ojo — Thu, 28 Jan 2021 10:13:05 +0000

Here's the second part of the post HotChocolate: Introduction to GraphQL for ASP.NET Core (Part 1). In part 1, we developed a GraphQL API using ASP.NET. In this part, we will focus on consuming the endpoint in an ASP.NET Blazor WebAssembly application.

We will create a page that will display all the Employees along-side their Department, another page that displays a single Employee and a page with a form to create an Employee.

The code for this sample can be found on the CloudBloq/GraphQLSampleAppUI repository on GitHub.

Prerequisites

Visual Studio 2019 (version 16.8.0 upwards)

Create the Solution and Project

After opening VS, click the Create a new project link then search for Blazor and select it:

Give the project and solution a name GraphQLSampleAppUI then click create:

Select the Blazor WebAssembly App option then click create:

Install Dependencies

We will make use of GraphQL.Client(v 3.2.0) library to consume the GraphQL API. To install the package, right click the solution in the solution explorer and select Manage NuGet Packages for Solution. Under the Browse section, search for GraphQL.Client and click on it, then in the preview panel click the Install button:

Also, install the modernhttpclient and GraphQL.Client.Serializer.Newtonsoft packages respectively.

Consume The API

In part 1, to test the Query or Mutation, we used Banana Cake Pop:

In this case, we will make use of the GraphQL.Client library to consume the GraphQL API. We will write all the code required to perform the Query and Mutation in a folder. Add a new folder called DataAccess.

Query

We will create a generic method that can be used to perform all the Query we need to perform. In the DataAccess folder, add a new C# class file called Query.cs:

The way GraphQL.Client works is to first create an instance of GraphQLHttpClient using an instance of GraphQLHttpClientOptions (which takes in the URI of the GraphQL Server) then use an instance of the GraphQLRequest to create the Query or the Mutation string that will be sent to the Server.

Line 15 defines the field graphQLHttpClient of type GraphQLHttpClient. Line 26 assigns a new instance (using an instance of GraphQLHttpClientOptions) to the graphQLHttpClient field. The other parameter is used to specify the deserializer used to deserialize the output.

Note that if you modified line 8 in launchSettings.json in part 1, you also have to modify line 19 here.

Lines 30 to 54 define the generic method ExceuteQueryReturnListAsyn. The generic type T is the type the output will be deserialized into. The parameter graphQLQueryType is the Query type and the parameter completeQueryString will contain the complete Query. From the example below, graphQLQueryType will be allDepartmentsWithEmployee and completeQueryString will be the entire string.

query{
  allDepartmentsWithEmployee{
    name
    employees{
      email
      name
    }
  }
}

In line 34, an instance of GraphQLRequest is created which takes in the complete query string. The Query is sent to the server in line 39. The result gotten from the server will contain the Query type in the beginning so we replace it with an empty string in line 42. The output is deserialized in line 46 to the generic type.

The ExceuteQueryReturnListAsyn is used when the output we are expecting is an array of the generic type. The other method ExceuteQueryAsyn in line 56 is the same as ExceuteQueryReturnListAsyn except that it should be used when we are expecting a single object of the generic type as the output. The ExceuteQueryReturnListAsyn and ExceuteQueryAsyn methods can be used to perform any Query we need and deserialize the output accordingly.

Mutation

In the DataAccess folder, add a new C# class file called Mutation.cs:

This code in this class follows the same pattern as the Query.cs. Line 27 defines the generic method ExceuteMutationAsyn, it takes in the same parameter as the ExceuteQueryAsyn, sends the Mutation to the server and deserializes the output to the generic type T.

The Models

We will need the models that the Query and Mutation output will be deserialized into. Add a new folder Model in the DataAccess folder. Add a new C# class Employee.cs to the Model folder:

Add another C# class file Department.cs to the Model folder:

We need another model that will be used for the form used to create an Employee. Add another C# class file CreateEmployeeModel.cs to the Model folder:

The validations added to the properties (Required, EmailAddress...) via attributes will be used by the form to validate the input.

We need another model that will be used to hold the output of response when we create an Employee. Add another C# class file CreateEmployeeReturnModel.cs to the Model folder:

Blazor Pages

First, we will create the page that will be used to display all the Employees. In the Pages folder, add a new Razor Component called EmployeeView.razor:

The variable AllEmployees on line 43 is used to store the Employees. You see how I called the ExceuteQueryReturnListAsyn passing in the type Employee and the parameters. The variable AllEmployees is checked first if it contains any item on line 13, if it does then the data is displayed in the table in lines 19 to 39.

Next, we will create the page that will be used to display the individual Employee detail. In the Pages folder, add a new Razor Component called ViewEmployee.razor:

The ExceuteQueryAsyn method is used to get the Employee details in line 54. The Id is gotten from the URI and is attached to the string sent to the server in line 51.

Next, we will create the page that will be used to create an Employee. In the Pages folder, add a new Razor Component called CreateEmployee.razor:

Lines 12 to 33 define the form used to receive the data. The form uses the model CreateEmployeeModel via the variable createEmployee. The method ExceuteMutationAsyn is called on line 56, and the type CreateEmployeeReturnModel is passed as the return type. Notice how the form data is bound to the string on line 53.

Finally, we need to modify the NavMenu.razor in the Shared folder:

Running the Application

Ensure the GraphQL server https://localhost:44351/graphql/ is running first. Right-click the solution and click Build Solution. After Build is successful, click CTRL + f5 or f5 to run the application. You should get:

If you click the Employee link, you should get:

Click the Create an Employee link and try to click the Create button without entering any data in the form, you should get the response:

If you enter an invalid email or an age that isn't between 20 and 50, you should get the validation failed error.

This isn't the end of learning Blazor and GraphQL. There are a lot of things to explore, especially in Blazor WebAssembly. The links below will help you if you are really interested in learning more:

Awesome! Please share it with anyone you think could use this information. Thanks for reading. As always, if you have any questions, comments, or concerns about this post, feel free to leave a comment below.

HotChocolate: Introduction to GraphQL for ASP.NET Core (Part 1)

John Ojo — Mon, 04 Jan 2021 10:38:07 +0000

In this post, I will walk you through how to get started with GraphQL in ASP.NET Core using HotChocolate.
In this part, we will only focus on GraphQL Server, configuring it with ASP.NET Core.
In part 2, We will focus on integrating the GraphQL endpoint into a Blazor WebAssembly application.

Introduction

GraphQL is a query language for your API, and a server-side runtime for executing queries by using a type system you define for your data. It lets you specify exactly what data you need.

HotChocolate is a .NET GraphQL platform that can help you build a GraphQL layer over your existing and new application. HotChocolate is very easy to set up and takes the clutter away from writing GraphQL schemas.

We will create a simple application that can be used to Create, Read, Update and Delete an Employee together with a department.

The code for this sample can be found on the CloudBloq/GraphQLSampleApp repository on GitHub.

Prerequisites

The Database

The database schema and data script can be found in the folder CloudBloq/GraphQLSampleApp/tree/master/GraphQLSampleApp/Script. Ensure to run the script in SQL Server Management Studio or in Visual Studio SQL Server Object Explore to set up the Database and tables.

Create the Solution and Project

We will need an ASP.NET Web API project. After opening VS, click the Create a new project link and select the ASP.NET Core Web Application, give the project a name GraphQLSampleApp and the solution a name GraphQLSampleApp click Create. Next, select the API, then click Create.

Install Dependencies

We need to install the HotChocolate.AspNetCore(v11.0.2) package. The package contains the GraphQL API's for ASP.NET. To install the package, right click the solution in the solution explorer and select Manage NuGet Packages for Solution. Under the Browse section, search for HotChocolate.AspNetCore and click on it, then in the preview panel click the Install button:
Also, install the Microsoft.EntityFrameworkCore, Microsoft.EntityFrameworkCore.SqlServer and Microsoft.EntityFrameworkCore.Tools packages respectively.

The Entities

We will make use of EntityFrameworkCore as the ORM. Entities (classes) are used to define the database objects (tables) and those entities are mapped to the database table in the DbContext.

We will define two Entities, one for the Department and the other for the Employee tables.
We will keep everything simple by using folders to hold the data access code, the GraphQL code and the Database Entities. In a real application, a Class library would be better for the different layers.

Create a folder called DataAccess in the GraphQLSampleApp project, create another folder called Entity inside the DataAccess folder.

Department Entity

Add a C# Class file called Department.cs to the Entity folder. This class will be used to define the Department entity. The code for this file is:

Line 9 is used to mark the DepartmentId property as the primary key of the entity, line 10 is used to ensure that the value of the DepartmentId is unique and auto generated if it is not provided by the user. Line 15 is used to define a relationship (navigation property) between the Department and Employee entity. The relationship:

public ICollection<Employee> Employees { get; set; }

means that a single Department can have many Employees assigned to it. This line will throw an error now unless you have created the Employee entity earlier.

Employee Entity

Add another C# class to the Entity folder called Employee.cs. After adding this class, the error gotten earlier in Department.cs will no longer be there. The code for this file is:

Line 24 defines the foreign key property used to link an Employee to his/her department. Line 25 defines the navigation property that states that an Employee can only have one department.

The DbContext

The DbContext will be simple and straight forward. In a real application, most of the database configurations are done in this file. Add a new C# Class file to the Entity folder, name it SampleAppDbContext.cs. The code for this file is:

Line 11 defines the Employee property which is used to map the Employee.cs entity to the Employee table in the database. Line 13 defines the Department property which is used to map the Department.cs entity to the Department table in the database.

Repositories

Next, we will write the layer that will send and receive data to and from the database. In the DataAccess folder, add another folder called DAO.

Department Repository

Add a C# class file called DepartmentRepository.cs to the DAO folder:

Lines 19 to 22 define the GetAllDepartmentOnly() method which is used to return a list of all the Departments.
Lines 24 to 29 define the GetAllDepartmentsWithEmployee() method which returns all the Departments along-side the Employee(s) in each Department. The navigation property defined in line 15 in Department.cs made it possible to return the Departments along-side the Employee(s).
Lines 31 to 36 define the CreateDepartment() method which takes in a new Department and adds it to the database.

Employee Repository

Add a C# class file called EmployeeRepository.cs to the DAO folder:

This class defines the GetEmployees() which returns a list of all the Employees, the GetEmployeeById() which takes an Id and returns the Employee only if the Id is valid and exists, the GetEmployeesWithDepartment() which returns a list of all the Employees along-side the Department, and the CreateEmployee() which takes in an Employee and adds them to the database.

GraphQL

If you are familiar with REST API, we use GET to fetch data from the data store, we use POST, PUT DELETE to modify the state of data. GET in REST API is same as Query in GraphQL. POST, PUT, DELETE, is same as Mutation. In GraphQL, there is also Subscription which is used to set up event listeners.

Query

Add a new C# class file called Query.cs to the DataAccess folder. This class will contain all the Queries we need to perform:

AllEmployeeOnly() on line 12 returns all the Employees; we inject an instance of the EmployeeRepository into this method by marking the input parameter with the [Service] attribute. AllEmployeeWithDepartment() on line 15 returns the Employees along-side the Department.
Line 18 defines GetEmployeeById(), the parameter ITopicEventSender eventSender is used to create a subscription with Topic "ReturnedEmployee". Anytime this method is called, the event is sent on line 22 and any method that subscribes to this event will receive the data.

Mutation

Add a new C# class file called Mutation.cs to the DataAccess folder. This class will contain all the Mutations we need to perform:

Line 15 defines CreateDepartment(). The parameter departmentName will be provided by the user. The parameter ITopicEventSender eventSender is used to create a subscription with Topic "DepartmentCreated".

Line 29 defines a method CreateEmployeeWithDepartmentId() that can be used to create a new Employee passing in an existing DepartmentId i.e. create an Employee with an existing department. The parameters name: string, age: int, email: string, and departmentId: int will be provided by the user. Notice we set the DepartmentId on line 37 to the departmentId passed by the user.

Line 44 defines CreateEmployeeWithDepartment(). This method is used to create a new Employee. In the process of creating the Employee, a new Department is created and the DepartmentId property of this Employee is set automatically to the DepartmentId of the newly created Department. Notice how this is done on line 52 by setting the navigation property to a new Department.

Subscription

Subscriptions are used to set up event listeners e.g. we can set up an event listener that responds when we create a new Department or Query the database for an Employee etc. Add a new C# class file called Subscription.cs to the DataAccess folder:

OnDepartmentCreate() on line 17 defines a subscriber of Topic "DepartmentCreated" likewise line 25 defines OnEmployeeGet() a subscriber of Topic "ReturnedEmployee".

Configuring GraphQL

We need to configure the application to make use of GraphQL. Replace the code in your Startup.cs with:

Line 32 is used to add the Service for GraphQL, Query, Mutation and Subscription. The AddQueryType(), AddMutationType() and AddSubscriptionType() takes in the Query, Mutation and Subscription classes respectively.

Line 55 is used to map the endpoint to use GraphQL.
The middleware on line 50 is used to configure the application to ensure the Subscription gives the output.

Line 27 adds the DbContext service; we configure this service to make use of SQL Server as the database. The connection string is defined in the appsettings.json file as:

Don't forget to change the Data Source value to that of the database you ran the schema and data script in.

Running the Application

Right-click the solution and click Build Solution. After Build is successful, click CTRL + f5 or f5 to run the application. In your browser, navigate to https://localhost:44351/graphql/, you should get

HotChocolate gives you a GraphQL IDE (Banana Cake Pop) right in your browser out of the box, you don't need to install any GraphQL IDE.

The methods defined in Query.cs, Mutation.cs and Subscription.cs have been categorized under Query, Mutation and Subscription respectively.

Test Query

In Banana cake pop, you can add a new tab by clicking the + in the top right corner of the IDE. You can run your schema by clicking the Execute/Play button in the top right corner. Add and run:

query{
  allDepartmentsWithEmployee{
    name
    employees{
      email
      name
    }
  }
}

You should get the response in the image below:
.

Try out other Query types, defining the Schema of the data you need.

Test Subscription and Query

Add and run the schema below:

subscription{
  onEmployeeGet{
    name
    departmentId
    department{
      name    
    }
  }
}

This Subscribes to the event "ReturnedEmployee" in line 28 of Subscription.cs. You won't get any output yet as shown in the image above, but you can see from the console that we have Subscribed to the event.

Launch https://localhost:44351/graphql/ in another browser tab. Next, we will run a Query that will trigger the event. Add and run the below in a new Banana Cake Pop tab in the new browser tab:

query{
  employeeById(id: 5){
    name
    email
  }
}

If you go back to the first browser tab where we subscribed to the event, you will get the response

If you change the value of the id and run the Query, you will see the changes in the Subscription.

Test Mutation

Add and run the schema below:

mutation{
  createEmployeeWithDepartment(name: "newemp", age: 30,
  email: "newemp@gmail.com", departmentName: "Mechanical Engr"){
    departmentId
    name
    department{
      departmentId
      name     
    }
  }
}

You would get

Whoa! We have come to the end of this tutorial. I only added the functions to Create and get an Employee and Department. You are free to explore more and add more functions.

Reference

Exploring ASP.NET Web API (.NET 5)

John Ojo — Thu, 26 Nov 2020 09:30:57 +0000

.NET 5 is finally out 🎉🥳 and it comes with lots of interesting new features that make software development easier, faster and better. In this article, we will focus on ASP.NET Web API.

Developing RESTful API using .NET has really improved over the years from ASP.NET MVC to ASP.NET Core and now .NET 5.
Every major release has come with great features that make developing RESTful API better. With .NET 5, it just got a whole lot easier and better by allowing you to focus more on your business logics and creating an Endpoint with few lines of code.

With .NET 5, you can configure your API to automatically make use of the OpenAPI Support and generate Swagger Docs with just the check of a box (no single line of code is required from you). Isn't it amazing?

To demonstrate this, we will create an Employee management system that will be used to Create, Read, Update and Delete an Employee's record.
The code for this sample can be found on the CloudBloq/EmployeeManagement repository on GitHub.

The technology used in this article are:

Prerequisites

Creating the Database

We will make use of CockroachDb to store the data. You can download CockroachDb from here.
After starting a cluster in insecure mode, we need to connect an SQL shell to the cluster in insecure mode. In the SQL Shell, create a database employee by running:

CREATE DATABASE employee;

Next, point the SQL shell to the newly created database by running:

 USE employee;

Next, create a table employee

CREATE TABLE employee(
    Id UUID PRIMARY KEY DEFAULT gen_random_uuid(), name STRING, department STRING, email STRING
);

The gen_random_uuid() is used to tell the DB that if we don't pass in a unique and valid UUID/GUID, it should generate one for us automatically.

Finally, we will insert a row into the table:

INSERT INTO employee (name, department, email) VALUES ('Test', 'Computer Science', 'Test@gmail.com');

You can check the data in the table by running:

SELECT * FROM employee;

Create the Solution and Projects

We will need an ASP.NET Web API project that will define the endpoints.
After opening VS, click the Create a new project link and select the ASP.NET Core Web Application, give the project a name Employee.API and the solution a name EmployeeManagement:

click Create
Next, select the ASP.NET Core Web API and verify that the check box labelled Enable OpenAPI support is checked (see image below):

click Create.
We will make use of a Class Library to define our Database Entity and data access codes. Add a new Class Library (.NET Standard) to the solution named Employee.DataAccess

Install Dependencies

We need to install the Npgsql.EntityFrameworkCore.PostgreSQL package. This package will be used to communicate with the database. To install the package, right click the solution in the solution explorer and select Manage NuGet Packages for Solution. Under the Browse section, search for Npgsql.EntityFrameworkCore.PostgreSQL and click on it, then in the preview panel, select the Employee.API and Employee.DataAccess checkbox so it will be installed only into the projects Employee.API and Employee.DataAccess and click the Install button:

We need to add a reference for project Employee.DataAccess to Employee.API. To do this, right click the Dependencies and click Add project Reference, select the Employee.DataAccess checkbox and click ok.

The Employee Entity

We will make use of EntityFrameworkCore as the ORM. Entities (classes) are used to define the database objects (tables) and those entities are mapped to the database table in the DbContext.

Create a folder called Model in the Employee.DataAccess project, and in that folder add a new class file called Employee.cs. The naming of this class/entity(Employee.cs) doesn't need to match with the table name(employee), but the property used to map the entity with the table in the DbContext must match. The code in the Employee.cs is:

Four properties (id, name, department and email) are defined to represent the four columns in the employee table.
The [Key] attribute on line 9 is used to specify that the property id is the primary key of the Entity, while the [DatabaseGenerated(DatabaseGeneratedOption.Identity)] attribute on line 10 tells EntityFramework that the value of the primary key will be generated by the database if the User doesn't provide a value.
The [Required] attribute defined on lines 13, 16 and 19 is used to specify that the name, department and email fields are required when sending an Employee's data to the Server via Create or Update; if any of the three is missing, the data for that employee won't be stored in the database.
The [EmailAddress] attribute tells EntityFramework that the value in the email field must be a valid email address.
If you are confused right now, I assure you that after we are done and we run the application, you will see how everything fits into the big picture.

Defining the DbContext

The DbContext class is an integral part of Entity Framework. It is used to perform most of the database configurations needed. Add a C# file to the Model folder called EmployeeManagementDBContext.cs:

Line 11 define a property (employee) of type DbSet<> which is used to map the Employee object to the employee table. The name of this property must match with the table name else an error will occur. The data in the employee table can now be accessed and manipulated through this property.

Accessing the Data

Create a new folder called Repository in the project Employee.DataAccess. Next, create an interface called IEmployeeDAO.cs:

Line 9 defines the GetAll method which will be used to return a list of all the Employees in the database. Line 13 defines the GetEmployeeById method which takes in the id as a parameter and returns the Employee with that id if one exists. Line 11 defines the CreateEmployee method which takes in a new Employee and adds it to the database. Line 15 defines the method UpdateEmployee which takes in an Employee and updates that record in the database only if the Employee exists in the database.
The delete method is left as an assignment for you, but I will explain what you need to do.

Next, we will create a concrete class EmployeeDAO.cs that will implement the IEmployeeDAO.cs interface which will then provide implementations to the methods. In the same Repository folder, create a C# class called EmployeeDAO.cs:

Line 12 defines a field dbContext of type EmployeeManagementDBContext. Through this field, we can access the employee property hence get access to the employee table data.

At this point you might be wondering that we haven't added the database connection string to the EmployeeManagementDBContext, but worry not. We will add it in a more secured file in Employee.API. Won't we have to write another set of logics to read the connection string since it will be in a different project? Not at all! One thing you should understand is that at runtime, i.e. when we call an endpoint, the Employee.API will be the entry point of the Http request. At that point, all other libraries (Employee.DataAccess in this case) that are being used by the Employee.API are also executed under the same Http request hence if those libraries need any configuration settings, and such settings aren't present in the library itself but present in the entry point (Employee.API), it will be gotten automatically from the entry point project. Confused? Don't worry. When we get to the Employee.API, you will understand better.

Lines 38 to 41 provide the implementation to the GetAll method. When we call the property employee of EmployeeManagementDBContext, we get back an object that implements the IQueryable and Enumerable interface. Hence, we are able to run some Linq query against the result. As you can see on line 40, we called the employee and converted the result to a list using the ToList() function. It can be broken down as:

    var employeeDbResult = dbContext.employee;
    return employeeDbResult.ToList();

The first line calls the employee property which we then apply the ToList() function on in the second line to convert the result to a List. Those 2 lines of code perform the same function as line 40.

Lines 26 to 36 provide an implementation to the GetEmployeeById method. We will ensure the id isn't null from the controller before sending it to this method so there is no need to check for null or empty values again here. After calling the employee property, we use the FindAsync(id) method to get the Employee with that id on line 28, then we return the Employee only if it exists.

Lines 19 to 24 provide the implementation to the CreateEmployee method. The API Controller will validate the Employee object before passing it to this method so there is no need to validate the input here again. We use the AddAsync() method on line 21 to add the new Employee to the database but take note that the database won't be updated until we save the changes as shown on line 22.

Lines 43 to 65 provide the implementation to the UpdateEmployee method. We check if the Employee exists in the database on line 45 before proceeding. Lines 49 to 57 are used to detach the entity before we update. The entity is being tracked due to line 45 hence, if we try updating it without detaching it first, we will get an error. We set the State of the Entity to Modified on line 59 hence notifying EntityFramewoke that the value(s) of that particular Employee has(have) been modified. We update the database and save the changes on lines 61 and 62.

With this, we are done with the data access code. Now we will move to the Employee.API project to create our endpoints.

Creating the Endpoints

First, we need to perform some configurations. Replace the code in the Startup.cs file with:

Lines 15, 38 to 42 and 54 are used to enable CORS on the endpoint. Line 36 is used to define the implementation of IEmployeeDAO interface that should be used during dependency injection which we specified as EmployeeDAO .
Lines 34 and 35 define the database configuration that should be used. It looks for the connection string named EmployeeManagementDBContext (this name can be anything else) from either the appsettings.json file, secrets.json file or Environment variable. It searches for the parameter EmployeeManagementDBContext from either of those 3 locations. We will be making use of the appsettings.json to store it. Replace the code in the appsettings.json with:

Next, add a new empty API controller to the Controllers folder, name it EmployeeController.cs. Replace the code in this file with:

We injected an instance of IEmployeeDAO through the constructor on lines 16 to 20.

Lines 57 to 65 define the action method GetAll which responds to the request GET /Employee used to get all the Employees.

Lines 41 to 54 define the action method GetById which responds to the request GET /Employee/{id} to get a particular Employee.

Lines 23 to 38 define the action method Create which responds to the request POST /Employee to create a new Employee.

Lines 68 to 79 define the action method Update which responds to the request PUT /Employee to update an Employee.

For the delete functionality, in the IEmployeeDAO.cs file add another method that returns a Task named DeleteEmployee which takes in the id of the Employee to delete. Add the implementation of this method to the EmployeeDAO.cs.
First thing to do in this method is to get the Employee by calling the GetEmployeeById(id), then use the .Remove(employeeObjectTodelete) on the dbContext.employee property to delete the employee and then save the changes to the database.
Next, in the EmployeeController.cs, create an action method Delete that takes in the id and marks it with the attribute [HttpDelete] then passes the id into the DeleteEmployee of IEmployeeDAO.cs and that is it.

Running the Application

Right-click the solution and click Build Solution. After Build is successful, click CTRL + f5 to run the application. If you have followed all the steps carefully, you shouldn't get any errors. Once the application starts running, you should get the image below in your browser:

Isn't it amazing that Swagger docs are generated automatically for all our endpoints. We don't need Postman or other tools to test the endpoints again. We can do everything we need to do inside the browser now.
Click the GET /api/Employee then click the Try it out button and Execute. You should get back the initial Employee we created earlier, alongside other Employees you created, if you did.
Click the POST /api/Employee then Try it out, you should get:

You can edit the details of the id, name, department and email. If you remove the id field, the database will auto generate a unique id. If you pass in an invalid id, the record won't be stored instead, you will get an error message. If you pass in a valid id that already exists in the database, you will get an error message stating that. If you remove either of the name, department or email field and Execute it you will get back an error stating that the missing field is required due to the [Required] attribute used on them. If the value of the email isn't a valid email, you will get back a response saying "The email field is not a valid e-mail address". If all went well, when you click the Execute button you should get back the newly created Employee.
Try the PUT /api/Employee. For this, you will need to copy one of the Employees from the GET /api/Employee and update any of the values except the id then Execute it.
Try out the GET /api/Employee/{id} too.

Whoa! We have come to the end of this tutorial. This is not the end of your learning, there are still lots of things to learn e.g. Authentication, Authorization, MiddleWare etc. Those are advanced topics. Trust me when I say you are done with the basis. With the knowledge you just gained, you can develop a RESTful API that any mobile, web or desktop application can use.

You can get the complete list of .NET 5 features here.

Fun with ASP.NET Core Blazor WebAssembly and ML.NET

John Ojo — Wed, 11 Nov 2020 16:11:49 +0000

What is Blazor WebAssembly?

In case this is your first time learning about Blazor, let me introduce you to what Blazor WebAssembly is all about.

Blazor is an open source and cross-platform web UI framework for building single-page apps using .NET and C# instead of JavaScript. Blazor is based on a powerful and flexible component model for building rich interactive web UI. You implement Blazor UI components using a combination of .NET code and Razor syntax: an elegant melding of HTML and C#. Blazor components can seamlessly handle UI events, bind to user input, and efficiently render UI updates.

What is ML.NET?

ML.NET is a free, cross-platform, open-source machine learning framework provided by Microsoft. It is made specifically for the .NET community. With the help of ML.Net framework, you can easily implement and integrate machine learning features into your existing or new .Net applications.
To start working on ML.Net, you don’t have to be a machine learning expert. You can just start building simple ML.Net applications while teaching yourself.

In this article, I am going to give a demo of a simple yet exciting ML.Net based project. We will develop an application that will be used to forecast bike rental service demand using univariate time series analysis.
The code for this sample can be found on the CloudBloq/BlazorWithML.NET repository on GitHub.

What You’ll Need to Get Started

Visual Studio 2019 with the ".NET Core cross-platform development" workload installed.

Needed Projects

We will be making use of three projects:

An ASP.NET Blazor WebAssembly project which serves as the UI to display the forecast.
An ASP.NET Web Api project which feeds the UI with data (the bike rental service demand forecast) from the service.
A service [C# Class Library (.NET Standard)] which will contain the logics used to forecast bike rental service demand.

If you clone the CloudBloq/BlazorWithML.NET repository, the database already resides in the Data folder in project BlazorWithML.NET.Api. When you run the application for the first time, the database will be automatically created and filled with the needed data. The database already contains the data that will be used to train the model.

Setting Up the Projects

ASP.NET Blazor WebAssembly project

Launch Visual Studio, click the Create a new project link. Next, type Blazor in the search box and click the first option that comes up (Blazor App):

Give the project and solution a name, e.g. BlazorWithML.NETPOC then click the Create button.

Next, select the Blazor WebAssembly App option and click the Create button. This will create the project and solution for you. This link will help you understand a Blazor WebAssembly project structure

ASP.NET Web Api project

Right click the Solution in the Solution Explorer and select Add -> New Project, search for Web Application and select the ASP.NET Core Web Application option:

click Next, give the project a name BlazorWithML.NET.Api, then click Create, select the API option:

and finally click Create.

C# Class Library (.NET Standard)

Right click the Solution in the Solution Explorer and select Add -> New Project, search for Class Library (.NET Standard) and select the C# version. Click Next, give the project a name Forecasting_BikeSharingDemandLib, then click Create.

Adding Projects Dependencies

We need to add the ML.NET and SqlClient package to the service: Right-click the Forecasting_BikeSharingDemandLib project, select Manage NuGet Packages for Solution. Under the Browse section, search for Microsoft.ML and click on it then click the Install button. After that, search for Microsoft.ML.Timeseries then click the Install button. Also search for System.Data.SqlClient and Install it.
We need to give the Api access to the Service: This can be done by adding a reference of the service to the Api. Right-click the Dependencies of project BlazorWithML.NET.Api and click Add project Reference, select the Forecasting_BikeSharingDemandLib checkbox and click ok.

Writing the Service

The service is the backend of the application. It contains the logics used to forecast bike rental service demand.
We will make use of univariate time series analysis algorithm known as Singular Spectrum Analysis to forecast the demand for bike rentals.

Understand the problem

In order to run an efficient operation, inventory management plays a key role. Having too much of a product in stock means not generating any revenue. Having too little product leads to lost sales. Therefore, the constant question is, what is the optimal amount of inventory to keep on hand? Time-series analysis helps provide an answer to these questions by looking at historical data, identifying patterns, and using this information to forecast values some time in the future.

The algorithm used in this article for analyzing data is univariate time-series analysis.

Univariate time-series analysis takes a look at a single numerical observation over a period of time at specific intervals, for example monthly sales. It works by decomposing a time-series into a set of principal components. These components can be interpreted as the parts of a signal that correspond to trends, noise, seasonality, and many other factors. Then, these components are reconstructed and used to forecast values some time in the future.

Create the needed files and folders

Right-click the Forecasting_BikeSharingDemandLib in the Solution Explorer then select Add -> New Folder and give it the name Model. This folder will contain the objects used to model the output.

The output will be in two forms:

The Evaluation, which will contain the Mean Absolute Error and the Root Mean Squared Error and
The Forecast, which will contain Lower Estimate, UpperEstimate, Forecast...

Create the Forecast output object by right clicking the Model folder and select Add -> class then name it ForecastOutput.cs. The code for it is:

 public class ForecastOutput
    {
        public string Date { get; set; }

        public float ActualRentals { get; set; }

        public float LowerEstimate { get; set; }

        public float Forecast { get; set; }

        public float UpperEstimate { get; set; }
    }

Next, right-click the Model folder and select Add -> class then name it EvaluateOutput.cs. The code for it is:

 public class EvaluateOutput
    {
        public float MeanAbsoluteError { get; set; }

        public double RootMeanSquaredError { get; set; }
    }

We also need to add the class that will contain the logic. Right-click the project Forecasting_BikeSharingDemandLib and select Add -> class then name it BikeForcast.cs.

The Logic

The entire code for the file BikeForcast.cs is (I will break it down bit by bit):

GetConnectionString(), lines 15 to 22, is used to get the connection string where the data is stored and also the Model path. MLModel.zip can be found in the root folder of project BlazorWithML.NET.Api.

Lines 144 to 151 define the ModelInput class. This class is used to hold the database objects and it contains the following columns:

RentalDate: The date of the observation.
Year: The encoded year of the observation (0=2011, 1=2012).
TotalRentals: The total number of bike rentals for that day.

Lines 153 to 161 define the ModelOutput class. This class contains the following columns:

ForecastedRentals: The predicted values for the forecasted period.
LowerBoundRentals: The predicted minimum values for the forecasted period.
UpperBoundRentals: The predicted maximum values for the forecasted period.

Line 24 defines the GetBikeForcast() method that takes in a parameter numberOfDaysToPredict : int. The parameter will be gotten from the UI. The user will have to input the amount of days (between 1 and 500) he/she wants to predict for. This method holds the logic to forecast the demands with the help of two other methods for Evaluate and Forecast.

Line 26 defines the MLContext. The MLContext class is a starting point for all ML.NET operations, and initializing mlContext creates a new ML.NET environment that can be shared across the model creation workflow objects. It's similar, conceptually, to DBContext in Entity Framework.

Next we need to load the data from the database.
Line 29 creates a DatabaseLoader that loads records of type ModelInput. Line 32 defines the query to load the data from the database.

ML.NET algorithms expect data to be of type Single. Therefore, numerical values coming from the database that are not of type Real, a single-precision floating-point value, have to be converted to Real.

The Year and TotalRental columns are both integer types in the database. Using the CAST built-in function, they are both cast to Real.
We will create a DatabaseSource to connect to the database and execute the query on line 35, after which we then load the data into an IDataView on line 40.

The dataset contains two years' worth of data. Only data from the first year is used for training; the second year's data is held out to compare the actual values against the forecast produced by the model. Filter the data (line 43 and 44) using the FilterRowsByColumn transform.
For the first year, only the values in the Year column less than 1 are selected by setting the upperBound parameter to 1. Conversely, for the second year, values greater than or equal to 1 are selected by setting the lowerBound parameter to 1.

Next, we will define a pipeline that uses the SsaForecastingEstimator to forecast values in a time-series dataset, as shown on lines 47 to 56.

The forecastingPipeline takes 365 data points for the first year and samples or splits the time-series dataset into 30-day (monthly) intervals as specified by the seriesLength parameter. Each of these samples is analyzed through a weekly or 7-day window. When determining what the forecasted value for the next period(s) is(are), the values from the previous seven days are used to make a prediction.

The model is set to forecast a period of the number of day(s) the User input into the future as defined by the horizon parameter. Because a forecast is an informed guess, it's not always 100% accurate. Therefore, it's good to know the range of values in the best and worst-case scenarios as defined by the upper and lower bounds. In this case, the level of confidence for the lower and upper bounds is set to 95%. The confidence level can be increased or decreased accordingly. The higher the value, the wider the range is between the upper and lower bounds to achieve the desired level of confidence.
On line 59, we use the Fit method to train the model and fit the data to the previously defined forecastingPipeline.

Next, we will evaluate the model, and lines 74 to 104 defines an helper method that will be used to evaluate the model.
To evaluate performance, the following metrics are used:

Mean Absolute Error: Measures how close predictions are to the actual value. This value ranges between 0 and infinity. The closer to 0, the better the quality of the model.
Root Mean Squared Error: Summarizes the error in the model. This value ranges between 0 and infinity. The closer to 0, the better the quality of the model.

We evaluate how well the model performs by forecasting next year's data and comparing it against the actual values.
Inside the Evaluate method, forecast the second year's data by using the Transform method with the trained model on line 77.
We get the actual values from the data by using the CreateEnumerable method on line 80, and the forecasted values by using the CreateEnumerable method on line 85.
We then calculate the difference between the actual and forecasted values, commonly referred to as the error, on line 90.
We measure performance by computing the Mean Absolute Error and Root Mean Squared Error values on lines 93 and 94.

Next, we need to save the evaluated model. The model is saved in a file called MLModel.zip as specified by the previously defined modelPath variable from the GetConnectionString() method. Use the Checkpoint method to save the model on line 66.

Finally, we will use the model to forecast demand. Lines 106 to 140 define the Forecast helper method.
We create a List to hold the result of the forecast on line 108, then we use the Predict method on line 111 to forecast rentals for the number of day(s) entered by the user.
Then we align the actual and forecasted values on lines 113 to 131.

With this, we are done with the logic.

The Api

The Blazor WebAssembly app will need an Api to get the forecast from the service. First, we need to enable CORS on the Api. Go to the Startup.cs file of project BlazorWithML.NET.Api. Create a new field:

readonly string AllowedOrigin = "allowedOrigin";

Next, add the CORS service to your app by adding these lines of code to the ConfigureServices(IServiceCollection services) method:

services.AddCors(option => {
            option.AddPolicy("allowedOrigin",
                 builder => builder.AllowAnyOrigin().AllowAnyMethod().AllowAnyHeader()
                    );
            });

Finally, add the CORS middleware to the Configure(IApplicationBuilder app, IWebHostEnvironment env) method:

app.UseCors(AllowedOrigin);

Ensure that this middleware is added as the first middleware in the pipeline that is right after the:

if (env.IsDevelopment())
       {
              app.UseDeveloperExceptionPage()
       }

The Controller

Create an empty Api controller in the Controllers folder, and name it BikeDemandForcastController.cs. The code for this controller is:

The first endpoint, lines 11 to 17, (/GetEvaluateOutput/{numberOfDaysToPredict}) is used to get the EvaluateOutput.
The second endpoint, lines 20 to 26, (/GetForecastOutput/{numberOfDaysToPredict}) is used to get the ForecastOutput.

The UI

First, we are going to set the base Uri of the Api endpoints. This can be done in the Program.cs of project BlazorWithML.NET by replacing:

builder.Services.AddScoped(sp => new HttpClient { BaseAddress = new Uri(builder.HostEnvironment.BaseAddress) });

with

builder.Services.AddScoped(sp => new HttpClient { BaseAddress = new Uri("https://localhost:5001/api/") });

Model Objects

The Api returns data in JSON format so we need to map the response data to an object which would be displayed to the user. Create a new folder called Model and create a class called ForecastOutput.cs in the Model folder. The code for this class is:

Create another class called EvaluateOutput.cs:

We will also need an object to store the input from the user. In the Model folder create a class called BikeForcastInput.cs:

Blazor Page

Blazor pages end with .razor and they are called Razor component.
In the Pages folder, create a Razor Component and name it BikeForcast.razor This component will be used to get the input(number of day(s) from the user) and display the Evaluate and Forecast output. The code is:

Line 1 defines the Uri the BikeForcast.razor will be mapped to. Lines 11 to 70 define the components that will be displayed to the user. The GetBikeDemandForcast() (lines 78 to 82) is used to call the endpoints to get the Evaluate and Forecast.

Navigation

Replace the code in the NavMenu.razor component in the Shared folder with:

With this, we are done with coding. Now it's time to test our application.

Testing the application

First, we need to set the UI (BlazorWithML.NET) and Api (BlazorWithML.NET.Api) to start up at the same time when we run the application.
Right-click the solution then click Set Startup Projects. Select Multiple startup projects and set the Action of BlazorWithML.NET and BlazorWithML.NET.Api to Start. You should get the image below:

Click Apply then Ok. Right-click the solution and click Build Solution. After Build is successful, click CTRL + f5 to run the application. If you are getting an IIS error restart your VS Build the project again, and click CTRL + f5. Once the application is running you, should get the image below:

Click the Bike Demand Forcast link, enter a number between 1 and 500, and you should get the result of the Forecast in a table as shown below:

Do More Blazor or ML.NET!!

If you enjoyed this article and feel like learning more about Blazor WebAssembly or ML.NET, the links below might be just what you need:

Awesome! Please share it with anyone you think could use this information. Thanks for reading. As always if you have any questions, comments, or concerns about this post feel free to leave a comment below.

Develop a Simple CRUD App with ASP.NET Core Blazor WebAssembly and CockroachDB

John Ojo — Tue, 13 Oct 2020 15:09:52 +0000

In this post, I will walk you through how to get started with ASP.NET Core Blazor WebAssembly app by developing an app that performs basic CRUD. We will be making use of CockroachDB as the database, Entity Framework Core as the ORM.

The basic CRUD app will be a bank account with just name and balance where we can Create, Read, Update, and Delete accounts.

The source code on GitHub : BlazorAppWithCockroachDB

Prerequisites

Introduction

Blazor is an open source and cross-platform web UI framework for building single-page apps using .NET and C# instead of JavaScript. Blazor is based on a powerful and flexible component model for building rich interactive web UI. You implement Blazor UI components using a combination of .NET code and Razor syntax: an elegant melding of HTML and C#.

Cockroach Labs is the company behind CockroachDB, the cloud-native, distributed SQL database that provides next-level consistency, ultra-resilience, data locality, and massive scale to modern cloud applications.

Create the Solution and Projects

We will be making use of three projects:

BlazorAppWithCockroachDB- A Blazor WebAssembly project. After opening VS, click the Create a new project link. Next type Blazor in the search box and choose the first option that comes up (Blazor App):

Give the project and solution a name e.g. BlazorAppWithCockroachDB then click the Create button.

Next select the Blazor WebAssembly App option and click the Create button this will create the project and solution for you. This link will help you understand a Blazor WebAssembly project structure

BlazorAppWithCockroachDB.Api- A Web API project, which is used to get data across to the Blazor app. Right click the Solution in the Solution Explorer and select Add -> New Project, search for Web Application and select the ASP.NET Core Web Application option:

click Next, give the project a name BlazorAppWithCockroachDB.Api, then click Create, select the API option:

and finally click Create.

CockroachDbLib- A class Library which holds the database objects and data access codes. Right click the Solution in the Solution Explorer and select Add -> New Project, search for Class Library then select the Class Library (.NET Standard) option click Next:

Next give the library a name e.g. CockroachDbLib and click Create.

Install Dependencies

We need to install the Npgsql.EntityFrameworkCore.PostgreSQL package. This package will be used to communicate with the database. To install the package, right click the solution in the solution explorer and select Manage NuGet Packages for Solution. Under the Browse section, search for Npgsql.EntityFrameworkCore.PostgreSQL and click on it, then in the preview panel, select the BlazorAppWithCockroachDB.Api and CockroachDbLib checkbox so it will be installed only into the projects BlazorAppWithCockroachDB.Api and CockroachDbLib and click the Install button:

We need Newtonsoft.Json to Serialize and Deserialize the API response. Search for Newtonsoft.Json under the browse section and click it and in the preview section, select only the BlazorAppWithCockroachDB checkbox and click the Install button:

Finally, we need to add a reference for project CockroachDbLib to BlazorAppWithCockroachDB.Api. To do this, right click the Dependencies and click Add project Reference, select the CockroachDbLib checkbox and click ok:

The Database

After starting a cluster in insecure mode (this is for the Docker version), we need to connect a SQL shell to the cluster in insecure mode. In the SQL Shell create a database (bank) and a table (accounts) with columns id:int, name:string and balance:int. The query is given below:

CREATE DATABASE bank;
CREATE TABLE bank.accounts (id INT PRIMARY KEY, name string, balance int);

Add initial data to the database:

INSERT INTO bank.accounts VALUES(1, 'John', 1000);
INSERT INTO bank.accounts VALUES(2, 'Eric', 750);

Check the data by running:

SELECT * FROM bank.accounts;

Defining the Database Entity

Database objects (tables) are defined as Entities using Entity Framework Core. The database object, in this case, the accounts table will be represented as an entity (a C# class file). The naming of the class doesn't really matter here, when hooking up the class/entity in the DbContext, the name given to the variable/property used to link the entity to the DbContext must match exactly with the name of the table in the database.

Create a folder called Models in the CockroachDbLib project, and in that folder add a new class file called Account.cs. The code is:

Each column is mapped to a property as shown in the figure above. The name and data type of the property must match exactly with that of the column in the database table.

Defining the DbContext

Create a new class file CockroachDbContext.cs also in the Models folder. The code for the file:

Line 14: is used to link the Entity to the Dbcontext hence to the database table accounts: if the property name doesn't match the table name exactly, an error will occur.

The Repository

This is where all the data access code will be written. We will be making use of dependency injection (using interface) when calling the access code.
Create a new folder Repository in the CockroachDbLib project and in this folder add a new Interface file called IAccountRepository.cs The code for the file:

This interface defines all the methods needed to perform the CRUD.

Finally, create the AccountRepository.cs class file in the Repository folder also. This class implements the IAccountRepository.cs hence providing implementations to the methods defined in the interface:

With this, we are done with the CockroachDbLib.cs project

Creating the API Endpoints

Adding Configuration

The BlazorAppWithCockroachDB.Api project will be used to create all the required endpoints to get and store data.

We need to add some configurations.

First we need to enable CORS on the API which can be done easily in the Startup.cs file. Create a new field in the Startup.cs file:

readonly string AllowedOrigin = "allowedOrigin";

Next, add the CORS service to your app by adding these lines of code to the ConfigureServices(IServiceCollection services) method:

 services.AddCors(option => {
            option.AddPolicy("allowedOrigin",
                 builder => builder.AllowAnyOrigin().AllowAnyMethod().AllowAnyHeader()
                    );
            });

Finally, add the CORS middleware to the Configure(IApplicationBuilder app, IWebHostEnvironment env) method:

app.UseCors(AllowedOrigin);

Ensure that this middleware is added as the first middleware in the pipeline that is right after the:

if (env.IsDevelopment())
       {
              app.UseDeveloperExceptionPage()
       }

The next is to configure the DbContext which can be done by adding this line of code in the ConfigureServices(IServiceCollection services) method of the Startup.cs file:

services.AddDbContext<CockroachDbContext>(options =>
                options.UseNpgsql(Configuration.GetConnectionString("CockroachDb")));

This line requires the ConnectionString which will be added to the appsettings.json file as:

"ConnectionStrings": {
    "CockroachDb": "Host=localhost;Database=bank;Username=John;Port=26257"
  }

The Username should be the user you created and assigned to the database from the SQL shell or omitted if you didn't add any user. The Port must be the port number you used to start up the local cluster for CockroachDb.

The last step is to inject the IAccountRepository dependency which can be done by adding:

 services.AddTransient<IAccountRepository, AccountRepository>();

in the ConfigureServices(IServiceCollection services) method of the Startup.cs file.
Ensure to import all the required namespace in case you are getting an error with the added lines.

Defining the Endpoints

Create a new API Controller in the Controllers folder of project BlazorAppWithCockroachDB.Api and name it AccountController.cs. This controller is going to contain all the endpoints we need. The code for the AccountController.cs file:

The list of endpoints we have from the AccountController.cs are:

https://localhost:44368/api/Account/GetAll - to return a List containing all the Accounts in the bank
https://localhost:44368/api/Account/Get/{id} - to get an Account by the id.
https://localhost:44368/api/Account/GetLastAccountId - to get the id of the last Account.
https://localhost:44368/api/Account/Create - to create a new Account
https://localhost:44368/api/Account/Update - to update the details of an account.
https://localhost:44368/api/Account/Delete/{id} - to delete an account.

The 44368 will be replaced with the port VS uses for yours, this can be gotten from the launchSettings.json file in the Properties folder.

Now that the API is ready it's time to move to the last part which is the Blazor App (project BlazorAppWithCockroachDB).

Blazor App

The first thing to do is to set the base Uri of the API endpoints. This can be done in the Program.cs file by replacing

builder.Services.AddScoped(sp => new HttpClient { BaseAddress = new Uri(builder.HostEnvironment.BaseAddress) });

with

 builder.Services.AddScoped(sp => new HttpClient { BaseAddress = new Uri("https://localhost:44368/api/") });

Model Objects

The API returns data in JSON format so we would need to map the response data to an object which would be displayed to the user. Create a new folder called Model.
Create a class called Account.cs; this will be similar to the Account.cs in the CockroachDbLib.Models namespace. The code for the new Account.cs is:

The second Model object will be used when the user wants to create or edit an account. Create another class file AccountModel.cs in the Model folder. The code for this file is:

Blazor Pages

Blazor pages ends with .razor and they are called Razor component.
Create a new folder called Bank and in this folder create a Razor Component and name it Accounts.razor This component will be used to display all the Accounts and also contain other links to other components. The code is:

The accounts are displayed in a tabular form - line 17 to 35. The DeleteAccount function - line 50 to 75 defines the process called when the Delete button is clicked against an account - line 31.
Line 47 calls the /GetaAll endpoint and line 55 calls the /Delete/{id} passing the id of the account selected.

Next create another Razor component CreateAccount.razor in the Bank folder. This component will be used to add a new account. The code for this component:

Line 39 to 64 contains the code to add a new account. Notice I got the last account's id, added 1 to it and used it as the new id of the new account; you might not want to do this in a real application. A better alternative is to create the id as a Guid type and create a new Guid for every account.

Next, we need the component to edit an account. Add a new Razor component called EditAccount.razor. The code for this is:

Finally, we need a component to view the details of an account. Add a Razor component called SingleAccount.razor. The code is:

Navigation

Replace the code in the NavMenu.razor component in the Shared folder with:

Testing the application

At this point, we are done with coding. Save all the changes made to the files, build project, make sure there is no error in any file.
Run the BlazorAppWithCockroachDB.Api project first and test one of the endpoints using your browser or Postman, then run the BlazorAppWithCockroachDB project. You should get the:

If you click the All Account nav link you should get all the accounts we created earlier:

Try creating, editing or deleting an account to make sure all the functions are working fine.

Awesome! Please share it with anyone you think could use this information. Thanks for reading.

Below are some links to keep you going if you are really interested in Blazor WebAssembly or CockroachDb: