DEV Community: thamer_saraei

First Deep Learning Model : Dense Layer

thamer_saraei — Wed, 14 Apr 2021 16:20:46 +0000

The Basics: Training Your First Model ✨🎨

Welcome to this Tuto where you will train your first Machine Learning model. We will try to keep things simpler here, and we will only provide basic concepts.

The problem we will solve is to convert from Celsius to Fahrenheit, where the approximate formula is:

$$ F = C\times 1.8 + 32 $$

Notice: it would be simple enough to create a simple Python function that directly performs this calculation (Traditional Software Development) :

def FtoC(C):
    F = C*1.8 + 32
  return F

, but that wouldn't be machine learning. The main goal of this Notebook is show the main difference between the tow approachs : ML & Traditional Software Development ( you can find more in this link)

Let's start : So for build our Ml's model, we will give TensorFlow some sample Celsius values (0, 8, 15, 22, 38) ( called Input Data) and their corresponding Fahrenheit values (32, 46, 59, 72, 100) (Called Output Data).

Then, we will train ( with the Training Dataset ) a model that figures out the above formula through the training process.

Import Packages

First and to keep things so simple, we import TensorFlow as tf for ease of use.

Next, import NumPy as np : Numpy helps us to represent our data as highly performant lists.

from __future__ import absolute_import, division, print_function
import tensorflow as tf
tf.logging.set_verbosity(tf.logging.ERROR)

import numpy as np

Set up training data

As we knew, supervised machine learning essentially consists of looking for a performance algorithm from a set of inputs and outputs.

As the objective of this Tuto is to create a model that can convert temperature in degrees Fahrenhet to degrees Celsius, we should create two lists celsius_q and fahrenheit_a that we can use to build our model.

celsius_q    = np.array([-40, -10,  0,  8, 15, 22,  38],  dtype=float)
fahrenheit_a = np.array([-40,  14, 32, 46, 59, 72, 100],  dtype=float)

for i,c in enumerate(celsius_q):
    print("{} degrees Celsius = {} degrees Fahrenhet".format(c, fahrenheit_a[i]))

Some IMPORTANT Machine Learning terminology

Feature : The inputs to our model. In our case, a single value : the degrees in Celsius.
Labels : The output of our model predicts. In our case, a single value : the degrees in Fahrenhet.
Example : A pair of inputs/outputs used during training. In our case a pair of values from celsius_q and fahrenhet_a to a particular pointer, such as(38,100).

Create the model

Now we will create the model. We will use the simplest model called Dense network : This kind of model will require only a single layer, with a single neuron ( Since the problem is so simple )

Build a layer : `l0`

We'll call the layer l0 and create it by this function tf.keras.layers.Dense with the following configuration:

input_shape=[1] : This specifies that the entry in this layer is a single value. That is, the shape is a one-dimensional array with a member. Since this is the first (and only) layer, this input form is the input form of the entire model. The unique value is a floating point number representing degrees Celsius.
units=1 : This determines the number of neurons in the class. The number of neurons determines how many internal variables the class should attempt to learn how to solve the problem (later). Since this is the last layer, it is also the output size of the model: a single float value that represents a degree of Fahrenheit.

l0 = tf.keras.layers.Dense(units=1, input_shape=[1])

Assemble layers into the model

After defined our layers, we need to group these layers to create the model. The **Sequential model ** definition takes a list of layers as argument, specifying the calculation order from the input to the output.

This model has just a single layer, l0

model = tf.keras.Sequential([l0])

Note

We can define our layers inside the model definition as shown below :

model = tf.keras.Sequential([
  tf.keras.layers.Dense(units=1, input_shape=[1])
])

Compile the model, with loss and optimizer functions

After defining and before training, the model has to be compiled.

Once compiled for the training, the model is given:

Loss function : A way to measure the distance between forecasts and the desired result. (The measured difference is called "loss").
Optimizer function : A way of adjusting internal values in order to minimize the loss.

model.compile(loss='mean_squared_error',
              optimizer=tf.keras.optimizers.Adam(0.1))

These are used during training (model.fit()) to first calculate the loss at each point, and then improve it.

During training, the optimizer function is used to calculate adjustments to the model's internal variables. The goal is to adjust the internal variables until the model (which is really a math function) mirrors the actual equation for converting Celsius to Fahrenheit.

What is useful to know about these parameters are:

The loss function (mean squared error) and the optimizer (Adam) used here are standard for simple models like this one, but many others are available. It is not important to know how these specific functions work at this point.

Note Very Important : One part of the Optimizer you may need to think about when building your own models is the learnign rate (0.1 in the code above). This is the step size taken when adjusting values in the model. If the value is too small, it will take too many iterations to train the model. Too large, and accuracy goes down. Finding a good value often involves some trial and error, but the range is usually within 0.001 (default), and 0.1

Train the model

Train the model by calling the fit method.

During training, the model takes the Input Data : values in degrees Celsius, performs a calculation using the current internal variables (called "weights"), and generates values that are supposed to be the equivalent in Fahrenheit.

Since the weights are initially randomly defined, the output will not be close to the correct value. The difference between the actual output and the desired output is calculated using the loss function (mean squared error), and the optimization function indicates how the weights should be adjusted.

This cycle is controlled by calculation, comparison and modification in a fit method. The first argument is the input data, and the second argument is the desired output. The epochs argument specifies the number of times this session should be run, and the verbose modulus controls the amount of output produced by the method.

history = model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)
print("Finished training the model")

Display training statistics

The fit method returns a history object. We can use this object to plot how the loss of our model goes down after each training epoch.

P.S : A high loss means that the Fahrenheit degrees the model predicts is far from the corresponding value in fahrenheit_a.

We'll use Matplotlib to visualize this .

As we can see, our model improves very quickly at the beginning, then progresses slowly and gradually until it is almost "perfect" towards the end.

import matplotlib.pyplot as plt
plt.xlabel('Epoch Number')
plt.ylabel("Loss Magnitude")
plt.plot(history.history['loss'])

Use the model to predict values

Now we have a model that has been trained to detect the relationshop between celsius_q and fahrenheit_a.

So we can use the prediction method to make it calculate degrees Fahrenheit to previously unknown degrees.

So, for example, if the Celsius value is 100, what do you think the Fahrenheit result will be? (Take a guess before you run this following code ).

print(model.predict([100.0]))

The correct answer is $$100 \times 1.8 + 32 = 212$$. So our model is doing really well.

To review

We created a model with a Dense layer (Only One Layer )
We trained it with 3500 (7*500) examples (with : 7 pairs, over 500 epochs).

Our model modified the variables (weight) of the dense layer until it was able to return the correct Fahrenheit value to any Celius value. (Remember that 100 ° C was not part of our training data, it can be called a Test dataset ).

Looking at the layer weights

Finally, let's print the internal variables of the Dense layer using the get_weights() method.

print("These are the layer variables: {}".format(l0.get_weights()))

The first variable is close to 1.8 and the second to 32. These values (1.8 and 32) are the actual variables in the real conversion formula.

Since the form is the same, the variables should converge on the standard values of 1.8 and 32, which is exactly what happened.

With additional neurons, additional inputs, and additional outputs, the formula becomes much more complex, but the idea is the same.

Just for fun, what if we created more Dense layers with different units, which therefore also has more variables? (Show below )

l0 = tf.keras.layers.Dense(units=4, input_shape=[1])  
l1 = tf.keras.layers.Dense(units=4)  
l2 = tf.keras.layers.Dense(units=1)  
model = tf.keras.Sequential([l0, l1, l2])
model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1))
model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)
print("Finished training the model")
print(model.predict([100.0]))
print("Model predicts that 100 degrees Celsius is: {} degrees Fahrenheit".format(model.predict([100.0])))
print("These are the l0 variables: {}".format(l0.get_weights()))
print("These are the l1 variables: {}".format(l1.get_weights()))
print("These are the l2 variables: {}".format(l2.get_weights()))

As we can see, this model is also able to predict the corresponding Fahrenheit value really well. But when you look at the variables (weights) in the l0 and l1 layers, they are nothing even close to ~1.8 and ~32. The added complexity hides the "simple" form of the conversion equation.

Thanks for your attention

I wish that you enjoyed this Notebook 👏✌.

Join me

Python 3.10 : What's the new ?

thamer_saraei — Sun, 11 Apr 2021 03:11:22 +0000

Python 3.10 : What's the new ?

The release of ✨Python 3.10✨ is getting closer, so it's time to take a ride with the new version of Python and see what awesome new features will come with this new release👌 😍.

Please join me in ghithub : https://github.com/th-rpy/python_3.10_all_new_features

Install Python 3.10 Alpha version

To try these new features, we will have to install the Alpha/Beta version of Python 3.10. Remember that this last version is not yet stable.

If you are under Linux (Ubuntu), you just have to follow the steps below :

 # Download the latest version for Linux
 wget https://www.python.org/ftp/python/3.10.0/Python-3.10.0a6.tgz
 # Unpack Python source code
 tar xzvf Python-3.10.0a6.tgz
cd Python-3.10.0a6
# Compile Python source with static libraries
./configure --prefix=$HOME/python-3.10.0a6
make
make install
$HOME/python-3.10.0a6/bin/python3.10

If you are under Windows, you just have to Download Python Executable Installer from here, then you need to Run Executable Installer.
If you are on MacOs, I can't help you. I am not rich enough to buy a Mac!!! 😒, but this link may help you.

Yeeeep, Python 3.10 is finally installed ✌ , now we can take a look at all the new features . Let's start 😉😎.

New Features

The new version of python has arrived with many features. In this article, I will show you the most important of them. So, So buckle up, the adventure with 🐍Python will begin in a few seconds 🚀🕓.

New Type Union Operator

Instead of using typing.union to express the syntax "either type X or type Y", the new version of python introduces the new union operator of type X | Y. This new operator allows us to code more cleanly and efficiently.

Old Version

from typing import Union
def square(number: Union[int, float]) -> Union[int, float]:
    return number ** 2
isinstance('3', int | str)

New Version

def square(number: int | float) -> int | float:
    return number ** 2
isinstance('3', int | str)

This features was contributed by Ken Jin. Visit this link (PEP 612) for more details.

TypeAlias Annotation

The TypeAlias annotation concept was first introduced in PEP 484 (Python-Version: 3.5) . A reimplementation of this concept will be presented in PEP 613 (Python-Version: 3.10). The main reason for this reimplementation is that the old concept is very difficult for type checkers to distinguish between type aliases and ordinary assignments. See the following example:

Old Version

StrCache = 'Cache[str]'  # a type alias
LOG_PREFIX = 'LOG[DEBUG]'  # a module constant

New Version

StrCache: TypeAlias = 'Cache[str]'  # a type alias
LOG_PREFIX = 'LOG[DEBUG]'  # a module constant

This features was contributed by Mikhail Golubev. Visit this link (PEP 613) for more details.

Better error messages in the parser

Suppose you want to write a code that manipulates for example a dictionary (or tuple , list or set ) and you forget to close the brackets (or the parentheses). If you are working with python 3, when you execute your code, the interpreter will display a syntax error like this one "SyntaxError : unexpected EOF".
However, with this new version, when you try to parse code that contains unclosed parentheses or brackets, the interpreter will displays a more informative error with the location of the unclosed parenthesis or brackets.

Old Version

File "example.py", line 3
some_other_code = foo()
                ^
SyntaxError: invalid syntax

New Version

File "example.py", line 1
expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
           ^
SyntaxError: '{' was never closed

This features was contributed by Pablo Galindo and Batuhan Taskaya.

Structural Pattern Matching

We can say that the most important feature will be introduced in this new Python 3.
Pattern matching will be presented in the common form: match statement and case statements of patterns with associated actions. Patterns can be: sequences, mappings, primitive data types as well as class instances. By using pattern matching, we are able to, for example, extract information from complex data types, plug into the data structure, and apply specific actions based on different data forms. This is not just the switch/case syntax we all know from other programming languages, but it also adds powerful functionality that we should explore.

Example 1: Simple pattern: match to a literal

def func(x):
    match x:
        case "x1":
            return "x1 .."
        case "x2":
            return "x2"
        case "x3" | "x4":  # Multiple literals can be combined with `|`
            return "Yay, "
        case _:
            return "Just another x..."

Example 2: Patterns with a literal and variable

def func(X):  # X = (x, y, z)
    # point is an (x, y) tuple
    match point:
        case (0, 0):
            print("Origin")
        case (0, y):
            print(f"Y={y}")
        case (x, 0):
            print(f"X={x}")
        case (x, y):
            print(f"X={x}, Y={y}")
        case _:
            raise ValueError("Not a point")

Example 3: Patterns and classes

class Point:
    x: int
    y: int

    def location(point):
        match point:
            case Point(x=0, y=0):
                print("Origin is the point's location.")
            case Point(x=0, y=y):
                print(f"Y={y} and the point is on the y-axis.")
            case Point(x=x, y=0):
                print(f"X={x} and the point is on the x-axis.")
            case Point():
                print("The point is located somewhere else on the plane.")
            case _:
                print("Not a point")

Example 4: Guard
We can add an if clause to a pattern, called a guard. If the guard is false, match moves on to try the next case block. Note that the value capture takes place before the guard is evaluated:

match point:
    case Point(x, y) if x == y:
        print(f"The point is located on the diagonal Y=X at {x}.")
    case Point(x, y):
        print(f"Point is not on the diagonal.")

Example 5: Nested Patterns
Patterns can be nested in arbitrary ways. For example, if our data is a short list of points, they could be matched in the following way:

match points:
    case []:
        print("No points in the list.")
    case [Point(0, 0)]:
        print("The origin is the only point in the list.")
    case [Point(x, y)]:
        print(f"A single point {x}, {y} is in the list.")
    case [Point(0, y1), Point(0, y2)]:
        print(f"Two points on the Y axis at {y1}, {y2} are in the list.")
    case _:
        print("Something else is found in the list.")

If you want to see more examples and a full tutorial, check out PEP 636.

Conclusion

Python 3.10 brings many new interesting features, but as it is an alpha version (not yet stable), it is still far from being fully tested and ready for production. So it is not recommended to start using it right away.

DEV Community: thamer_saraei

First Deep Learning Model : Dense Layer

The Basics: Training Your First Model ✨🎨

Import Packages

Set up training data

Some IMPORTANT Machine Learning terminology

Create the model

Build a layer : l0

Assemble layers into the model

Compile the model, with loss and optimizer functions

Train the model

Display training statistics

Use the model to predict values

To review

Looking at the layer weights

Python 3.10 : What's the new ?

Python 3.10 : What's the new ?

Install Python 3.10 Alpha version

New Features

New Type Union Operator

TypeAlias Annotation

Better error messages in the parser

Structural Pattern Matching

Conclusion

Build a layer : `l0`