DEV Community: SanExperts

Artificial Neural Networks: types, uses, and how they work

ABuftea — Thu, 28 Jan 2021 10:52:22 +0000

Hi all,

This is the second post of the series Deep Learning for Dummies.

Below you have the lists of posts that I plan to publish under this series. I will keep it updated with each published post. Unpublished posts titles and used libraries can change:

Introduction to Deep Learning, basic ANN principles
Artificial Neural Networks: types, uses, and how they work
CNN Text Classification using Tensorflow (February - March)
RNN Stock Price Prediction using (Tensorflow or PyTorch) (April - May)
Who knows? I may extend it. Perhaps some SNN use case.

In this post you'll learn about the different artificial neural networks, how do they work, and what is each type most suited for. This is still going to be a fully theoretical post with no code involved. No worries, next posts are going to be fully practical. We'll apply all the concepts learned in the introduction post and use one of the neural networks in this post to solve real-life use cases.

Let's go with the table of contents that we'll cover:

Introductory Concepts
Artificial Neural Networks
Convolutional Neural Networks
Recurrent Neural Networks
Spiky Neural Networks
Final Thoughts

Introductory Concepts

The most complete topologies guide for the different types of artificial neural networks used in deep learning can be found here. I encourage you to have a look since you can also see a short description of what each one is suitable for.

In deep learning, we classify artificial neural networks based on the way they operate. Then, as we'll see through this post, all the artificial neural networks used currently in production fall under four main categories, artificial neural networks (ANNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), and spiking neural networks (SNNs).

Artificial Neural Networks

It may seem confusing since all the neural networks used in computing are artificial neural networks, however, we reserve this definition for all the neural networks that are distributed in several layers of artificial neurons and only performs the operations of those neurons to obtain the output.

The neural network used as an example in my previous post where I explained how to do the house prices estimations, was an ANN. The typical structure of those neural networks can be seen in figure 1.

Figure 1 (Artificial Neural Network)

As you can see. We have the inputs to the neural network, which is always formed by a set of numbers, we have the input layer, the hidden layers, and the output layer. That is the structure of an artificial neural network "ANN". In my previous post I explained how this works, so let's switch to the next type, the CNN.

Convolutional Neural Networks

The name of this type of artificial neural network comes from the two operations that they perform over the inputs before feeding them to one usual ANN. Those two are the convolution and pooling operations. Before zooming into what are them, see a graphical representation of a CNN in figure 2.

Figure 2 (Convolutional Neural Network)

Most usually CNNs are used for image processing, that is why the input is the figure is an image. As we can see, the first and second layer is a convolutional layer and a pooling layer respectively. Those came hand in hand with each other, which means that every time we insert a convolutional layer we have to insert a pooling layer afterward. We could insert as many convolution-pooling layers as we want, of course in reality it will depend on our application and the size of the input. The output of the last convolution-pooling operations will be a matrix of numbers. This matrix is converted into one vector which is called the flatten vector, thus sometimes you will hear the name flattened layer. The flattened vector is afterward inputted to a fully connected layer which is just a layer of ANN neurons as we saw in the previous post. Lastly, we have the output layer which provides the prediction. Now let's see what the convolution and pooling operations mean.

Convolution operates on two signals (in 1 dimension) or in two images (in 2 dimensions). We could think of the first signal as the input and the second signal, called the kernel, as the filter. Convolution takes the input signal and multiplies it with the kernel, outputting a third modified signal. The objective of this operation is to extract high-level features from the input.
There are two types of results. Where the output is reduced in dimension (this is called Valid Padding) and where the output has the same dimension or even bigger (this is called Same Padding).

Figure 3 shows an illustration of the convolution operation. An image is nothing else than a matrix full of pixels, the convolution operation consists of passing a filter (the kernel matrix) through the image matrix, this means that the kernel matrix slides step by step, one position in every direction, over the image matrix. It multiplies each image matrix cell value by the cell value in the kernel matrix, sums all the obtained values, and stores the result in the respective cell of the output matrix. The weights of the kernel matrix are initialized by default to some value and are updated through the training process.

Figure 3 (Convolution Operation)

The idea of the convolution operation is to mimic the human eye detecting features in one image. When we see a car image, there are some features in the image that makes us recognize there is a car (shape of the car, material, roads, intersections, etc). Multiplying the image matrix by a filter and updating the
filter parameters through training extracts the most important features of the image, those that make us understand a car is in the image but we don't know how to detect algorithmically.

Figure 4 shows an illustration of the pooling operation. It is a sample-based discretization process where we down-sample an input reducing its dimensions and allowing for assumptions to be made about features contained in the selected sub-regions. This is done to decrease the computational power required to process data while extracting dominant features. There are two main types of pooling operations. Max pooling of which we can see a graphical representation in figure 4 and average pooling. As their name suggests, one of them is based on picking the maximum value from the selected region while the other is based on calculating the average of all the values.

Figure 4 (Pooling Operation)

The output of the last pooling layer in a CNN is a multidimensional matrix named pooled features map. We need to flatten this final output into a vector containing one value per index to be able to feed it to the fully connected layer, this is the flattened vector represented in figure 5. The features filtered through the convolution-pooling steps are encoded in the flatten vector. The role of the fully connected layer is to take all the vector values and combine features in order to make a prediction about the probability of each class.

Figure 5 (Flattening the pooling layer output)

Just as a side note, when working with color images there are three matrixes, one per color (Red Green Blue "RGB"). Then, we will have to work with three matrixes separately. In addition, one filter (or kernel) may not be the most adequate to extract all the features from the image. This is, in one portion of the image a filter with a specific length and parameters is most adequate while in other portions of the image another dimension and parameters may be better to properly extract the features. Thus, when applying the kernels over one image we will apply several kernels at once and each with different dimensions and different initial parameters. We'll see more about this in the next post, where we will use a CNN to predict text sentiments.

Obtaining the correct number of kernels and their size, as well as the correct pooling operation can be achieved only through practice. Another important factor is that we must be coherent with the images dataset that we are feeding the CNN. If we train a CNN to differentiate between terrestrial vehicles, but not planes, then don't introduce a plane image in your training set.

Why? Because the final output of a CNN represents the probability of inputs belonging to a specific class. When training we'll feed the CNN with photos of trucks, tractors, vans, and cars. Each of these vehicles represents a class and we'll correct the CNN each time it predicts that a vehicle is in a class it does not correspond. Sames as ANNs, through backpropagation, fully connected layer weights and parameters in the kernel matrix are updated, so the CNN learns how to predict which class is each vehicle. If you have a plane image in your training set and there is no plane class at the output, all you're doing is to worsen the parameter updating process. The CNN will associate features of a plane with another vehicle class.

Since CNN outputs are probabilities of the input to belong to each class, the sum of all outputs will always range between 0 and 1. If you use the trained CNN to differentiate between vehicles and you input an airplane image, in the best case scenario the four classes output may be 0.25 (meaning that the input hast the same probability of belonging to any class). If this is the case then you can detect that the input belongs to none of the four classes. But it is also possible that the CNN outputs a higher probability for one of the classes, so you may wrongly consider that there is a truck where in fact is an airplane.

Recurrent Neural Networks

This type of neural networks are ANNs designed in an algorithmic way that gives the network the ability to perform time analysis predictions. This means that we can work with inputs where the sequence of the input matters. For example language translation, text generation, or stock price prediction.

The order of the words in a phrase does matter, it is not the same to say "I have to read this book" than "I have this book to read". RNNs are used by the popular google translator, RNNs suggest you the next word when typing an email and they even suggest you a new easy reading and grammatically correct structure for the entire sentence, if you ever used Grammarly. So let's see how it all works.

Figure 6 represents a Recurrent Neural Netowrk. In order to process time series inputs and predict "ht", apart from the actual time input "Xt", RNNs use a new variable, called hidden variable or hidden state, whose value depends on the previous prediction when the input was "Xt-1".

Figure 6 (Recurrent Neural Networks)

The rectangular block "A" represents a usual ANN, with its input layer, hidden layers, and output layer. The loop arrow over block "A" represents the usage of information from previous predictions at predicting the current output. The right side of the equal symbol is a breakdown of the time series prediction process. Let's suppose we want to predict the stock price for the next week (Monday to Sunday) based on the sales perspectives announced by the company for that week. X0 to X6 are the inputs representing the expected sales at day one, day two, and so forth till day seven. Similarly, h0 to h6 represent the stock price predicted for day one to day seven respectively.

First, the ANN takes as input X0 and predicts the stock price for Monday "h0", during this prediction the value of the hidden state or hidden variable (both terminologies are used) is updated. This variable, together with Tuesday "x1" sales information, is used to predict the share price for Tuesday "h1". Meanwhile predicting the stock price, the hidden state (or hidden variable) is updated and then used in the prediction of "h2". This loop is maintained till the RNN performs the prediction of all seven days of the week.

Note that hidden state refers to a concept totally different from hidden layer. A hidden layer is a layer of artificial neurons enclosed between the input layer and the output layer.

Hidden State or Hidden Variable is just another input to the ANN which is calculated based on the predictions at previous time steps in the sequence. This is the way we can store the sequence information up to step "t-1", it gives the concept of memory to the RNN. The value of the hidden state at time "t", is calculated based on the input at time "t" and the value of the hidden state at time "t-1".

There are two common approaches to defining and updating the hidden states, depending on the problem. If the problem is that of contiguous sequences (for example working with text which is always about the same topic) then the hidden state of the next sequence is the last version of the hidden state in the actual sequence, this is, the hidden state is initialized once randomly and that initial value is being updated during all the training process. The other approach, used when working with distinct sequences (random tweets as an example), consists of initializing the hidden state with the same value each time we start predicting a new sequence.

Hidden states are updated through backward propagation using weights, those weights are what the RNN stores and uses to update the hidden state value at performing predictions in production.

As seen in my previous post, we use the gradient of the cost function to update the weights in a process which is called backward propagation. The backward propagation process is problematic for the RNNs because of the vanishing/exploding gradient problem.

Imposed by the structure of the RNN, information from every previous step is used when updating the weights at any given time step. If we make a mistake at updating the first steps, that mistake is going to be passed to the next steps and it will be worsened because of derivatives properties. If we mistake by updating to a lower value than needed, at each next step the updated value will be even smaller than needed, resulting in a vanishing gradient. In contrast, if we mistake by updating to a bigger value, the value will become bigger than needed each time step in the series, resulting in an exploding gradient. This prevents our RNN from learning. Figure 7 below is a summary of this problem.

Figure 7 (Vanishing / Exploding Gradient Problem)

There are different ways of solving this vanishing/exploding gradient problem, however, for the sake of the length of this post and because it is the most used in practice I am going to shortly explain the long short term memory "LSTM".

Long Short Term Memory consists of giving a specific structure to the ANN present in the RNN. This specific ANN consists of 4 layers of neurons interacting between themselves. Figure 8 is a representation of the LSTM showing us the structure of each ANN, "A", in the figure.

Figure 8 (Long Short Term Memory RNN)

The LSTM adds the new concept of cell states which gives the model longer memory of past events. Its function is to decide what information to carry forward to the next time step. This is achieved thanks to the combination of several steps. Have a look at figure 9 below. There are three entries to the ANN in one step, the input, the hidden state, and the cell state. First, by what is seen in the figure as "ft" (forget function) the RNN decides if to forget the information coming from the previous step or not. After, through what is seen in the picture as "Ct", it computes what information from the actual step should be carried to the next time step and adds that information to the cell state variable. Lastly, it computes the output and the hidden state.

Figure 9 (Cell State in LSTM)

To explain a practical use of this architecture. Let's say that we are translating the phrase "Adrian is being late", then when translating the first word, "Adrian", we store it into the cell state and send that information to the next steps, this way the RNNs can remember the subject of the phrase at translating. This is useful if, for example, in some language the correct structure is to put the subject at the end. So instead of "Adrian is being late" the translation should write "Is being late Adrian".

We'll end the RNNs explanation here. In the 4th post of this series I'll implement a simple stock prediction algorithm using this kind of neural network. We'll go into further detail about practical examples in python then.

This post is being already longer than I expected, nonetheless, I recently discovered a totally new concept of artificial neural networks which really intrigued me, so I would like to give you a brief overview about what are those, the Spiking Neural Networks.

Spiking Neural Networks

SNNs have been developed for neurological computing in an attempt to model the behavior of the biological brain. It seems that they have been around for a while, but have not sparked up deep learning interests mainly due to a lack of training algorithms and increased computational complexity. SNNs mimic more closely the neural connections in our brain, thus they are the most bio-inspired artificial networks created till the present.

Figure 10 is a representation of two biological neurons, our brain is made out of billions of these interconnected to each other. They use the axon terminals to transmit signals while the dendrites are in charge of receiving signals. From an engineering point of view, neurons work by transmitting and receiving electric spikes between them. This is, they produce a voltage increased for a short period of time and then it gets back to its neutral status. Biologists do not agree with this description because it is very primitive, however, it is a practical simplification and it works for computational brain modeling. Here you have a long series of videos from Khan Academy explaining the brain's functioning from a biological perspective. In the next paragraph, we'll briefly cover the concepts we need to later understand how a SNN neuron works.

Figure 10 (Biological Neurons)

Each receptor in our body (eyes, nerves, ears) activates one or several neurons with electrical impulses. The impulse is received by the neuron's receptive field and it needs to surpass a threshold value in order to activate the neuron. When this threshold is activated we say that it fires an action potential. Each neuron in the nervous system has its own receptive field threshold limit, meaning that impulses which activate one neuron may not activate another one.

The action potential travels along the neuron, reaching the axon's terminals, where it causes the release of neurotransmitters. Those neurotransmitters travel and attach to the dendrites of another neuron, which is the receiver. There exist several types of neurotransmitters in our brain, and the effect they have on the neuron depends on their type. Biological neurons are connected with thousands of other neurons, then they can receive different neurotransmitters at the same time. The combination of the received neurotransmitters determines neuron behavior.

When the amount of neurotransmitters received by a neuron surpass a certain threshold value, the equilibrium of the neuron changes and triggers the action potential, which travels to the axon terminals and produces the discharge of neurotransmitters. Each neuron contains a different amount of neurotransmitters in its axons, thus, the more neurotransmitters it has, the more it transmits, the stronger is its influence over the receiving neuron. On the same way, some neurons dentrites are more prone at receiving neurotransmitters than others. Those factors determine the strength of the connection, meaning that a neuron does not have the same influence over all the other neurons it is connected with, and, a receiving neuron is not influenced equally by all the transmitting neurons it is connected to. Just for general knowledge, transmitting neuron is called presynaptic neuron while the receiving neuron is called postsynaptic neuron.

Now that we have the basics about how does our brain transmit signals, we can understand how SNNs works since it is an analogy. In figure 11 you can see that the structure of an SNN is different from those we have used for ANNs, CNNs and RNNs. The layers concept is lost and neurons are multiple-connected between each other forming a mesh. This emulates the way neurons in our brain are connected, being this the reason why SNNs are the correct tool to use at studying biological brain behavior.

I guess you heard about Elon Musk open AI company. Well, they are testing the implementation of chips to read and transmit signals to our neurons. Another important fact, a group of researches have achieved the control of a bio-inspired robot insect "RoboBee" through the usage of an SNN, while the Cortical Labs team in Australia have developed silicon chips combined with mouse neurons. If Yuval Noah Harari in his trilogy (Sapiens, 21 Leesons for 21 Century, Deus) is right and the evolution of homo-sapiens goes towards cyborgs, then it looks like SNNs are part of this transformation.

Figure 11 (SNN mesh used at reading brain's behaviour)

To mimick the way biological brains work, Spiking Neural Networks neurons must be defined in such a way that they can be activated asincornosly and shoot pulses as output. ANNs are not accurate models of the brain because layers are activated synchronously and they used continuous-valued activation functions. Each neuron outputs a continuous value. In our brain, after the action potential happens and the neuron transmits the neurotransmitters, it then passes back to its stable state, waiting for the next activation.

Several techniques can be used to emulate such a behaviour, but one of the most used and easy to understand is to use the Leaky Integrate and Fire (LIF) model to represent neurons.

Under the Leaky Integrate and Fire (LIF) model, neurons are represented as a parallel combination of a resistor (R), a capacitor (C) and a switch. See a graphical representation in figure 12. A current source (Idc) is added to represent the biological neuron activation impulse. First, the current source charges up the capacitor, which produces the potential difference Vc.

Figure 12 (LIF Artifical Neuron Model)

Part b) of the above figure shows the graph which represents the neuron output based on the current input Idc. We can see there is a threshold current input "Icrit" below which the neuron's output voltage "Vth" never reaches the threshold voltage "Vth". If at any given moment the input current "Idc" is bigger than the threshold current "Icrit", then the neuron output reaches the threshold voltage "Vth". Reaching "Vth" triggers the switch discharging the capacitor, so we see an immediate decrease of the voltage "Vdc". This discharge produces a voltage pulse that is called a spike. The spike's shape comes from the natural charging-discharging cycle of the capacitor. Have a look here to understand what and why a capacitor's charging-discharging cycle produces spike looking pulses.

This spike emulates what the biological neuron does when is activated through stimuli, the neuron receives neurotransmitters from thousands of others till its equilibrium is unbalanced (till the threshold "Icrit" is surpassed). This produces the action potential which discharges neurotransmitters over another neuron and it goes back to equilibrium waiting for next activation (the process of transmitting and getting back to equilibrium is represented by the capacitor charching and discharging cycle).

We've commented that there exist different neurotransmitters and their combination change neuron behavior. We emulate the transmission of different neurotranmsitters through the output frequency.

Frequency is defined as the number of waves that pass a fixed point in a unit of time. We can see in the output graph that at a bigger current input "Idc", bigger is the output frequency. Look at the red spikes, green spikes, and blue spikes. The bigger is Idc at the input, the more waves we have at the output Vc. Then based on the input, we can have different outputs. In addition, increasing or decreasing the threshold values for Idc and Vc we emulate different strength of connections. This can be achieved by changing the resistor resistance and capacitor capacity.

Then, this is how we simulate a biological neuron. Now we have to connect them between each other forming a mesh. How do we do that?.

Figure 13 represents how a connection between two LIF neurons works. First, D1 and D2 are two drivers that control the voltage input to the neuron. In this case, two neurons are connected to the receiving one and they are transmitting two different neurotransmitters (spikes at different frequencies). W1 and W2 are where the input voltage spikes are transformed to a flow of current. Those two currents are summed and they flow towards the "N3" neuron capacitor. If the input current is bigger than the threshold value, the output voltage produced by the capacitor will also be bigger than the threshold voltage. The switch activates and the capacitor discharges creating the spike shape output. This output is received by the output driver "D3" which converts the analog spikes response into proportional voltage pulses whose values depend on the frequency and intensity of the spikes. Those last pulses are the input to another neuron.

Figure 13 (LIF artificial neurons connection)

So we reached the point where we know how spiking neural networks are formed. But we still don't know how it works. How do they learn?. Backward propagation isn't valid anymore because in order to calculate the gradient descent you need to define a continuous variable for the output, and spikes are not like that.

Till present, SNNs have been used in computational neuroscience as an attempt to better understand how our brain is functioning. One really interesting and active field of investigation is neuromorphic engineering, where we design computers implementing physical hardware level SNNs to perform tasks of the biological nervous system.

For example. If you want a robot to go to a certain position based on the camera lecture, you need to implement a program detecting all the objects (perhaps use a CNN for this), calculate the adequate route (this is achieved by state-space representation and heuristic searches), activate the wheels based on that calculated route and so on. All this system is implemented over classical hardware architecture, where sensor outputs are stored in memory same as the algorithms and programs. All the information is processed by a CPU and several integrated systems which then dictates what actuators have to do, like moving the wheels at a certain angular velocity.

With neuromorphic engineering, we will have a device (computer) where physical spiky neurons are implemented with the use of oxide-based memristors, spintronic memories, threshold switches, and transistors. Sensors and actuators may be directly connected to this device and then the SNN trained to activate the actuators based on the outputs of the sensor. No more memory-cpu-controllers-integrated circuits connection needed.

The main benefit of neuromorphic devices is energetic efficiency, since information can be transmitted through the neural network using very weak signals.

Now the big questions. How does this SNN work? How do we train it? How and why does it learn?

I want to be honest here and let you know that I still don't know exactly how they do it, so I am not able to explain step by step what happens at each learning phase. However, I will shortly share some interesting information about this.

To make Spiking Neural Networks learn we have to use unsupervised learning algorithms, this is, to let them learn by themselves. Some bio-inspired used learning algorithms are Hebbian Learning or Spike Time Dependent Plasticity. These rules enhance the weights of the receivers (W1 and W2 in figure 13) if the activity of the connected neurons is correlated, and decrease if the activity is not correlated. Therefore, the strength of the connection is higher for neurons that perform related activities. A stronger connection means that the transmitting neuron has a bigger impact on the receiving neuron. Let's suppose the SNN has trained already. When an input is received, the path of the spikes is determined by the strength of the connection. With the training process, we modify the strength of the connection to activate the same path of neurons when a similar input is given. Then, no matter if a car, a rock or any other object is dangerously approaching. In both cases same path of neurons will be activated and the action would be to esquivate. We eliminate all the processing power of detecting the object, identify the danger, calculate an escape path, etc. In fact, we don't eliminate that process, is just that the SNN has learned it automatically, without us having to define all those steps. However, this technology is very underdeveloped. There's still a lot of work to do to achieve desired results.

Recently (in 2020), it has been implemented a fully neuromorphic optical platform able to recognize patterns learning by itself.

Final Thoughts

This post is longer than I expected. The most accurate and sincere final tough ever :D

Anyway, the main reason is the explanation of the Spiking Neural Networks which I believe are the most interesting of all. They are not widely known by most programmers since are not implemented in deep learning algorithms, so I've encountered many AI courses, blogs, articles where SNN are not even mentioned.

The next two posts will consist of practical implementations of CNN and RNN. In the third post, we'll see how to implement a CNN for text classification while in the fourth post we are going to see how to perform stock price prediction using RNN. In both cases, I am going to use python and the TensorFlow library.

Below, you can see a video simulating the different Neural Networks activities. It's just three minutes long. I recommend watching.

Image is just a link. It will redirect

If you have any suggestions, doubts, don't hesitate to use the comments. We are all here to learn and share our knowledge. And, don't forget to follow me on LinkedIn and/or Twitter.

Thank you !!!

Getting started with FX: Powerful and handy JSON manipulation from the command line

Jorge Barrachina Gutiérrez — Thu, 03 Dec 2020 09:46:33 +0000

Why this post?

If you spend a lot of your time-consuming APIs and having to build data pipelines, you will enjoy this post (if you don't know fx yet).

According to several sources, data scientists spend between 70-80% of their time normalizing data before they begin to play with it. That's a lot of time, so it's a good time investment to have powerful tools at your disposal without a steep learning curve.

When we talk about data processing, one of my favorite anecdote is the following one: Command-line Tools can be 235x Faster than your Hadoop Cluster. Sometimes people spend thousands of dollars on new software without realizing they can do the same task faster and cheaper with some alternatives. You've got me, I'm an old-school command-line guy ;-)

The point I'm trying to make here is that there is a ton of tools out there, but my advice to you is: "be conservative with your toolset". Spend more time with the tools you use daily. If you detect a recurrent issue in several scenarios, then it's time to look for alternatives.

I used jq for a couple of years but every time I struggled with some data transformation, the pain came from the same place: learning the concrete syntax for that tool. In that sense, fx freed me from that inconvenience.

Don't get me wrong. jq it's a very useful tool, but it takes time to control it. In my case, I don't want to spend more time learning a concrete syntax that is only useful with jq.

In this post, I'm not going to cover how to use jq, but if you're interested in it, here are some useful references:

fx

How to install

Prerequisite: You have to install nodejs on your computer.

npm i -g fx

fx can do a lot of things (see What can I do with fx below), but let me first explain the two modes on which fx operates:

fx modes

Interactive: When you are not familiar with the data (JSON) you're playing with, this mode is quite useful because it lets you explore the data structure, find values, filter them, apply some transformation... Think interactive mode as a playground. Here you can see a sneak peak of fx in action:

As you can see, it's pretty intuitive! You can explore any JSON data in the same way you do when you are accessing an object in Javascript. Bonus point: it supports auto-completion.

Commandline (CLI): Once you know the data, it's time to apply some transformations. This mode can be used in scripts, it's pipe-friendly, so you can concatenate several fx commands in a one-liner. Think cli mode as a grep,sed or awk command, but fx reads JSONs instead of lines. Let's see another visual example:

TIP: When you want to select text in interactive mode you need to press Alt / Fn key depending on your terminal

Once we saw the available modes of fx, let's practice with some examples.

What can I do with fx

If you want to follow the examples, just type in your terminal:

curl -sS "https://jsonplaceholder.typicode.com/users" -o users.json

Quick JSON exploration

When you are in interactive mode you can search for strings or use regular expressions. If you are familiar with vim editor you'll feel at home. If you're not familiar with regular expressions, you can start just typing "/" followed by the string you are looking for.

By default, the search is case-insensitive, so you don't have to worry about that.

To navigate across search results, just press Enter to go to the next match.

If you don't feel comfortable with regular expressions but you want to practice, I recommend you to take a look at RegEx101. It's a playground where you can start to master regular expressions.

Transform

From each user, I want to keep only the website and the "geo" keys.

Easy peasy! Because in fx we can use plain Javascript, let's translate this scenario to a Javascript code, and later apply this directly in fx:

For each user (object) we can make use of destructuring technique, to get the keys we want (website,geo) from the object and discard the rest:

var user = {
  "id": 1,
  "name": "Leanne Graham",
  "username": "Bret",
  "email": "Sincere@april.biz",
  "address": {
    "street": "Kulas Light",
    "suite": "Apt. 556",
    "city": "Gwenborough",
    "zipcode": "92998-3874",
    "geo": {
      "lat": "-37.3159",
      "lng": "81.1496"
    }
  },
  "phone": "1-770-736-8031 x56442",
  "website": "hildegard.org",
  "company": {
    "name": "Romaguera-Crona",
    "catchPhrase": "Multi-layered client-server neural-net",
    "bs": "harness real-time e-markets"
};

var {website,geo,...rest} = user; // user is our object
// website = "hildegard.org"
// geo = {"lat": "-37.3159", "lng": "81.1496"}

Now we want to apply this operation on each user, so let's do it with .map

users.map(({website,geo,...rest}) => ({website,geo}))

In fx, we will do it like this:

cat users.json | fx '.map(({website,geo,...rest}) => ({website,geo}))'

Isn't it beautiful?

You may think: "this is very basic stuff Jorge." Yep, indeed. But put yourself in the shoes of someone who has to do lots of different data transformations every day, or someone who is just getting insights from different data sources each time... Do you think that person is going to write a script every time?

That's the beauty of fx for me, it lets you do things very quickly without the need to learn anything more!

Filter

From our JSON file, I want to filter the company names of those which email has the domain .biz

cat users.json | fx '.filter(({email,...rest}) => /\.biz$/.test(email))' '.map(user => user.company.name)'

Got it!

Extra ball: Could I convert the results above in a CSV format (company and mail by line)?

cat users.json | fx '.filter(({email,...rest}) => /\.biz$/.test(email))' '.map(user => `${user.company.name};${user.email}`)' '.join("\n")'

Use your favorite npm module along fx

fx offers a way to include npm modules in the execution context.

When you are dealing with data structures in Javascript, lodash is a very handy option. Also, dayjs let us play with dates and time data easily.

Let's see how to use it along with fx:

Create .fxrc file in $HOME directory, and require any packages or define global functions.
Install lodash and dayjs globally in your computer:

npm i -g lodash dayjs

Set NODE_PATH env variable. This step is IMPORTANT to allow fx to make use of globally installed packages.

export NODE_PATH=`npm root -g`

Put in your .fxrc file:

Object.assign(global, require('lodash/fp'))
global.dayjs = require("dayjs");

Now, let's play this scenario: I want to have a list of the 5 most recent github repositories I have that includes the day of the week I created each of them.

curl -sS "https://api.github.com/users/ntkog/repos" |  \
fx '.map(({name,created_at,clone_url,...rest}) => ({name,created_at,clone_url}))' \
'sortBy("created_at")' \
'reverse' \
'take(5)' \
'map(repo => ({...repo, weekDay : dayjs(repo.created_at).format("dddd")}))'

Let's look at each step (we can chain several transformations in fx as you can see)

"\" at the end of the line it's just for separating one command into several lines

Get all info of my github repos and pipe it to fx

curl -sS "https://api.github.com/users/ntkog/repos" |

Keep only name,created_at,clone_url from each object

fx '.map(({name,created_at,clone_url,...rest}) => ({name,created_at,clone_url}))' \

Sort array by created_at key

'sortBy("created_at")' \

Invert the order of the results

'reverse' \

Take 5 objects

'take(5)' \

Add weekDay key to each object

'map(repo => ({...repo, weekDay : dayjs(repo.created_at).format("dddd")}))'

It's a very expressive way to transform the data step by step, don't you think?

Explore more possibilities

I didn't find a lot of articles talking about fx, but this talk from Антон Медведев gave me a lot of ideas on how to get the most out of fx.

Give fx a try, you won't regret it! Let me know if you find other tricks!

Happy hacking :-)

Photo by Samuel Sianipar on Unsplash

Postman quick tricks

Jorge Barrachina Gutiérrez — Sun, 22 Nov 2020 10:21:23 +0000

Postman is an awesome tool. It lets you automate a lot of the work when you are playing with API's. But are you really getting the most of out it?

I'm going to show some little tricks that can help you save valuable minutes in your day-to-day workflow.

If you want to reproduce these tricks, you will need to have installed nodejs and Postman Collection SDK in your computer.

Scenario #1: Rename all the items of a collection adding a prefix sequence index

Sometimes we are working on a large postman collection and we want to be explicit on what order of execution the user should follow.

Adding a prefix to each item of the collection seems like a good idea, but if we have several items in our collection, doing this manually is pretty boring..., there has to be a way to do it quickly...

Indeed! There is an easy way! Here is the code for the impatient:

Create a file called rename_items_collection.js and paste the following code:

// Import Postman Collection SDK
const fs = require('fs');
const {Collection} = require('postman-collection');
const FILENAME = './sample-collection.json';
const SEP = '-';

// Read our postman collection file
const myCollection = new Collection(require(FILENAME));

// Update list of items renaming each of them with a sequence prefix
myCollection.items.members = myCollection.item.members.map((item,idx) => item = {...item, name: `${idx+1} ${SEP} ${item.name}`});

// Output collection content
console.log(JSON.stringify(myCollection.toJSON()));

Open a terminal and type:

node rename_items_collection.js

You will see in your screen the contents of the collection. If you want to save it, run this one:

node rename_items_collection.js > renamed_collection.json

Now, you can import renamed_collection.json in your Postman App and you will see each item name prefixed with an index.

Scenario #2: Make requests with fake data

You need to test your API with some random and fake data, but you don't want to implement some function to randomize each data type.

Did you know that Postman has dynamic variables based on faker.js mocking data library?

The best part: There is some "Finance" data you can mock. Here are some examples:

Random IBAN account number ? : use {{$randomBankAccountIban}}
Random ISO-4217 currency code (3-letter) ? : use {{$randomCurrencyCode}}
Random Bitcoin address : use {{$randomBitcoin}}

Take a look of the complete variable list.

If you want to use these variables in a Pre-request section, you should use it as in the following example:

// Generate a random UUID

// This works
var uuid = pm.variables.replaceIn('{{$guid}}');

//This won't work
var uuid = {{$guid}}

Scenario #3: Check JWT claims with Javascript within Postman

I don't know you, but when I work I have several applications opened, sometimes too many.

When I have to test or debug an API that makes use of OAuth 2.0 with JWT, sometimes I need to check if a request has proper data in the JWT. It's useful to remember Occam's Razor :

" of two competing theories, the simpler explanation of an entity is to be preferred"

What does it have to do with this scenario?

When you are troubleshooting some requests, we tend to look for complex assumptions. It's better to start with the easiest ones, which are the most frequent. So, let's do it.

Imagine we have the following JWT:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

As we know, every JWT consists of 3 parts (the '.' "splits" each part). I've just given you a clue... .

If you want to know the claim content (Ignoring verifing the JWT signature), can you do it?

Yes! with 2 lines of Javascript!

Put the following lines in the Pre-request tab on the request you want to check

var [jose_header,payload,] = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c".split('.');
console.log(JSON.parse(atob(payload)))

with the atob javascript native function we can decode Base64

If you have the JWT content in a variable called assertion you can substitute the string with the following example

var [jose_header,payload,] = pm.variables.get("assertion").split('.');
console.log(JSON.parse(atob(payload)))

Here you have a reminder diagram on Postman supported variables and their scopes:

When you run this code, you will see in the Postman console:

{
  "sub": "1234567890",
  "name": "John Doe",
  "iat": 1516239022
}

Scenario #4: Signing JWT tokens directly within Postman

Maybe you know this amazing cryptography tool called jsrsasign : It supports a lot of the common tasks you have to do when working with secure APIs:

RSA/RSAPSS/ECDSA/DSA signing/validation
ASN.1
PKCS#1/5/8 private/public key
X.509 certificate
CRL
OCSP
CMS SignedData
TimeStamp
CAdES JSON Web Signature/Token/Key (JWS/JWT/JWK)

There are multiple ways to use this library within Postman. We, as developers, should evaluate which way is better for our use case. Here you have two ways of using jsrsasign:

Load jsrsasign from external URL

This is the simplest way to use it: postman-util-lib. Kudos to joolfe.

If you want to try this way, on the postman-util-lib website there is a good documentation on how to use it

But here are two corner cases you can think about:

Should we trust a site we cannot control?
What if you work in a restricted environment where every url needs to "be validated" beforehand in the firewall of your organization?

Thinking about those scenarios, I want to share with you a way of using this awesome library locally.

Load jsrsasign locally

So, let's do it!

Trial #1: Read the library from a local file

Unfortunately, this is not possible yet in Postman :-( . Take a look this issue in Postman App Support.

Trial #2: Serve the library from localhost

Let's grab the file from https://raw.githubusercontent.com/kjur/jsrsasign/master/jsrsasign-all-min.js.
Let's serve this file from localhost. We can use http-server nodejs package to do it. If you prefer to serve the file with another method, there are a ton of them.

mkdir jsrsasign_library
cd jsrsasign_library
wget https://raw.githubusercontent.com/kjur/jsrsasign/master/jsrsasign-all-min.js
npm i -g http-server
http-server . -p 8080

From your browser you can reach the file at http://localhost:8080/jsrsasign-all-min.js

Next, assume you have a variable in the Postman environment called sign_secret . If you just want to try it, you can substitute in the following code with a string (Although it's a bad practice)

Now go to Pre-Request tab , and copy the following

var URL_local_jsrsasign = "http://localhost:8080/jsrsasign-all-min.js"
if(!pm.globals.get("jsrsasign")) {
    pm.sendRequest(URL_local_jsrsasign ,(err, res) => {
      if(!err){
       pm.globals.set("jsrsasign", res.text());
      }    
    });
}
// Load jsrsasign library in global context
eval(pm.globals.get("jsrsasign"));

const jose_header = {
  "typ": "JWT",
  "alg": "RS256"
};
const payload = {
  "sub": "1234567890",
  "name": "John Doe",
  "iat": 1516239022
}

// Sign JWT
var jwt = KJUR.jws.JWS.sign("HS256", jose_header, payload, pm.environment.get("sign_secret"));
console.log(jwt);

// verify JWT
isValid = KJUR.jws.JWS.verify(jwt, pm.environment.get("sign_secret"), ["HS256"]);

I hope you find these little tricks useful. Happy hacking!

Cover Photo Credit: Photo by Barn Images on Unsplash

Introduction to Deep Learning, basic ANN principles

ABuftea — Sat, 21 Nov 2020 09:32:09 +0000

Hi All,
With my first contribution to the community I would like to share with you my knowledge about deep learning and its applications. This will consist of a series of posts which I call Deep Learning for Dummies, coming out approximately one post a month.

As of this special occasion, this is my first post, I would like to start thanking the Santander Dev community peers for their warm welcome and for giving me the opportunity to grow my knowledge collaborating at this amazing initiative.

I thought this series as both, a theoretical and practical introduction to Artificial Intelligence for someone who doesn't know anything about it. Expect to find easy-going language and explanations of all the concepts involved.

Below you have the lists of posts that I plan to publish under this series. I will keep it updated with each published post. Unpublished posts titles and used libraries can change:

Introduction to Deep Learning, basic ANN principles
Artificial Neural Networks: types, uses, and how they work
CNN Text Classification using Tensorflow (February - March)
RNN Stock Price Prediction using (Tensorflow or PyTorch) (April - May)
Who knows? I may extend it. Perhaps some SNN use case.

This is a long one but I think totally worth reading, specially if you are new to this world. In my opinion, it gives you a good base for understanding how any deep learning algorithm works. Let's see the contents.

What is deep leanring?
Artificial Neurons
Activation Functions
Artificial Neural Networks (ANN)
Learning process of an ANN
Review of the whole trainining/learning process
Final Thoughts

What is deep Learning ?

Figure 1 (AI subset domain)

Figure one is a graphical representation of the concepts that I am going to describe below.

The term Artificial Intelligence was coined back in 1955 by John McCarthy in a workshop at Dartmouth College where attendees created algorithms to play checkers. John and its team invented the alpha-beta algorithm which is an improvement of the min-max algorithm. Those algorithms are designed to search the optimal move for the player, the one that minimize the loss and maximize the gain. Clicking on the examples, you can see that the unique part of those algorithms are their design. The developers developed the logic which the algorithm follows in order to calculate the best move. We will see that deep learning is completely different to this.

Machine Learning, a term coined by Arthur Samuel in 1959, is a subset of AI algorithms that use structured and labeled data to learn by themselves and make accurate predictions when a not labeled input is given. There are various machine learning algorithms, I will list some of the most popular in case you want to research and learn more about them: Naive Bayes, Random Forest, Support Vector Machine, K-Nearest Neighbor. The idea between this technique is that you create a model which, while learning or training, it modifies the mathematical function that describes its output till that function outputs an accurate prediction when and input is given. This is achieved by training and testing the model with previously labeled data.

See a visual example in the "figure 2". Let's say that we want to predict how many ice-creams are going to be sold based on temperature readings. We take the historic data of ice creams sold at different temperatures and plot it as a cluster of points in a 2-axis graph. Now our ML algorithm can be represented as a line which represents our prediction, meaning that if we want to know how many ice creams will be sold at 30º degrees, we just have to choose that point in the graph over the line and read the ice-creams axis, "y". The process of learning involves improving the shape of the line till it better represents the distribution of the points cluster. As you can realize, the importance of the data quality here is crucial, since a nonsense cluster of points will make it impossible for the algorithm to adjust the line properly.

Figure 2 (Ice-Cream Readings and ML Model Linear Representation)

This is similar to what deep learning does, the difference here is the method we use to do so. Deep Learning, term that was first introduced by Rina Dechter in 1986, is a subset of Machine Learning algorithms that focuses on performing the prediction task by mimicking the human brain structure. This is why it has associated terms such as artificial neurons and artificial neural networks, however, don't make the mistake to believe that our brain works in a similar way that deep learning algorithms do. Our biological nervous system is much more complex such that as of today we still don't completely understand how does our brain works. Artificial Neural Networks that we use in the deep learning algorithms are just an extremely simplified mathematical model of our brain strucutre.

Let's dig more into the deep learning components.

Artificial Neurons

As I stated above, Deep Learning is about designing algorithms that learn by themselves in a such a way that simulates our biological brain behavior. This is achieved thanks to the implementation of Artificial Neurons, and, when those are connected to each other they form what we know as an Artificial Neural Network.

In "figure 3" you can see the representation of an artificial neuron. It is nothing else than a series of mathematical functions where Xi are the inputs, Wi are the weights associated with every input, b is the bias unit and Y is the output. Ass we can see, the neuron computes the sum of the inputs multiplied by the weights, and then adds the bias term. After, it passes the result of this operation to the activation function. Finally, the output of the activation function "Y", is the final output of the neuron.

Figure 3 (Representation of an Artificial Neuron)

The value of the weights and the biases are assigned at the beginning when creating the algorithm, afterwards, those parameters are automatically updated through the training process. In fact, when a neural network is learning, it is updating the values of the weights and biases of each neuron till the output is the desired one. After training, those values can be stored and introduced to a copy of the neural network in another computer, which will perform with the same accuracy as the one we have been training. We will see more about this later on.

The role of the activation function is to encode the result of the summation between a fixed scale which has to be the same for all the neurons in the same layer, so the neural network can learn from the output of each neuron. When the algorithm is predicting, it bases its conlcusions on the relationships between the outputs of each neuron, this is the reason why we need the output of each neuron in the same layer to be scaled between the same range, so the algorithm can see what is the impact of every neuron from the same layer to the final output. This allows for correct weight and biases recalculation when training, what makes our model have a better prediction each time. We'll see what is a layer and how does the process of training works later on.

Activation Functions

The activation function is what we call the mathematical operation that the neuron performs over the sum of the inputs. One important aspect to consider is that neural networks used in industry consist of thousands of neurons and every single activation function has to be computed, then, the function has to be simple enough to reduce the computation complexity of the algorithm, allowing for quick predictions.

Now I am going to list the most popular activation functions employed in neural networks and you'll see they are extremely simple. Theoretically any given function can be used as an activation function, however, due to the reason covered in the paragraph above, those are the most common:

Sigmoid Function: This function represented in the figure 4 scales the output between 0 and 1, normalizing the output of each neuron. Its advantage is that the function is smooth, preventing unexpected jumps at the output. On the other hand, the main disadvantage is that scaling between 0 and 1 makes it difficult to differentiate clearly between a multiple set of large inputs, so we have a bad resolution. In addition, as we see on the graph, if the input is larger than 6 or smaller than -6, the output will allways be 1 or 0 respectively.

Figure 4 (Sigmoid activation function)
Hyperbolic Tangent (TanH): This function represented in the figure 5 has a similar behavior as the softmax but scaling the output between -1 and 1 instead. The positive part is that we have a higher resolution, meaning that the we are able to differentiate better between similar inputs. It also allows the network to identify more features of the dataset, since the output can have both signs, plus and minus, so it helps the prediction. The negative part is that the slope of the function increased, decreasing the maximum input range that we can manage. If we have an input smaller than -3 or bigger than 3, the output will allways be -1 and 1 respectively.

Figure 5 (TanH activation function)
Rectified Linear Unit (ReLU): This function represented in the figure 6 solves the problem of the big inputs. If the input is positive, it returns the same value, while if the input is negative is returns 0. The best part of this function is that it does not activates all the neurons at every input, reducing the computational complexity. On the other side, we don't have any sensibility for negative inputs. There are some modifications of this function that solves the problem of negative inputs insensibility, like the LeakyReLU. Another aspect to consider is that this function does not scale the output between a range, so depending on the problem you want to solve it may be the one you need or not.

Figure 6 (ReLU activation function)
Softmax: This function represented in the figure 7 is different from the others and it is mainly used at the output layer when the task of the neural network is to classify the input between some previously established classes. This function divides the exp() of one input by the sum of all the exp() of all the others inputs. As a result, the sum of all the outputs of the neural network will always be one, meaning that each output represents the probability of each possible class. You need at least two outputs from the neural network, this is clasifying between two classes. Otherwise, if you only have one output it will allways be one.

Figure 7 (Sigmoid activation function)

Artificial Neural Networks (ANN)

By now we know what is deep learning and what is the main component of it, the artificial neuron. It is time to join several neurons between them forming what is known as an artificial neural network.

Figure 8 shows a graphical representation of the ANNs. Looking at it you can understand what a layer means. A layer is a group of neurons acting at the same level. Neurons on the same layer are not connected between each other. We can also see that all the neurons at one layer are connected with all the the neurons of the previous and next layers.

Figure 8 (Artificial Neural Network)

The minimum amount of layers we need to form an ANN is 3, meaning that we'll have the input layer, one hidden layer and the output layer. This structure forms what we know as a shallow neural network, shallow because we have only one hidden layer. We call hidden layers to all the layer which are in between the input and the output layer. When an ANN consists of more than one hidden layer, it forms what is known as a deep neural network.

Let's make it easy to understand how it works using an example, let's say we want to predict the price of a house based on three known characteristics, the number of rooms, the neighborhood where the house is located and the size of the house. Those would be the inputs to the ANN and the output is the price of the house. Figure 9 graphically shows the situation I am describing.

Figure 9 (Inputs and outputs for house pricing prediction)

Since artificial neural networks always work with numbers, the first step we have to do is to define the neighborhood information using numbers. This is achieved considering the context of the problem. For example, if we want to predict the price of homes in San Diego area, we could simply assign one random number to every neighborhood in San Diego and use that as the indication of the neighborhood. However, this methodology is not the most appropriate because the algorithm may automatically build wrong relationships between that numbers. Let's say the neighborhood 8 is a good one and prices there are more expensive. The algorithm could consider one house in the neighborhood 7 to also be more expensive, because numbers 7 and 8 are close to each other then it concludes the neighborhoods are similar. However, since we assigned the numbers randomly, the neighborhood 7 can be totally different than the 8. The process we've used to transform the location into a numeric input is making our algorithm take wrong conclusions. This is very important, we have to be careful with the way we process the information.

In this specific case, we can search which neighborhoods from San Diego are more expensive and which are cheaper. We could assign similar numbers to similar neighborhoods and totally opposite numbers to contrarian neighborhoods. The algorithm will still make the conclusion that closer numbers are related with similar priced neighborhoods, but this assumption will be true. In fact we are helping the algorithm make a better predictions, which is exactly what we want.

If the case is that we cannot decide any logic of assigning similar numbers to similar neighborhoods, then we could vectorize the information of the neighborhood to make it unrelated. This means that instead of a number we could build a vector of let's say 3 numbers (the size of the vector has to be chosen in function of the problem analysis), and all of those numbers together represent the location of the house. Working with vectors makes it easier for us to assign random numbers to the location, building an algorithm that assigns those numbers in such a way that all the vectors are different from each other so the ANN cannot make conclusions from them.

There is a question which you may be asking at this point. If we have three neurons at the input layer and 5 values at the input, ¿can we actually input those values to the neural network?. The answer is no. We would need to either add two more neurons to the input layer, or transform the location input vector in a single value. One way of doing it is calculating the magnitude of the vector, and use that value as a representation of the neighborhood.

As you will see later, designing ANNs is somehow like an art at which you get better with experience. You don't know what conclusions the algorithm will make, so you cannot predict the output. You can only see if the output is correct or wrong, but you cannot determine why did the ANN predicted that value. This is a bit of a fake statement, since reproducing all the calculations of every single neuron at every iteration would allow us to fully understand how neural networks make conclusions. However, we need millions of data to train industry used algorithms and they consists of many thousands of neurons, it is not practically feasible to perform this task. Also, depending on the problem and your dataset, tracking several outputs and using intuition would help us to somehow predict what characteristics of the data is making the ANN output those values. However, this is just an intuition and it takes a lot of practice to master.

Learning process of an ANN

Now that we know what is a ANN and how it works, the last step is to see how it learns. Remember, ANNs are formed by layers of neurons where every neuron is a simple mathematical function whose input term is the sum of the outputs of the previous layer multiplied by some weights and with a bias term added. Figure 10 represents this, where W are the weights, b is the bias term and a is an artificial neuron. The sub indexes "i" and "j" represent the position of the neuron in the layer and the upper index "l" represents the layer in which the neuron is located.

Figure 10 (Weights and biases of an ANN)

The key to make the ANN learn are the weights and the bias terms. To put it simple, during the process of training we know what the correct output should be. After the ANN made its prediction we can calculate how far away that number is from the expected output. Now we just have to update the weights and the bias term in such a way that the next time the predicted output is closer to the expected one.

Sounds easy right? But how do we do that? We could simply use our intuition, or assign new weights and biases randomly but this probably will not take us anywhere. Instead, we need to define a clear and systematic method of training, which is achieved thanks to the use of the loss and optimization function.

The first step of the training process is the forward propagation where the inputs cross through the input layer towards the output layer. Each time the network outputs a prediction, we compute the difference between the predicted and expected result through the loss or cost function. Once we calculated that difference, we push back this value to every neuron adjusting its weights and bias values in such a way that after each iteration the value of the error obtained is getting smaller. This pushing back process is known as backward propagation and is efficiently performed through the optimization function. The code that makes all this process of training possible is known as the optimization algorithm. Look at figure 11 for a graphical explanation of this.

Figure 11 (Backward propagation process)

We could design a simple optimization algorithm. Using the house price example, we can calculate the difference between the predicted price and the real price so we know if the predicted price is bigger or smaller. Now, we divide that difference between the number of neurons at each layer, we multiply the result of this last operation by one parameter and we substract or add this result to the weights and biases. This is one way of updating weights after each iteration but of course, this is not the best way to perform this task. The network may not be able to ever learn anything, or what is the same, the loss functions will not converge to its minimum value at any moment.

The main idea of the optimization algorithm is to make the loss function converge to its minumum after each training iteration. The loss function measures the error, thus, the smaller it is, the smaller the error is, therefore, better is the prediction.

This is the reason why optimization functions usually calculates the gradient of the loss function. The gradient is the partial derivative of the loss function with respect to the weights. After, the weights are modified in the opposite direction of the calculated gradient. Ass we see in figure 12, the gradient vector of one function at a point (x, y) points towards the direction of the greatest rate of increase of the function at that point. If we want to reduce the function, we should move in the opposite direction. Have a look here for more information about gradients.

Figure 12 (Gradient of 3D map at a point)

So we calculate the gradient of the loss function with respect to the artificial neuron weight at every layer and we variate the weights and biases of that layer in the opposite direction of the gradient. The amount which we variate the weights is called the learning rate and is represented as alpha (α).

In function of the outputs we are looking for, we have three main categories of loss functions to choose. Regressive loss functions, used in the cases where the target variable is continuous. Embedding loss functions, when we deal with problems where we have to measure if two inputs are similar or not. Classification loss functions, when we deal with problem where we have to determine what class describes better the input from a set of given classes. I am not going to cover any loss function in this post. Only to give you an idea, the mean squared error is a commonly used loss function which calculates the sum of squared values between our target variable and predicted values. Have a look here if you want to see some examples of loss functions.

As of optimization algorithms, they fall in two classes. Constant learning rate algorithms, which use a constant value for the learning rate and adaptive learning rate algorithms which modifies the value of the learning rate while training. Researches suggested that the Adaptive Moment Estimation algorithm (ADAM) compares favorably to any other adaptive learning rate algorithms. It works with momentums of first and second order, storing an exponentially decaying average of both, past gradients and past squared gradients. On the same way as with the optimization functions, I am not going to cover more about the optimization algorithms because is not the goal of this post, but be free to learn more about them here.

On the third and fourth post of this series, where we'll see practical examples where I will explain with more depth the loss function, optimization algorithm and why did I chose the specific activation function for the neurons at every layer.

Now that we've seen all the components of an ANN and what are them for, I think it's a good moment to review the whole process of building a neural network, so you leave with a clear picture of it.

Review of the whole trainining/learning process

Figure 13 shows us a visual interpretation of the training process of a neural network.

Figure 13 (Training process of an ANN)

We'll keep using the houses price example. Let's say we have a dataset of 10 thousands homes in San Diego for which we know the price, we know the number of rooms, their size and the neighborhood. As I explained above, the first step is to transform the neighborhood information from text or a map position to numbers. We will use the map coordinates (longitude and latitude) to perform this task and we'll represent both of them as a single decimal number ranging between (-3 and 3). The number of rooms in the house isn't such a big so it should range between 6 and 0, then we could not to apply any transformation to it. Lastly, we will transform the house size information from square meters or feets to a decimal value, so a house of 632 square meters will be inputted as 0.632 to our neurons and a house of 6000 square meters as 6. Figure 14 shows a diagram of this transformations.

Figure 14 (ANN Inputs)

At building the ANN you first have to decide how many hidden layers and what kind of neurons you want for each layer. Let's say that we have chose neurons with the sigmoid activation function for the input layer. That's why we did the transformations to the inputs, since the sigmoid has sensibility for an input range of (-6, 6). If we don't transform the inputs to ensure they never get outside that range, our ANN would not be able to consider the house size information nor the location in its predictions, since those values would always be bigger than 6.

Now we have to choose the neurons for the hidden layers. Let's say that we choose the Tanh function for the first hidden layer and the Sigmoid function for the second hidden layer. Remember that the output range for the Tanh function is [-1, 1], and the price of a home is always positive, then the sigmoid on the second hidden layer will transform the output from the Tanh function to a always positive value between 0 and 1. Finally, since the price of a house can vary unexpectedly between the range (0, infinity), we will choose the ReLU activation function at the output layer because we don't want to limit the output between a specific range. What's more, let' say that we want the price to come out of the algorithm in actual dollars, so we don't make any transformation to our expected output.

Knowing that each neuron sums all the input terms and then adds the bias to it, and knowing the input and output range of our chosen activation function, we can assign initial values for the weights and the biases.

In this case, we don't want the input to the first hidden layer to be bigger than 3 or smaller than -3. As the input layer has three sigmoid neurons, each neuron on the first hiiden layer will have a maximum of three inputs that will range between 0 and 1. Then we chose the initial weights value for the input layer to be 1 and the bias term to be 0. Using a similar logic, we set the weights of the first hidden layer to be 1.5 and the bias value as 1.

Finally, since we decided to not transform the expected output, our ANN oputput has to be of the order of several hundred of thousands (the price of a house in dollars). On the second hidden layer we have sigmoid neurons whose output range between 0 and 1. Then we have to set the weights and bias of the second hidden layer in such a way that the sum of all the neurons output is able to result in the price of a house in dollars, because the ReLU neuron at the output will just return the value that it takes as input. Then, we assign weigths of 250000 and a bias term of 100000 to the second hidden layer. Figure 15 shows the distribution that I've commented.

Figure 15 (inicial weights and bias values)

This is a bad distribution. Not normalizing the output was a bad choice, we should have transformed the expected output so we could afford to reduce the weights keeping the network balanced and able to learn at a stable learning rate. For example, to correctly design this ANN we should have tranformed the house price to the order of hundreds or tens. So you can build a scale where the ouput 100 represents a house worth 1 milion.

It make no sense to have such a big weight as 250000. Even if the algorithm would be able to learn, it will progress so slowly, because we have to vary such a big weight with every itteration, that we may not have enough data in all San Diego to make the algorithm learn at the rate it is doing so.

However, I chose to explain it using this bad design to make you understand that the values of the parameters and the types of neurons that are used is the choice of the developer, and there are an infinite number of combination. Of course, you can do whatever you want, but it does not mean that what you are choosing to do works. Only testing and experience will give you the ability to make the right choices.

After we have designed the artificial neural network and initialized the weights and biases, we input the information of our first home from the dataset. Forward propagation process starts and the ANN outputs a value which is the predicted price for the house whose characteristics are given as inputs.

Let's imagine the output is 100000 while the real price is 250000. We use the optimization algorithm to first calculate the value of the loss function and then the gradient of the loss function regarding the weights at each layer. We apply the learning rate to every weight and bias, adding or substracting, whatever is the opposite direction of the gradient.

After this, we input the information of the second home in our dataset and so on and so forth till we have used all the 10 thousands homes from our dataset. Each time we have iterated once with every house from the training set, we completed what is known as an epoch. An epoch is the term that defines the process of training the algorithm once with the full testing dataset. We usually perform several epochs over the same dataset when training an ANN.

We have two ways of visualizing if the training is going well. We could see that the value of the loss function is decreasing at every iteration, and we also could see that the predicted price is getting closer to the expected price at each iteration. Usually, since the value of the loss function can be normalized, tracking the loss function is a better way of monitoring the training process.

Final Thoughts

By now I hope you completely understand what is Deep Learning and how does it works. You should also have a vision about how to choose the appropriate types of neurons and how important is the quality and processing of the dataset. For example, it may be a better option to divide our housing price prediction algorithm in two, one for luxury homes and one for median household owners. In this way we would have a smaller range of inputs and outputs, making it easier for the model to learn, or what is the same, for the loss function to converge at a minimum value. However, it also depends on the application and the client you are developing for.

Another key idea that I want to share about deep learning is that to develop a good model you need a wide knowledge about the problem you want to solve, and you need to understand pretty well the data that you have access to and what can you do with that data. Before deep leaning we had to build the logic of the algorithm in order to predict the price of the house. Now, the logic is automatically learned by the algorithm, but you need to help him do so by defining the correct hyperparameters and preprocessing the data, ensuring it is of high quality. Its like you need to guide the algorithm through the learning process.

By the way. In deep learning we call hyperparameters to every parameter whose value is external to the model and cannot be estimated from the data, (number of hidden layers, number of neurons per layer, activation functions, loss functions, optimization function, optimization algorithm, learning rate, number of epochs at training, etc). We call model parameters to every parameter whose value can be learned by the model during the training process or can be estimated from the data (weights, biases, input/output range, error, etc).

Finally, I want to comment that there are some important aspects of deep learning that we haven't covered. Like overfitting, meaning that we trained the algorithm too many times with the same dataset so it performs almost perfectly for any given value form that dataset but it fails at predicting when a value outside that dataset is inputted. Also, when we have a dataset of values, we have to be sure that the values inside represents all the possible options. For example, if we have an 80% of medium class houses in the data, our model performance will be low when predicting the price of a luxury home. Furthermore, we need to track the performance of our model after training, so we don't use all the data that we have in our database for training. We split the dataset between the training set and the testing set, ensuring that both sets contain examples from all the possible values.

We will cover this concepts in the third and fourth posts, where I will share with you two practical examples of this, using python.

The next post, coming out in approximately a month, will be about the different types of artificial neural networks and what are them most suited for.

Thank you !!

How to create an API generator

Rafael Borrego — Fri, 20 Nov 2020 19:48:36 +0000

Development teams are creating dozens of APIs per project since the arrival of microservices and we should try to make as easy as possible for them to create new ones, reduce repetitive tasks and standardise different patterns and configurations so they are applied the same way everywhere.

Do you want to learn how to create a tool that allows to generate new APIs within seconds and achieve the above goals? If so we are going to show you how to do it with Maven archetypes, a tool that allows to generate any type of project, even non-Java ones.

What can be automated?
I guess your pipeline, Docker and configuration files are quite similar, they just change some Git urls and project name.

What about the actual code?
I guess in most projects you do the same type of actions (create, read, modify and delete) and the main difference is the type (loan, mortgage, trade...) and they follow a similar structure (endpoints on the top layer, business logic in the services, queries on the repositories, and so on).

What about the validations and tests?
I guess you have special ones focused on specific scenarios but there are also some generic ones that feel repetitive to implement them again. All of them can be automated so developers don't spend hours writing them again or copying and pasting from previous projects. They can spend their valuable time focused on writing specific business rules (such as ensuring shops cannot sell more products than what they have on stock) or building some amazing features none of your competitors has yet.

How to do it?

There are a few options:
1) Create an API generation pipeline: this is a great idea and can be done using e.g. Ansible scripts. The problem is that these scripts are usually managed by a centralised team like SRE that may not have capacity to generate projects that suit the needs of every team, not to mention that each team may use completely different technologies (Java vs Kotlin, Angular vs React, ... ).
2) Create your own Spring Initializr: Pivotal have created a tool to create APIs and we can customise it, but it is focused on Spring Boot projects and we may want to provide generators for also other technologies.
3) Create your own archetypes: Maven allows to create projects from a template providing some input parameters. We will show how to create one and use it.

Steps to create your first archetype:
1) Choose a good project to use as a reference. You can modify it later but it adds extra work to re-implement one or change its structure, rename package paths, etc.
2) Create an archetype: you just have to run the command mvn archetype:generate and choose the base template you prefer.
3) Create a configuration file in which you specify which folders should be picked and which files inside them. You can find an example here.
4) Copy the code of your reference project inside the archetype-resources folder.
5) Replace the variable parts (like the entity names) by placeholders. The syntax for the folder names is __text-to-replace__ and inside the code it is ${domain}. You may want to do some smarter replacements what can be done using Velocity templates.
6) Add it to your Maven repository, either to your local one (using mvn install) or to your remote one (using mvn deploy, ideally in a pipeline).

That's it. Now you will want to create your own API using the archetype. You just have to run a command in which you provide its basic details (Maven groupId, artifactId and version) and the values to replace on the placeholders, and it will be generated within seconds.

An example would be:
mvn archetype:generate -DarchetypeGroupId=com.mycompany -DarchetypeArtifactId=my-first-archetype -DarchetypeVersion=1.0.0-SNAPSHOT -entity=customer

Do you want to see the full code of an archetype? You can find one here.
Do you want to share other ways to automate your project creation or want to ask how to automate other tasks? If so we would love to read your comments!

QuantumRNG-aaS - Making use of Quantum Algorithms

Mark C. — Mon, 26 Oct 2020 10:29:37 +0000

We have all heard the hype about Quantum Computing...

"Quantum Computers are gonna break everything!!"

"Quantum is coming... the wind is blowing..."

In a world where 'quantum' is equally likely to be used to sell you dishwasher tablets or toilet paper, as much as it is to be used as a way to scare people with headlines like "Quantum Computing and the End of Encryption" (really, really not helpful, Hackaday...) there is now somewhat of a skills rush to be able to have technicians who can make sense of this bold new world of quantum computing.

What this Hacktober project will demonstrate is the following:

That quantum computing is certainly within the understanding of most of us
Cloud quantum offerings can be implemented into real-world systems beyond the realm of 'novelty'
With the right approach, a quantum circuit can be used to provide something actually useful for modern applications.

TL;DR - we're going to work through the theory to arrive at this proof of concept - and then offer up the model that we can build up and collaboratively code as part of the Hacktober activities so that we can arrive at a pretty cool output: Quantum-RNG-as-a-Service! #QRNGaaS

DISCLAIMER - this code is NOT for use in production systems without a significant amount of extra engineering and checking. YMMV and Caveat Developer!!

Primer on Quantum Algorithms

It's impossible to do much with quantum computers at the moment without delving into some deep-tech ideas of what is going on at the fundamental level of a quantum computer - principally discussing the idea of a 'qubit'.

But why do we have to do this? Well, the reason is that quantum computers are much closer to a Z80 processor than they are to a modern Intel i9. And like with the Z80, in order to make them workable, you have to have a good idea what is going on deep under the hood. One day we will have a wonderful, iterative abstraction model for quantum computing - but sadly, that day is not today!

So what isn't a quantum computer?

Put simply, a quantum computer is not 'just a faster computer'! Quantum algorithms operate in a totally different way to regular, 'classical', computers.

You're not gonna get MS Flight Sim running any better with quantum computers - especially not as they are right now! Quantum computers are still specialist pieces of equipment.

NB - our emphasis for 'quantum computing' here is on the 'computing' rather than the 'quantum' part... so we'll keep this bit brief :P

So what's the big deal?

The fundamental deal is that quantum technologies (usually, but not exclusively) make use of two quantum effects:

Superposition - the idea that a particle is in a combination of states simultaneously.
Entanglement - the idea that two particles can be, in some strict mathematical sense, inseparable to the point that a measurement of one gives you some facts about the other (without measuring it first).

This leads us to the next question - how do you actually do this?

Well, first up, let's discuss the structure of a qubit in comparison to a regular 'bit'. A bit is just a one or a zero - and it is always exactly one OR the other, NEVER both. A qubit, however, can be in some superposition of the 'zero state' and the 'one state' - when you measure it, you will always get just a '1' or a '0' as your output, just like a bit. The difference is that it can be either 0 or 1, with some probability of being one or the other across many measurements.

To understand this better - take a look at the following diagram:

This is called the 'Bloch Sphere' - and it represents the 'sphere' of possibilities for the position of a qubit's state, which we usually write as |ψ⟩.

Now, we do have to do a little maths - as we can't just have any old values wandering around - they won't stay on the surface of the sphere! So, we need to define what |ψ⟩ is - let x and y be complex numbers (that is, of the form m+in where i is the imaginary constant, and m and n are real numbers, aka 'floats'), then we say that |ψ⟩ = a|0⟩ + b|1⟩, letting |0⟩ be the column vector (1,0), and |1⟩ be the column vector (0,1). We only require that |a|^2 + |b|^2 = 1 (to preserve the probabilities of the system and keep the tip of the state vector |ψ⟩ on the sphere surface).

I'm not going to go into things like unitary gates, the 4 postulates of quantum mechanics, linear algebra, inner/outer products, tensor products, seperable states, etc. etc. - if you would like a reasonable primer, look at these slides.

Before we go on - here's an emergency post-mathematics kitten:

Building things for multiple qubits...

Now, this was actually important for the next piece of the puzzle - how you program a quantum computer! Because everything is now reducible to two-place column vectors that have nice properties - we can now manipulate them easily with 2x2 matrices! This is exactly what forms quantum gates.

But what do we mean by 'quantum gate'? Well, classical computers have logic gates; AND, OR, NOT, NOR, NAND, XOR, etc. which are placed in various combinations to make computer programs. All of our computation is fundamentally reducible to these gates.

In the same way, a quantum algorithm is formed from the composition of quantum gates, these 2x2 matrices, in sequences we call 'quantum circuits' (analogous to 'boolean circuits' for those familiar with them).

Our first Quantum Circuit!

What do these quantum circuits look like? Well, we can see an example here:

We assume that on the far left, all qubits are set to |0⟩ (with the vector pointing up on the sphere). Now we can apply gates - and some of these gates have special names;

the circular blue gate with an '+' in the middle is called the 'Pauli X' - this flips the gate from |0⟩ to |1⟩ or a similar inversion if the qubit vector |ψ⟩ isn't wholly up or down.
The red gate with a 'H' is called the Hadamard Gate, and this serves to put the qubit into superposition - more on this later!
The circular blue '+' with a tie to an upper qubit is called a CNOT gate - this is a very cool gate that is involved with entanglement and other cool operations on a quantum computer, but which we'll skip over here.
The black box with a vertical line is the 'measurement' gate.

For the ultimate output, the classical bit 'buffer' is the lowest line in these diagrams - and when we measure qubits we get either a 0 or a 1, and these are placed into the buffer sequentially.

So how do we get an output? Well, the quantum computer will run our circuit more than once, and give us a graph of the outputs - so when I ran the above I got the following output graph:

You will notice that we 'almost always' got a '11' (which if you check the matrices is what we should have gotten) but there is just over 13% of the outputs that were the other three possible states for 2 qubits. This inherent noise is why quantum computers such as these that don't do quantum error correction are called 'Noisy Intermediate Scale Quantum computers' or NISQs.

NB - There is also a Quantum Assembly language called QASM! For the above circuit, it looks like this:

OPENQASM 2.0;
include "qelib1.inc";

qreg q[2];
creg c[2];

x q[1];
h q[0];
h q[1];
cx q[0],q[1];
h q[0];
h q[1];
measure q[0] -> c[0];
measure q[1] -> c[1];

It is also worth mentioning that the above circuit is not what is actually run - the commercial quantum computers implement a reduced set of gates that are equivalent under combinations to every theoretical gate. This transpiling is very common - for the above circuit, the transpiled circuit looks like this:

Quick aside - resources

I'm skipping over MANY details here, so if you want to learn more have a look at these resources:

Qiskit Documentation - This has many excellent pages covering the basics of quantum gates, but also in-line summaries of the matrix and vector stuff in case that is a little rusty. A really good resource!
Quantiki - a really good website with lots of details and descriptions of various quantum algorithms and their uses!

Building Something Useful

You'll note that I didn't explain what was really going on in the circuit above - it's not that important (it is a circuit that is equivalent to an inverted CNOT, part of the Bernstein-Vazarani algorithm, for the curious). But the next bit will require us to go into some depth - we're going to show how to acquire some of the best quantum-ly random bits from a quantum computer! We can then use these to fold into a local entropy pool for generating random numbers.

Quantum Supremacy and Randomness

When Google announced quantum supremacy what they had achieved was the measurement of randomness in a few hundred seconds that would take a supercomputer 10,000 years to perform the same fidelity of operation. This is a similar process that companies such as Cambridge Quantum Computing use (we presume, anyway) to measure the randomness of a source and help it be 'better' randomness.

But who cares about this?

Who would find this useful? Well - consider the following clients and use cases:

Anyone generating keys or doing cryptography will always need good entropy seeds, and quantum is one of the best sources.
Anyone doing financial simulations and modelling (such as Monte Carlo sims)
Anyone doing AI/ML model generation will need plenty of good quality randomness.
Gaming applications - obviously, when you roll a die in a game, you don't want it being predictable! The gaming industry has very strict requirements on randomness for its purposes.

Our Grand Design

We are not going to do anything so fancy here - we will, however, use the following algorithm:

Put a sequence of qubits into superposition with a Hadamard gate
- This means each qubit has a balanced probability of 1/2 of going into state 0 or 1 on output, which is our source of randomness.
Take this output, and blend it with our local randomness entropy pool.
Use this pool to seed a CSPRNG (Cryptographically Secure Pseudo-Random Number Generator) that can generate random numbers very quickly for general use.

The rough block diagram is here:

So we will setup a background job to use the IBM-Q python API in qiskit, and whilst we wait, we will generate random numbers on the fly for as long/as many as we need to provide.

So what does our circuit look like? Well, something like this:

Now, we note the following calculation:

The 15 maximum qubits on the ibmq_16_melbourne quantum computer means that we can run for 15 random bits of output for each shot.
The max number of shots is 8,192 (2^14).
This means that we can get up to 15*8192 = 122,880 bits in the output!
These aren't secret, as IBM can also know these bits, so we will blend them with local randomness that we can assume is known only to us for this use case.
- It is important that any input into the RNG is not widely known, else we may compromise our random numbers!
- For this, we will use SHA3 (Keccak) hashing to keep things as high-entropy as possible. This is a PQC algorithm with a large internal state, so this should maintain a good level of security.

And this is the core of our service! 😁

But what is there to hack?

Well, so far - this just shows how to generate random numbers locally from a class with the background process occasionally asking IBM-Q very nicely! (NB - you'll need an IBM-Q account and API token for the script, but these are free!!)

What can we build? Well, consider the following:

MQTT randomness source - there are projects such as Entropy Broker that allow you to 'import' entropy from various sources. We can maybe write these to support MQTT and then distribute randomness across a network (with appropriate levels of security, ofc!) to better seed local entropy pools.
A Randomness API! - We could build a python API that would allow us to provide high-quality randomness derived from our beautiful quantum-ly random bits to anyone who asks, in bulk, and at scale!

Or anything else you can think of to get good quality randomness to those who need it!

Show us the money!

For those who want to see a PoC flask-based API service (only two endpoints, but it should be quite illustrative) then have a look at the following github repo:

Santandersecurityresearch / QuantumRNG

A Quantum computer based CSPRNG, written in python, as a PoC for using QCs in services.

But if you want an all-in-one script to play with, we have that, too! Now I have discussed the base theory - here is the proof-of-concept script!

Special thanks to Dr. Joe Wilson (@jmaw88) for his help in proofreading this post. :)

Making your first Open Source contribution

Ana Enríquez — Mon, 19 Oct 2020 13:14:13 +0000

What is Open Source Software?

According to opensource.com Open source software is software with source code that anyone can inspect, modify, and enhance.

"Source code" is the part of the software that most computer users don't ever see; it's the code computer programmers can manipulate to change how a piece of software—a "program" or "application"—works. Programmers who have access to a computer program's source code can improve that program by adding features to it or fixing parts that don't always work correctly.

Why should I contribute to Open Source?

There are hundreds of reasons to contribute to Open Source projects. Here are a few of them:

Show yourself that contribute is not as intimidating as it seemed
Win a battle to the impostor's syndrome
Return to the Community and/or the project by helping to its development
Grow as a professional by adopting new code styles and working with different teams and architectures.
Get out of the comfort zone

There are more, those were just a few that I always think about :)

Can I contribute to Open Source?

One of the biggest challenges when it comes to contributing to open source is ourselves.

At first, we have the feeling that only senior programmers recognized in their field can contribute to Open Source. And often a mixture of fear and shame keeps us from taking the step of contributing.

But the truth is that Open Source needs the help of all kinds. From syntactic corrections, generating documentation, creating a new feature, there are tons of ways to help and contribute. In this post, I describe step by step the last contribution I made as an example of a first issue to start with.

Looking for a first good issue

Starting something new is always hard. You probably have a lot of doubts and probably you don't know where to start.
Gratefully, Github got you covered.

For every repository, Github offers good-first-issue as a default label. You can filter issues by the label to find simpler contributions. Search by a topic or by a specific project and apply the label filter.

You can also use github.com/<owner>/<repo>/contribute and look for available tasks for a specific project.

There are also some websites that group projects with good first issues.

firsttimersonly.com - A guide with links to help you contribute
codetriage.com
up-for-grabs.net

In this post, we are going to contribute to one of these webs with a simple issue, fix some broken imagen links from the avatar of the Open Source projects firstcontributions.github.io.

Preparing to start

Before you start working on the issue you have chosen it is important to review the repository and check if there are guidelines about how to contribute and any code standards. Usually, this information is in CONTRIBUTING.md.

First step: Fork the project

As you are not (yet) a contributor, and as a good practice, you should fork the project to have a space to work on it. If you fork it into an organization instead of your profile, you would grant access to the issue to the rest of members of the org.
Once you have forked, you can clone it into your computer. For this example:

git clone https://github.com/this-is-you/first-contributions.git

Go to the new directory and then you should add the original repository as an upstream remote (so that you can git pull if new changes happen while you are working on the issue).

git remote add upstream https://github.com/firstcontributions/firstcontributions.github.io.git

Create a new branch

It is important to create a separate branch to work on the issue. Also, remember to add a prefix depending on the type of work you are doing. I normally use bugfix/ feature/ or hotfix/. For this issue, I named my branch bugfix/fix-images-open-source-list

Working on the issue

Normally, the Issues will give you information about the problem and what it needs to be fixed. This information could be more or less specific.

In our example, the Issue is very specific and the reporter told us the file where the changes needed to be made, but if this is not the case, my advice is to first run the project in our local and inspect the code and the behaviour of the project until we found the piece of code to edit or the piece of functionality to modify/create.

Even in this example, I ran the app in my local to find out the logos that were broken and to test if my changes solve the issue.

Committing the changes

Once you think he issue has been resolved you can commit the changes and send them to your fork to the new branch you have created. Remember to fit the contributing guidelines of the repo that you are trying to contribute to.

Creating the pull request

Once you have uploaded your changes to your fork you will be able to create a pull request through the web browser or the console. Trough the web browser you would see something like this when looking at your fork:

If you click on the compare and pull button you can start creating the pull request with the details of your solution.

Make sure that you select the branch to compare and that the changes are being reflected. Look at the arrow to know exactly the process of the pull request.

Waiting for review

You are almost done! Now you need to wait until a reviewer (codeowner) can take a look at your pull request and discuss with you the solution.

If the PR is not accepted

Sometimes the reviewers need more info or context to be able to evaluate all the code changes. Don't be afraid to talk and discuss the solution.

The reviewer/s normally would give feedback about why the pull request is not ready or why they had to close it. Talk to them and continue working to improve the solution.

Congratulations! You have just made your first pull request!!

DEV Community: SanExperts

Artificial Neural Networks: types, uses, and how they work

Table of Contents

Introductory Concepts

Artificial Neural Networks

Convolutional Neural Networks

Recurrent Neural Networks

Spiking Neural Networks

Final Thoughts

Getting started with FX: Powerful and handy JSON manipulation from the command line

Why this post?

fx

How to install

fx modes

What can I do with fx

Quick JSON exploration

Transform

Filter

Use your favorite npm module along fx

Explore more possibilities

Postman quick tricks

Scenario #1: Rename all the items of a collection adding a prefix sequence index

Scenario #2: Make requests with fake data

Scenario #3: Check JWT claims with Javascript within Postman

Scenario #4: Signing JWT tokens directly within Postman

Load jsrsasign from external URL

Load jsrsasign locally

Trial #1: Read the library from a local file

Trial #2: Serve the library from localhost

Introduction to Deep Learning, basic ANN principles

Table Of Contents

What is deep Learning ?

Artificial Neurons

Activation Functions

Artificial Neural Networks (ANN)

Learning process of an ANN

Review of the whole trainining/learning process

Final Thoughts

How to create an API generator

QuantumRNG-aaS - Making use of Quantum Algorithms

Primer on Quantum Algorithms

So what isn't a quantum computer?

So what's the big deal?

Building things for multiple qubits...

Our first Quantum Circuit!

Quick aside - resources

Building Something Useful

Quantum Supremacy and Randomness

But who cares about this?

Our Grand Design

But what is there to hack?

Show us the money!

Santandersecurityresearch / QuantumRNG

A Quantum computer based CSPRNG, written in python, as a PoC for using QCs in services.

Making your first Open Source contribution

What is Open Source Software?

Why should I contribute to Open Source?

Can I contribute to Open Source?

Looking for a first good issue

Preparing to start

First step: Fork the project

Create a new branch

Working on the issue

Committing the changes

Creating the pull request

Waiting for review

If the PR is not accepted