DEV Community: asaf g

Long pollng with nodejs

asaf g — Sun, 01 Aug 2021 14:51:59 +0000

Long polling is just like short polling, but with a bigger pole! No, not really!

Sometimes, we need to get updates from a server, but we don't know if the resource that we're after is there at the moment.

Examples for that can be:

Checking if a file exists on a server
Waiting for some field to be updated
Checking for new messages on a chat app I bet you can find more similar scenarios.

So, what are your options today in our http driven world?

Short polling
Using something like Server Sent Events (SSE)
Websockets (which is actually tcp)
Long polling

Of course if you don't limit yourself to http, you can use GRPC, a PUB/SUB server or roll your own with tcp.

So what is long polling anyway?

Long polling is a technique to achieve a continuous connection between a server and a client.

The server:

Receives a request
Checks if it has data to return to the client
If it has data, it returns it in the response
If it doesn't have data, it waits and checks if the data is available repeatedly (polling)
After a certain time has passed the server "gives up" and returns an empty response

The client:

Send the requests
Waits
Receives the response
Checks for data (and if there is, does something with it)
Repeats

How is long polling better than just polling?

"Just polling" or short polling can only get results at the interval it polls.
So, in short polling this is how it goes:

The client sends a request
The server immediately return a response
The client checks for data (and do something if it's available)
The client sleeps an "interval" and repeats

The connection here is not continuous. If our interval is 5 seconds, it means we actually get results only every 5 seconds.
Our result in this case can be 4.99 seconds old.
With long polling on the other hand, we get the result almost immediately. I say almost, because there's the time between getting the response and sending another request.
Let's assume that, that time is 100 milliseconds plus 200 for latency. In the worst case your result can be 300 milliseconds old. This is for sure a lot more "real time"-ish.

Alright! I assume that you are now convinced. Let's move on to the implementation.

The client

This is the easiest part, just loop and make the requests.

client.js


const axios = require('axios').default;

async function main() {

    while (true) {
        var hadErr = false;
        console.log("requesting...");
        await axios.get('http://127.0.0.1:8080', { timeout: 10000 })
            .then(function (response) {
                console.log(response.data); // This will sometime be empty
            })
            .catch(function (error) {
                console.log('People we have an error!', error);
                hadErr = true;
            });
        if (hadErr) {
            // break out of the loop in case of error
            // maybe in a real live situation we could do something here*
            break;
        }
    }
}

main();

This code is pretty simple, loop forever and print whatever you got.

Note that, the timeout is 10 seconds which should be more time than the time the server will wait (server returns after 5 seconds if there's no data).

The server

Now, this is where the magic happens.

Let's start with our "resource". I have implemented a very generic "one time" event emitter.
The calling code can register to the "event", and when the event happens it is automatically unregistered.
You'll see how I'm using it in a sec.

event.js



// A very generic event emitter
class EventEmitter {

    listeners = {};

    fire(event) {
        for (var k in this.listeners) {
            let listener = this.listeners[k];
            this.unregister(k); // unregister this listener
            listener(event);
        }
    }

    register(id, listener) {
        this.listeners[id] = listener;
        console.log("Register", id)
    }

    unregister(id) {
        return delete this.listeners[id];
    }
}

module.exports = EventEmitter;

This is pretty standard. You can check out another implementation of it here.

I'm using express.js for the server, it's easy enough and take cares of most of the stuff. Let's start with a simplified version of our server and continue working on from there.

index.js


const express = require('express');
const EventEmitter = require('./event.js')
const app = express();

// create an instance of our event emitter
const eventEmitter = new EventEmitter();

app.get('/', function (req, res) {
    res.status(200);
    res.end();
})

 var server = app.listen(8080, function () {
    var host = server.address().address
    var port = server.address().port
    console.log("Example app listening at http://%s:%s", host, port);
 })

Ok, so this is pretty standard also. Taken from some express.js example or something. What it does is basically return a 200 OK header and closes the request.

Let's add code for our event emitter to actually emit events.


async function sleep(ms) {
   return new Promise((resolve) => {
     setTimeout(resolve, ms);
   }).catch(function() {});
 }   


async function main() {
   while (true) {
      const waitTimeMS = Math.floor(Math.random() * 10000);
      await sleep(waitTimeMS);
      eventEmitter.fire({time: waitTimeMS});
   }
 }

The main function is looping forever, sleeping a random amount of milliseconds (0 - 10000) and calling fire with an object.
Great, every few seconds an event will be fired. Let's see how we use that.

...

app.get('/', function (req, res) {
   const id = Date.now().toString(); // milliseconds of now will be fine for our case
   const handler = function(event) {
      console.log('event', event);
      res.status(201);
      res.end( JSON.stringify(event));
   };
   eventEmitter.register(id, handler);

...

Here, we register to the event with the handler function. When the loop in the main function will call the fire function, this function will run and output the event to the client along with 201 status code.
But, what if there isn't an event? Let's add the part that handles the returning of empty data to the client.

...

app.get('/', function (req, res) {
   const id = Date.now().toString(); // milliseconds of now will be fine for our case
   var timer = null;
   const handler = function(event) {
      clearTimeout(timer);
      console.log('event', event);
      res.status(201);
      res.end( JSON.stringify(event));
   };

   eventEmitter.register(id, handler);
   timer = setTimeout(function(){ 
      console.log('timeout');
      const wasUnregistered = eventEmitter.unregister(id);
      console.log("wasUnregistered", wasUnregistered);
      if (wasUnregistered){
         res.status(200);
         res.end();
      }
   }, 5000);
});

...

We added a timer in the start of the function and we are also clearing it in case the event fires.
In case the event didn't fire, we unregister the handler function from the event.
The unregister function returns a boolean, if it returns false the function was already unregistered! So, we write the empty data response only if the event didn't fire.
This is to avoid a race condition between the handler function and the timeout function.

What situation remains unhandled?

You might have noticed, that the event can fire exactly in the time that the client is getting the response and before it makes another request.
The code doesn't handle that and there's a few possible solutions that we can implement.
I can think of a few examples, can you?
One example would be to run two sets of requests and another would be to hold off and "retry" the event if there's no listeners.

I'm not going to implement any of those, although they sound pretty cool.

I would just love it, if you make a pull request on the github repo

Hope you enjoyed reading this post.

And of course if you did, please share and tell your friends!

Common Pitfalls Developing a Chatbot With Rasa and How to Overcome Them

asaf g — Fri, 30 Oct 2020 15:15:26 +0000

Written by Rashit Bedi for https://www.turtle-techies.com

Introduction

Rasa is Open source conversational AI apparatus.

NLU in Rasa represents Natural Language Understanding.
Befuddled among NLP and NLU?
Well NLP represents Natural Language Processing while NLU is short form Natural Language Understanding.
Both offer a shared objective of understanding the collaboration between spoken language (e.g English) and the language of the computer.
Rasa NLU is Natural Language Processing framework for aim characterization, named retrieving responses, element extraction and some more.
It is exceptionally used for - distinguish sentiments, conversational chatbots, Named-Entity Recognition and recognizing the reason for a sentence (intent).

Objective

In this article, we are going to talk about common issues while developing a bot with RASA framework.
We will also discuss ways to overcome these by functionalities provided by RASA and few other tricks applicable across all bot development frameworks.

Common Issues

Generating NLU Data

NLU (Natural Language Understanding) comes bundled with the Rasa framework that performs intent identification, entity extraction, and response retrieval.

NLU will take in a user query such as "Please transfer 500 $ to Mike" and return structured data like:

{
  "intent": "fund_transfer",
  "entities": {
    "amount": "500",
    "beneficiary": "Mike"
  }
}

The major pain factor for any problem that we choose to solve with ML/AI is data and NLU problems are no different.
Building NLU models is tough, and building ones that are production-grade is even tougher.
Below are some useful tips for designing your RASA NLU training data to get the most out of your chatbot.

Conversation-Driven Development for NLU

Conversation-Driven Development (CDD) means letting real world user conversations guide your bot building process.
This process will help your bot understand conversations like common slangs, synonyms or abbreviations that are hard for a bot developer to remember and incorporate in training data.

Capture Real Data

With regards to working out RASA NLU training data since it requires a lot of data to get trained efficiently, bot builders frequently depend on text generation tools and programs to rapidly build the number of right training examples for preparing models.
These tools are great for building bulk training data.
While this methodology can be efficient, it makes NLU models inclined to overfit information that isn't illustrative of things a genuine client would state.
The variance in any data set is key for a well-trained NLU model.

To dodge such an issue, it is a smart thought to gather consistently however much genuine client interactions or conversations as could be expected to use as training data.
Despite the fact that your bot will commit errors at first, the way toward preparing and assessing on client interactions will empower your model to sum up substantially more successfully in real world situations.

Did you know?

Chatette is a Python program that generates training datasets for Rasa NLU given template files. If you want to make
large datasets of example data for Natural Language Understanding tasks without too much of a headache, Chatette is a project for you.

Specifically, Chatette implements a Domain Specific Language (DSL) that allows you to define templates to generate a large number of sentences, which are then saved in the input format(s) of Rasa NLU.

Share with close group users Early

So as to accumulate genuine training data, you will require genuine client messages.
A bot designer can just concoct a restricted range of examples, and users will consistently astonish you with what they state.
This implies you should impart your bot to test users outside the advancement group as ahead of schedule as could be expected under the circumstances.
See the full CDD guidelines for additional subtleties.

Avoiding Intent Mismatch

Intents are characterized utilizing character and word-level features generated from your training examples, contingent upon what featurizers you've added to your NLU pipeline.
At the point when various intents contain similar words requested along these lines, this can make disarray for the intent classifier.

Making smart use of Entities

Intent mismatch or confusion often occurs when you want your assistant's response to be conditioned on information provided by the user, the user query will match to more than one intents with high confidence.
For example, "How do I block my debit card?" versus "I want to block my credit card."

Since each of these user messages will lead to a different response from the bot, your initial approach might be to create separate RASA intents for each card type.

e.g. credit_card_block and debit_card_block.
However, these intents are trying to achieve the same goal (blocking the card) and will likely be phrased similarly, which may cause the model to confuse these intents.

To avoid intent mismatch, group these training examples into a single card block intent and make the response depend on the value of a categorical card_type slot that comes from an entity.
This also makes it easy to handle the case when no entity is provided, e.g. "How do I block my card?" For example:



stories:
- story: block my debit card
  steps:
    - intent: block_card
      entities:
      - card_type
    - slot_was_set:
      - card_type: debit
    - action: utter_debit_card_block

- story: block my credit card
  steps:
    - intent: block_card
      entities:
      - card_type
    - slot_was_set:
      - card_type: credit
    - action: utter_credit_card_block

- story: block my card
  steps:
    - intent: block_card
    - action: utter_ask_card_type

Extracting entities

Common entities such as names, addresses, and cities require a large amount of training data for an NLU model to generalize effectively.

RASA provides many options for entity extractions, it gets quite difficult to understand which method to use when.
Let’s drill down on a few of them.

RASA framework provides two great options for pre-trained extraction models: first is the DucklingEntityExtractor and SpacyEntityExtractor.
Because these extraction models have been pre-trained on a very large corpus of data, you can use them to extract the entities they support without the need of annotating them in your training data.
Apart from these pre-trained extractions, RASA provides tools for extraction of entities.

First up, Regexes!

A RegEx, or Regular Expression, is a sequence of characters that forms a search pattern (but I assume you already know that).
RASA provides provision to set regexes against any entity.
Regexes are useful for performing entity extraction on structured patterns such as 11-digit bank account numbers.
Regex patterns can be used to generate features for the NLU model to learn, or as a method of direct entity matching.

To set this up follow the below example:

Data look like this in your nlu.md file


## intent:inform
- [AB-123](customer_id)
- [BB-321](customer_id))
- [12345](transaction_id)
- [23232](transaction_id)
## regex:customer_id
- \b[A-Z]{2}-\d{3}\b
## regex:account_number
- \b\d{11}\b

And the config file


language: "en"
pipeline:
- name: "SpacyNLP"
- name: "SpacyTokenizer"
- name: "SpacyFeaturizer"
- name: "RegexFeaturizer"
- name: "CRFEntityExtractor"
- name: "EntitySynonymMapper"
- name: "SklearnIntent

Second, Lookup Tables!

You can think of look up tables as databases of information, which you can set against any entity for matching.
Lookup tables are processed as a regex pattern that checks if any of the lookup table entries exist in the training example.
Similar to regexes, lookup tables can be used to provide features to the model to improve entity recognition, or used to perform match-based entity recognition.
Examples of where you can use lookup tables are types of Apple IPhone models, a la carte menu options or ice cream flavours.

Lastly, Synonyms!

Adding synonyms or similar words to your RASA NLU training data is helpful for mapping certain user query words to a solitary standardized entity.
Synonyms, however, are not meant for improving your model's entity recognition and have no effect on NLU performance.

A good use case for synonyms is when normalizing entities belonging to distinct groups.
For example, a virtual assistant that asks users what type of loan they're interested in, they might respond with "house loan," "I need home loan," or "housing." It would be a good idea to map house, home, and housing to the normalized value house so that the processing logic will only need to account for a narrow set of possibilities.

Take care of Misspellings!!

Running over incorrect spellings is inescapable, so your bot needs a successful method to deal with this.

Remember that the objective isn't to address incorrect spellings, however, to accurately distinguish intents and entities.

Consequently, while a spellchecker may appear to be an apt arrangement, changing your featurizers and training data regularly are adequate to represent incorrect spellings.

Adding a character-level featurizer gives a viable protection against spelling mistakes by representing portions of words, rather than just entire words.

You can add character level featurization to your pipeline by utilizing the char_wb analyzer for the CountVectorsFeaturizer

Notwithstanding character-level featurization, you can add basic incorrect spellings to your training data.

Conclusion

Apart from the above points, generally speaking if there’s any downside to RASA’s feature rich framework, it might be the steep learning curve with respect to how these features work in tandem.
As some other chatbot developers have noted, Rasa isn’t necessarily for beginners.
Unless you’ve built chatbots before using other chabot development frameworks, Rasa’s framework might strike you as a bit intimidating at first.

Having said that, there are some great tutorials and material available for the beginners to start working on this fabulous framework.

Most importantly it has amazing community support.

A diverse group of makers and conversational AI enthusiasts. Where people of different backgrounds and locations help each other to understand the framework and create better AI assistants.

Certificate Pathways to Cyber Security

asaf g — Fri, 09 Oct 2020 07:17:45 +0000

Written by Nipuna Dilhara

If you are reading this article it means you are already interested in cybersecurity.
That’s the first symptom of becoming obsessed with this fascinating IT discipline.
You might have heard about all these cyber attacks, data breaches, system vulnerabilities at least a few times a week.
If you are an IT professional already, you might be aware of how fast the cybersecurity industry is booming and governments and companies are struggling to find the right talent and keep up with the growing demand.
It doesn’t matter if you are either already into IT or a total beginner.
If you want to uplift your life for the better, then you have come to the right place.

Is Cyber Security Really in Demand?

If you are having any doubts, first check the below image provided by cybserseek, which shows cybersecurity job openings in the USA.

I couldn't recall another career path that gathered the attention of this degree in recent years as cybersecurity.
According to the US Bureau of Labor Statistics, the expected job growth for information security professionals is around 31% between 2019 to 2029 which is much faster than other IT-related occupations.
Most of the recent researches say that the demand for skilled cybersecurity professionals won’t go down in the foreseeable future.

When thinking about the current IT industry, those statistics don’t surprise me.

There aren’t enough people with the required analytical and innovative skills to tackle the growing threats in the digital world.

So from where all these opportunities are coming from? What cybersecurity jobs have more demand and changes than others?

Let’s have a look at the most in-demand cybersecurity career pathways in brief.

reference: https://www.cyberdegrees.org/jobs/

Full Disclosure: this post might have some affiliate links.

Career Pathways in Cyber Security

Information Security Analyst

If we put it in simple words, the main responsibility of a Security Analyst is to protect an organization's data from malicious attacks.

An Information Security Analyst monitors the system for potential security breaches.
Apart from that, the main job duties might include:

Developing organization security policies
Informing and training employees regarding security measurements, new applications, and hardware
Conducting penetration testings
Installing software tools to secure the environment, etc.

It doesn’t sound like an easy job. So it’s no wonder why Security Analysts are in such high demand.

Ethical Hacker

Becoming an Ethical Hacker or White Hat Hacker might be one of the best things that can happen to your life.
It’s such an interesting job but not meant for everyone.
Ethical hackers are certified security professionals who are authorized by the employer or a hiring organization to penetrate their system and find existing and potential system vulnerabilities.
They use the same techniques as Black Hat Hackers do, but for a better purpose.
Responsibilities of an Ethical Hacker don’t end from finding vulnerabilities.
In most cases, they have to find a way to patch the exposed vulnerability and prevent malicious attackers from using them.

Penetration Tester

Penetration testing is a subset of the broad responsibilities of an Ethical Hacker.
However, I’m listing it separately since it’s one of the most demanded jobs currently in the world.
Organizations are eager to find Penetration Testers with correct skills.
The responsibility of Penetration Testers is mainly to identify system vulnerabilities.
But Penetration Testers are focused on a specific area defined for testing whereas Ethical Hackers work on a wider range.
Penetration is one of the functions of Ethical Hackers that have to be carried out while searching for vulnerabilities.
And yet Penetration Testers are still in high demand and being recruited separately from Ethical Hackers.

Secure Software Developer

The Secure Software Development is for those who are passionate about programming as well as the security aspects of it.
Secure Software Developers have to focus on developing secure applications by integrating security best practices and protocols.
Becoming a Secure Software Developer, you have to understand every aspect of software development based on the Software Development Life Cycle (SDLC).
They should have specialized knowledge of software design, software security, computer systems and networks, and programming.
The knowledge on software-based security strategies categorizes Secure Software Developers apart from traditional programmers.

Digital Forensic Analyst

Digital Forensic Analysts specially trained to retrieve digital evidence from various digital assets used in cybercrimes.
In some cases, you will see they work with law enforcement agencies to expand investigations into the digital world.
Their role is to provide consultant and technical support whenever necessary.
Within an organization, Digital Forensics is involved with securing sensitive data, educating employees about cybersecurity issues, identifying data breaches and weaknesses, and many more.

Digital Forensic Analysts are often hired by government agencies, corporations, law firms, etc.
If you are interested in stuff like investigating crimes, following digital fingerprints then this might be for you.
You will never know unless you give a try.

Security Architect

Security Architects design, plan and supervise systems and networks that will prevent security threats.
They identify the strengths and weaknesses of their organization’s systems and networks and take necessary measurements accordingly.
It’s an exciting job role which often needs to be researching new technologies to make things more secure and better in performance.
Security Architects possess an in-depth knowledge of software and hardware design, computer networks, computer programming, and risk management.
A well experienced Security Architect often has strong analytical and problem-solving skills which highlight them apart from other cybersecurity professionals.

How to get in?

Well, it wasn’t easy to isolate and list down the career and responsibilities of this field with vast variety.
But we did our research and identified positions which might help you most.
Apart from what we have mentioned here, there are several other positions in the industry which are coming with a handsome salary.
Also, you might see in other places that the aforementioned job roles and responsibilities have been mentioned under different job titles.

Cybersecurity is a vast field with a variety of specializations.
Many governments are trying to gather these specializations under one umbrella called Cybersecurity.
So there can be some mismatches between how each job role is titled by two people.
But we suppose that you already have the picture to get a basic idea.

So how can we get into this booming industry?

Indeed, having an undergraduate or postgraduate degree in Cybersecurity or a relevant area will put you one step ahead.
What about others? Everyone deserves to be a part of this if he or she has the interest and keen to learn.
We are living in a generation where our life is not getting decided by just a university degree.
Companies like to hire higher candidates with the correct technical skills, mindset, and the potential to achieve greater things.

But how do we prove them we are the people they were looking for?
How do we prove we have the knowledge and qualifications to deliver value to their organization?

That’s when cybersecurity professional certificates are coming into the play.

Cybersecurity Professional Certificates

Unlike other IT disciplines, Cybersecurity has a well known set of professional certificates for anyone with the hunger to learn more and achieve more.
You may not have a 4 years university degree, but you have been given the opportunity to develop your career via these certificates.

It’s up to you how far you wish to go and in which direction.
But if you have the characteristics that I was mentioning, then following certificates will give you a fair chance to land on a job in cybersecurity.

CompTIA Linux+
CompTIA Network+
CompTIA Security+
Certified Ethical Hacker (ECH)
System Security Certified Practitioner (SSCP)
Offensive Security Certified Professional (OSCP)

Let’s have a deeper look at each and every one of the above.

CompTIA Linux+

If you are at the beginner level, completing the Linux+ will come in handy for sure.

Kali Linux is known to be the most loved Linux distro among Cybersecurity professionals.
We suppose that it will remain the same in the foreseeable future as well.

The Linux+ has been designed to provide you with the fundamental but essential skills to work in the Linux environment.
The certification will test your abilities on Linux commands, maintenance tasks, installing and configuring workstations, and networking.

It’s guaranteed that this certificate will provide you with a confidence boost and more importantly a kick start to the career in Cybersecurity.

Linux+ Udemy course

https://www.comptia.org/certifications/linux

CompTIA Network+

CompTIA Network+ is a well known entry-level certificate provided by CompTIA.

It will provide you with an in-depth understanding of:

Networking technologies and devices
Network security practices
Network security policies
Network security threats
Defense mechanisms
Damage recovery

and other relevant knowledge areas regarding both wired and wireless networks.
Networking is an essential aspect to gain the required skills which play a vital role in information security and policies.
No company would like to make headlines for a critical data breach.
It will tear apart their reputation to a state that cannot be rebuilt.
So companies tend to invest more and more in ensuring their data and information security in each passing day.
Due to these reasons Network+ has become one of the major certificates that any employer considers during an interview.

TOTAL: CompTIA Network+ Certification

https://www.comptia.org/certifications/network

CompTIA Security+

The CompTIA Security+ is another great entry-level certificate and according to most professionals, it’s the best certificate to be followed at this level.
The Security+ is a vendor-neutral qualification that has been approved by the US Department of Defence and employees around the globe.

This certificate has been designed to cover a wide range of security-related skills, use case simulations, and technical knowledge.
You have to gain knowledge on a variety of security threats, vulnerabilities, security tools, risk management, secure application development, and many relevant areas.

Once you have the Security+ under your belt, it will open the doorway to more advanced careers such as:

Network Security Engineer
Network Security Administrator
Security Analyst
Senior Technical Support Engineer

It’s totally up to you how to unitize the broad knowledge gathered for this certificate program.
You will be able to build your future career upon it.

It’s true that the CompTIA Security+ sounds complex and will need a lot more training to gain the required knowledge and experience in these areas. But the effort you put to complete the certificate will be worth it for sure.

https://www.comptia.org/certifications/security

Comptia Security+ on Udemy

CEH: Certified Ethical Hacker

Here comes some interesting stuff. Hacking!

The CEH certificate is mainly designed for those who are interested in hacking.
It’s the most well-known certificate that will clear the path of becoming an ethical hacker.
Even though doing ethical or unethical stuff is totally up to you, this certificate will provide you with the foundation knowledge of the world of hacking.

Hacking is a double-edged sword.
It can be used for both good and bad purposes.
The CEH certificate covers the knowledge required to become a White Hat hacker.
A White Hat hacker uses the same tools and techniques as a malicious attacker to identify vulnerabilities and enhance the organization's security.

You might already have found this certificate interesting.
But we don’t recommend you to jump straight into this.
The CEH is not considered as an entry-level certificate.
It’s more likely an intermediate certificate that allows security professionals to get into a more specific job category of ethical hackers or penetration testers.

The CEH is not an easy certificate to collect.
It covers a wide range of skills and knowledge on different attacking methodologies and prevention techniques relevant to:

Network scanning
Social engineering
DoS attacks
SQL injections
Session hijacking
Viruses and Trojans
Cryptography

And much more.

Once you become a certified ethical hacker, you will be able to work on the aforementioned cyber attacks, identifying vulnerabilities, plan and implement defense strategies for your organization.

Certified Ethical Hacker (CEH 10) A to Z Course

https://www.eccouncil.org/programs/certified-ethical-hacker-ceh/

System Security Certified Practitioner (SSCP)

The SSCP is another globally recognized certificate that is offered by (ISC)2.
As the name indicates, SSCP mainly focuses on technical aspects of system security.
In order to earn the SSCP, you might have to have practical knowledge of security over the theoretical part.

A few of the widely covered areas of this certificate are:

Access and management control
Risk analysis and identification
Incident response and recovery
Network and system security
Security operations and administration
Cryptography

If you are still at the beginner level, unfortunately, this might not be the certificate for you.
The SSCP is more of an intermediate certificate for those who are having the experience of a couple of years in the cyber securing or networking fields.

However, if you are already having a few years of experience and planning to move into positions such as:

Network security engineer
Network administrator
Security analyst
System engineer

and other similar roles then SSCP would be the ideal certificate to earn.
It will give you the required qualifications to follow your passion in one of the aforementioned areas.

SSCP Certification

https://www.isc2.org/Certifications/SSCP

Offensive Security Certified Professional (OSCP)

The OSCP is solely designed for those who are wishing to become penetration testers.
Becoming a penetration tester is a wonderful career choice which most cybersecurity professionals dream of having.
It’s the dream job of many cybersecurity professionals due to its fascinating job role.

The OSCP is provided by the Offensive Security organization who is the founder of beloved OS, Kali Linux.
So you know OSCP is worth considering.
Many employers consider the OSCP is the foundation certification for a pen testing job role.
It will cover topics such as:

Active and passive information gathering
Vulnerability scanning
Web application attacks
Client-side attacks
Buffer overflows
Privilege escalations And more.

The OSCP is not an easy certificate to achieve.
It includes heavy pen-testing practicals which need much more time and effort to practice than for other certificates.
Even though the certificate content is comparatively difficult, the end result will be highly rewarding.

https://www.offensive-security.com/pwk-oscp/

Is that All?

These are only a few of the important certificates which will guide you from the beginner level to the more advanced level.
The CompTIA has published its own pathway into cyber security which suits people at different levels of their careers.

The EC-Council, who issued the Certified Ethical Hacker (CEH) certificate has its own certification path to become ethical hackers, security analysts and penetration testers.

If you are really passionate about cyber security and looking for a way to get into it, this is the correct time than ever before.
You won’t need university degrees any more.
You have more than enough resources available at your doorstep.
There are plenty of opportunities currently available for those who are with the right mind set.

However, it won’t be not an easy path.
You have to work hard and spend most of your time achieving your passion.
After all, cyber security is for hardworking and passionate people.
You should be always ready to run the extra mile.

If you feel like cyber security is for you, don’t be afraid to follow these certificates and build your career step by step.
It will be hard for you at first.
You will feel like quitting.
However, if you love the process and keep pushing yourself, you will be able to land on your dream job in cyber security most certainly.

How to Write XML Changesets in Liquibase

asaf g — Mon, 21 Sep 2020 15:31:33 +0000

Written by Suresh Regmi
If you want to have a basic understanding of Liquibase, I would suggest you look into our previous article on database version control with liquibase https://www.turtle-techies.com/database-version-controler-with-liquibase/.

We are assuming that you have already installed liquibase in your system.
If you haven’t, please refer to this official documentation from liquibase https://www.liquibase.org/get-started/first-steps.

In this article, we will be discussing changelogs, changesets and some examples of how to write changesets in XML format.
The database I am referring to while writing changesets is Oracle.
You might need to change some of the values from code I provided according to the database you are using for example the driver detail, the connection parameter, data types etc.

Some background information

What is a changelog

A changelog is a collection of the changesets. Changelogs can be written in SQL, XML, JSON and YAML format.

Changelogs can include nested elements like preConditions, property, changeSet, include, context etc.

What is a changeset

Changeset in liquibase represents a single change to your database.
Each changeset is executed in a transaction and will be either committed or rolled back depending on whether there is an error or not.

Each changeset is uniquely identified by “ID”, “AUTHOR” and the directory and filename of the changelog file.

Let’s start with organizing your changelog files

Changelog files can be managed in two ways - either to create a single changelog file and add all of your changesets to that file, or create multiple files and split the changesets across them.

In my experience, the latter is more manageable and has a lot of advantages.

Segregating changesets based on database objects and release versions help to manage the changes in a more organized way.

If you already have a base database and you want to implement liquibase to that project, you can add changesets for initial tables, data and procedures in different folders and manage database updates based on release versions.
Managing changelog files based on release will help you to understand the changes that were part of the release and will be easier for other developers.

Working in an agile environment, here is an example of organizing changelogs that I use most and suggest others to follow.
I also prefer organizing changesets in multiple files and using the include tag in the database changelog of master changelog file to define the files and order of execution.

We can also define a master changelog file containing nested include elements to define the execution order of the files. An example of a master changelog file including other changeset files to run in order is given below.



<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
        xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.1.xsd">

    <include file="Initial-Changesets/Initial-Tables/Initial-Tables.xml" relativeToChangelogFile="true" />
    <include file="Initial-Changesets/Initial-Procedures/Initial-Procedures.xml" relativeToChangelogFile="true" />
    <include file="Initial-Changesets/Initial-Data/Initial-Data.xml" relativeToChangelogFile="true" />
    <include file="Updates/Release-1.1-Updates.xml" relativeToChangelogFile="true" />
    <include file="Updates/Release-1.2-Updates.xml" relativeToChangelogFile="true" />

</databaseChangeLog>

Here, I will explain to you different methods of writing changesets in liquibase with an example of two tables EMPLOYEE and DEPARTMENT.

1. How to create tables with liquibase XML

We can create one or more than one table in a single changeset.
We can define NOT NULL constraint, Primary Key and Foreign Key etc inside the createTable tag.
Constraints can also be defined outside the create table tag.
Here is an example of creating tables DEPARTMENT and EMPLOYEES in a single changeset.



<changeSet author="Author1" id="1">
 <createTable tableName="DEPARTMENT">
   <column name="ID" type="NUMBER">
     <constraints nullable="false" primaryKey="true"/>
   </column>
   <column name="DEPT_NAME" type="VARCHAR2(100 BYTE)">
   <constraints nullable="false"/>
   </column>
   <column name="DEPT_DESCRIPTION" type="VARCHAR2(1000 BYTE)">
   </column>
 </createTable>

 <createTable tableName="EMPLOYEES">
   <column name="ID" type="NUMBER">
     <constraints nullable="false" primaryKey="true"/>
   </column>
   <column name="NAME" type="VARCHAR2(100 BYTE)">
   </column>
   <column name="DEPARTMENT_ID" type="NUMBER">
     <constraints nullable="false" foreignKeyName="DEPARTMENT_ID_FK" references="DEPARTMENT(ID)"/>
   </column>
   <column name="DESIGNATION" type="VARCHAR2(250 BYTE)"/>
   <column name="SALARY" type="NUMBER"/>
 </createTable>

 <addNotNullConstraint columnName="NAME" tableName="EMPLOYEES"/>
</changeSet>

While creating changesets, I would suggest using a single database change in changeset so that if there is an error in one database change in a changeset, we don’t need to roll back the whole changesets.
For example, in the changeset given, if there is an error in EMPLOYEES table, Liquibase will fail after creating the DEPARTMENT table, so in the next execution after resolving the error it will throw an exception saying table DEPARTMENT has already been created. So, we need to manually rollback the changes that come before the one with the error.

2. Let’s insert some data using SQL file

Liquibase allows us to run the SQL stored in an external file.
This is valuable in the event that when the changesets are not supported by Liquibase, if there should be an occurrence of bulk insertions and furthermore for the situation where users incline toward utilizing SQL contents over the other Liquibase changelog alternatives, as XML, JSON, and YAML.
Here is how you can insert records in the table EMPLOYEES using an external SQL file.


<changeSet author="Author1" id="2">
 <sqlFile dbms="oracle"
          encoding="utf8"
          endDelimiter=";"
          path="Release1_SQL/insert_into_employees_file.sql"
          relativeToChangelogFile="true"
          splitStatements="true"
          stripComments="true"/>
</changeSet>

SQL File

---- DEPARTMENT
Insert into DEPARTMENT (ID,DEPT_NAME,DEPT_DESCRIPTION) values (1,'D1','D1 Department');

---- EMPLOYEES
Insert into EMPLOYEES (ID,NAME,DEPARTMENT_ID,DESIGNATION,SALARY) values (1,'John Doe',1,'Developer',50000);

3. How to write SQL statements using SQL tag

Liquibase also allows us to write SQL inside XML changeset to run whatever SQL you want.
It can be useful in the cases when complex changesets aren’t supported by liquibase automated changesets.
Below is an example of writing complex changes using the SQL tag.


<changeSet author="Author1" id="2">
 <sql dbms="oracle"
      endDelimiter=";"
      splitStatements="true"
      stripComments="true">
   MERGE INTO EMPLOYEES  dest
   USING  (SELECT 2   AS id, 'John Doe2'  AS Name,  1 AS DEPARTMENT_ID, 'Clerk'  AS DESIGNATION, 40000 AS SALARY  FROM   dual) src
   ON (dest.id = src.id )
   WHEN NOT matched THEN
   INSERT (dest.ID,dest.NAME,dest.DEPARTMENT_ID,dest.DESIGNATION,dest.SALARY) VALUES  (src.ID,src.NAME,src.DEPARTMENT_ID,src.DESIGNATION,src.SALARY) ;
 </sql>
</changeSet>

4. How to create Procedures and Triggers

Procedures and triggers can be created/run using the createProcedure tag in Liquibase or using custom SQL changesets.
If you want to use the CREATE OR REPLACE feature in Liquibase, runOnChange='true' can be set which will either create or update stored procedures while running.


<changeSet author="Author1" id="4" runOnChange="true">
   <createProcedure
           encoding="utf8"
           procedureName="LIST_EMPLOYEES_BY_DEPT">
<![CDATA[CREATE OR REPLACE PROCEDURE "LIST_EMPLOYEES_BY_DEPT" (
 P_PARAM1    IN EMPLOYEES.DEPARTMENT_ID%TYPE,
 RETURNCURSOR    OUT SYS_REFCURSOR) AS
BEGIN
 OPEN RETURNCURSOR FOR
 SELECT * 
 FROM EMPLOYEES
 WHERE DEPARTMENT_ID=P_PARAM1;
END LIST_EMPLOYEES_BY_DEPT;
]]>    
   </createProcedure>
</changeSet>

5. How to work with changelog parameters

Liquibase allows us to define global changelog parameters which will allow dynamic substitution of the parameter in the changelogs described using the ${parameter-name} syntax.
There are many ways to configure parameter values including command line parameter, properties file, parameter block of the database changelog file etc.

Here is an example of using the changelog parameter in the liquibase.properties file:


changeLogFile: target/classes/changelog/master.xml
driver: oracle.jdbc.OracleDriver

url: ${schema.connection.url}
username: ${schema.user}
password: ${schema.password}

# ChangeLog Parameters
client.name: ${dept.name}

Using changelog parameter in changeSet:


<changeSet author="Author1" id="5">
 <sql dbms="oracle"
      endDelimiter=";"
      splitStatements="true"
      stripComments="true">
  UPDATE DEPARTMENT SET DEPT_DESCRIPTION='D1 Department New' WHERE DEPT_NAME= '${dept.name}' ;
 </sql>
</changeSet>

Changelog parameter in changelog file


<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
        xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.1.xsd">

    <property  name="table.name"  value="TEST_TABLE"/>  

    <changeSet  author="Author1"  id="6" >  
         <createTable  tableName="${table.name}">  
             <column  name="id"  type="int"/>  
             <column  name="column1"  type="varchar2(10)"/>  
             <column  name="column2"  type="int"/>  
         </createTable>  
    </changeSet>  
</databaseChangeLog>

That's all folks!

As the application development process goes on, increasingly more database changes are done by the development team which should be kept up and overseen.

With these examples, you will be able to complete the majority of migration tasks for your database including object creation, data insertion and migration, composing DDL and DML utilizing SQL tag, defining the changelog parameters and so on.

How to Package Your Multi-platform Electron App

asaf g — Sun, 06 Sep 2020 11:47:58 +0000

Written by Andres Acevedo

In part one and two of these Electron app development series, we created a basic text loader, and then addressed some security issues.
But at the end of the day, we have not yet experienced the best feature of Electron apps:

Their ability to run on multiple platforms.

In this tutorial, we will use the codebase that we ended up with in the last part. you can get it here.

Note: I usually provide a link to a repository with the completed project at the end of the tutorial, but for this one I think is important that you have the package.json at hand to compare it with yours and find any possible differences in case you run into problems.

You can find the app configured to build on macOS, Windows, and Linux here:

https://github.com/mran3/Text-File-Loader-Build

macOS windows management

Before delving into packaging our app, let's do a small adjustment to our code to respect the conventions of the macOS platform, where usually applications remain open even if they don't have any active windows.
We can implement that behavior easily with electron by adding the following code to app.js.


app.on(
    "window-all-closed",
    () => process.platform !== "darwin" && app.quit() // "darwin" targets macOS only.
);

Other possible values for process.platform are:

freebsd
linux
openbsd
win32 (applies to any Windows).

If you are on a mac, relaunch our app and check that when you close the window, the app remains on the dock.
Don’t worry about the electron icon, we will solve that next.

Icons

Also, if we are talking about distributing our app and provide a good user experience, we can not do it with the Electron logo as the icon. Create or find an icon of your liking for your app, I am going to use this one:

macOS uses a .icns format, Windows requires .ico and Linux prefer .png, fortunately, you can create icon files from a .png using online tools like https://cloudconvert.com/png-to-icns .

Once you have them in both formats, create a new folder in the root of the project (I called it assets) and put both icon files there.

Packaging and distribution toolset

To distribute your app with Electron, you need to package it for each operating system you want to target.
The goal is to generate a .exe file for Windows, a .app for macOS, a .deb for Debian-based Linux distributions, and so on.

Electron Packager is the official Electron tool to help us convert our source code to a bundle specific for macOS, Linux, or Windows.

For Windows, Electron Packager will create a functional .exe along with a bunch of .dll and config files.
Although you can put this together on a .zip file and send it to your users, that does not provide a great user experience.
So we can go a step ahead and consider not only how to package our app but also how to distribute it.

That's when Electron Forge comes into play, allowing us to create nice step by step installation wizards for our application.
Internally Electron Forge uses Electron Packager, so it is not necessary to install them separately, let's run the following command to install them both:


npm install @electron-forge/cli -D

Note

Be careful not to install electron-forge (without the @), as this will install electron-forge v5 which has problems creating apps for the latest versions of macOS.

Besides that, the official documentation relates to v6.

Then, let's import our app into the Electron Forge workflow by running on a terminal located at the root of our project:


npx @electron-forge/cli import

Have in mind that if you want to start a new Electron app with Forge included, you don't have to import your project, instead, you can start a new project with the create-electron-app command (more info on the official Electron Forge documentation: https://www.electronforge.io/).

Although in theory, you could generate all your packages from one platform, that requires installing and configuring a lot of tools (i.e. wine and mono on mac to be able to build for Windows), so I recommend generating each executable in its own platform.

macOS

We are all set for making our executables.

I will start with macOS, but you can skip ahead if you are interested in the process for Windows or Ubuntu Linux.

Open your package.json and in the scripts section, add one new entry:


    "make-mac": "npx @electron-forge/cli make --platform darwin",

Also, modify the name field so it does not contain spaces or uppercase letters.
I am sure you don't want to present your app to your users with that hyphens on it, to avoid that, create a new field below name named productName and put whatever name you like.

The beginning of your package.json file should look similar to this:


{
  "name": "electro-text-reader",
  "productName": "ElectroReader",
  "version": "1.2.0",
  "description": "Multi platform Text Loader",
  "main": "main.js",
  "scripts": {
     "test": "echo \"Error: no test specified\" && exit 1",
     "make-mac": "electron-forge make --platform darwin"
  },

Now we need to add some configuration for Electron Forge, inside your package.json there should be a config field:


"config": {
    "forge": {
      "packagerConfig": {
        "icon": "./assets/icon"
      }
    }
  }

As you can see, the icon extension is not specified, but Electron Packager (under the hood) will add the correct extension (.ico or .icons) depending on the platform.

Now run the script we created by running the following command on your terminal:


npm run make-mac

At the end of the process, you should end up with a new folder named out.

Inside of it there should be two folders one with a long name containing the .app file generated by Electron Packager and the other named make contains a .zip file that you can distribute to your users.
Uncompress, run and enjoy:

Doesn't it look nice?

Note

For each platform, you can have different makers that produce different types of distributable files, maker-zip is the default maker for macOS, but others allow for example exporting to DMG files

(you can read more about it here: https://www.electronforge.io/config/makers/dmg ).

Ubuntu (Debian) Linux {#linux}

To create a .deb package, we will need to make sure that our system has two utilities installed: fakeroot and dpkg, these are not npm packages, but actual applications of the operating system.

To install them, run the following commands on your terminal:


sudo apt install dpkg
sudo apt install fakeroot

Add a new entry inside your scripts section with the following content:


"make-linux": "npx @electron-forge/cli make --platform linux --targets deb"

You could just execute npx @electron-forge/cli make without the platform or targets arguments and electron forge will try to identify in what platform it is running and try to create the corresponding packages.

Getting the icon to work on linux required some additional steps.

Make sure the configuration of maker for debian (in your package.json) specifies what icon to use:



{
          "name": "@electron-forge/maker-deb",
          "config": {
            "icon": "./build/icon.png"
          }
}

This will set the icon of the app, but for it to appear in the dock, we will need to jump off from our package.json for a moment and modify our main.js adding the icon to the call of new BrowserWindow.



const win = new BrowserWindow({
    width: 800,
    height: 600,
    icon: path.join(app.getAppPath(), 'build/icon.png'),
    webPreferences: {
      worldSafeExecuteJavaScript: true,
      contextIsolation: true,
      preload: path.join(app.getAppPath(), 'preload.js')
    }
  })

Now on a terminal let's run:


npm run make-linux

If everything went well you should have again an out folder with two sub-folders.

One with a bunch of files (the output of electron-packager), and make where you will find a nice .deb file ready to be double-clicked and install our beautiful app:

The app installed on Ubuntu

One configuration that builds your app without problem in one platform, might need to install additional packages on another.
For example, macOS does not have a problem if the description field on package.json is not present, but it is required for Debian.

Windows {#windows}

As you might guess, we need to add a new entry to npm scripts.
Why not test the platform inferring capabilities of Electron Forge and call it without parameters:


"make": "npx @electron-forge/cli make"

Squirrel is the default maker for Windows.
It is intended to be lightweight and provide a hassle-free experience to the user, not requiring admin permissions or long installation wizards.

As a side note, this makes Squirrel great for creating apps that can run on a pen drive.

Right now Squirrel will work, but it won't generate a nice Desktop icon, but for that, we just need to add a simple line to our main.js file:


if(require('electron-squirrel-startup')) return;

Run npm make from the terminal, install your application, test it, and if everything goes right, edit your CV and add "Developer of multi-platform desktop applications". 😉

Electron app installed on Windows

If you run into any problems, remember that you can find the completed project on:

https://github.com/mran3/Text-File-Loader-Build

Database Version Control With Liquibase

asaf g — Sun, 06 Sep 2020 08:51:30 +0000

Motivation

Let me give you a scenario,

You have a project with multiple database instances in different environments (Dev, QA, Production) and you need to manage the database schema changes that are done against those environments.
Let’s assume that you are managing those changes by creating a git project or a shared file on a drive and adding a new SQL file for each database changes you are doing.
To implement your changes in that database, for each SQL file, you need to run the changes in each environment manually and add a flag or note to indicate which change is run on which environment.

Would it be able to complete your task?
Yes, Yet, is it a decent method to manage schema changes?

Of Course Not.

Here are some of the many problems you might face while doing so

Hard to synchronize database and application code changes in different environments
The tedious process to run each change manually in different environments
Collaboration across the development team on what change is deployed and what is not
Hard to roll-back to the previous version of the database
Possibility of data loss

Here comes Liquibase

Liquibase is an open-source library for tracking and managing database schema changes that can be used for any database with a JDBC driver.

It is a platform-independent database migration tool that allows the database changes referred to as ‘changesets’ to be written in various formats including XML, JSON, YAML, and SQL.

Features

Supports almost all databases that have a JDBC driver.
Changesets can be written in different formats like XML, JSON, YAML, and SQL.
Can be used to automatically generate changesets for an existing database
Easy to integrate with build tools like Jenkins, Maven etc
Supports database rollbacks
Supports context-dependent logic allowing us to use global context and preconditions
Can be executed via command line, Apache Maven, Apache Ant, Spring Framework
Has feature to generate changeset from an existing database and can also generate schema difference as changesets

Different ways to run liquibase

Embed liquibase with your app: Embedding liquibase with your application code will automatically deploy liquibase on the app startup.
Run liquibase using build tools: Integrate liquibase into your build process (with build tools like Jenkins, Ant, Maven, and Gradle) and update them without being tied up with the application.
Generate the SQL and run it manually: Using update SQL, Liquibase provides the SQL generated from the changeset along with the database changes required to keep the tracking tables up to date. DBA will then inspect the SQL and run them against the database.

Installation Process

Prerequisites: Liquibase requires Java 8+

There are two ways to install Liquibase, Manual installation and using liquibase installer.

If you set up liquibase using the liquibase installer, dependencies, directories, config and properties files will all be in place already.
It also provides some examples which will provide you with the core concepts required to understand the changesets.

In the case of manual installation, you need to download the compressed liquibase file and extract it in your workspace.

For windows users, you need to add a new PATH variable in the Environment Variables.

For macOS users, the path should be added to the bash.profile file.

For detailed instruction on Installation please follow this document .

Core Concepts

liquibase.properties: The file liquibase.properties is a text-based file that stores common properties like database connection parameters, driver details, classpath parameters, global changelog parameters etc. If you install liquibase using liquibase installer, it will provide pre-written liquibase.properties file while in case of manual installation, you need to create liquibase.properties file using a sample file provided.

DatabaseChangeLog: Databasechangelog is a file where all changesets go. Each database changelog can include one or more changesets.

Changeset: In liquibase, a changeset is represented as an atomic change to the database. Each changeset should be uniquely identified using author and id fields. The database handles each changeset as a single transaction. Changesets can be written in JSON, XML, SQL and YAML formats.

DATABASECHANGELOG & DATABASECHANGELOGLOCK: These two tables are created by liquibase to track the changes that are run against the database and to make sure that no other migrations age going on.

What if I don’t like Liquibase?

If you told me that you don’t like liquibase and are looking for alternatives, I would ask why not Liquibase first.

Liquibase is a sophisticated tool for database migration that has all features that you need for professional database refactoring and versioning.

But still, if you don’t want to use liquibase, here are some alternatives.

Flyway: Flyway is an open-source Apache licenced tool for database migration where you can write migrations in database-specific SQL or using Java code. For more details on Flyway, you can refer to this website. https://flywaydb.org/
YUNIQL: YUNIQL is also an open-source schema versioning and database migration engine that uses plain SQL scripts which can be integrated with CI/CD pipelines. If you want to check out YUNIQL, you can refer to this website. https://yuniql.io/

Checking Out Quest DB

asaf g — Mon, 31 Aug 2020 15:52:44 +0000

What is Quest DB?

QuestDB is an open-source Time Series database that uses a column-oriented storage approach and vectored execution for simple and extremely quick query execution.

It was developed by QuestDB Limited in 2014.

It is written in Java and supports Linux, macOS and Windows as the server operating system.

The query language utilized for QuestDB is SQL (Structured Query Language) alongside its expansion for time series-analysis.

QuestDB depends on the relational data model, normally found in some of the popular relational databases like PostgreSQL, MySQL, SQL Server, Oracle, and so forth which means that each time-series estimation is recorded in its row, with a time field followed by other fields.

Features

Some of the best features of QuestDB are as follows.

Time series database with a relational data model.
SIMD optimized analytics
Uses column-oriented storage technologies with each column stored continuously in sequential blocks
It allows columns to be chosen as a designated timestamp to utilize it’s high-performance time series functions
Comes with a built-in SQL optimizer
InfluxDB line and Postgres wire protocol support
Supports time series and relational joins
It supports horizontal partitioning (by timestamps)
Supports ACID

The setting up of QuestDB can be done easily by following the QuestDB documentation.

It runs on port 9000 (by default) and provides a web console to interact with the database. The web console has a schema/table browser, query tool and visualization window where the query results can be shown either in tabular form or using graphs.

Here is how the web console looks like:

How to setup QuestDB

If like me, you want to setup QuestDB for learning or for a development environment, I would recommend using Docker.
Here are the instructions I used:
https://questdb.io/docs/packages/docker/

Your other options are:

Using binaries:
For setting up QuestDB using binaries, you can refer to QuestDB's documentation https://questdb.io/docs/packages/binaries/ .
Homebrew:
For macOS users, QuestDB is available via Homebrew https://questdb.io/docs/packages/homebrew/ .

Why should you choose QuestDB?

The one thing I like most about QuestDB is that despite the fact that its engineering is planned from the ground up to be close to the hardware.
It offers advanced execution and is additionally centered around making it simpler for designers to integrate and query.

Following are the reasons you should choose QuestDB

Easy to set up and use

QuestDB is a lightweight open-source database and is anything but difficult to set up without any conditions.
It will take just a couple of minutes to download and you can begin playing with it.
Its official website provides detailed technical documentation which makes installation, configuration and running the database very simple.
Its web console enables us to perform real-time analytics with easy visualization using different types of charts like bar, line area etc.

Supports SQL

One reason for utilizing QuestDB is its SQL-like query language.
On the off chance that you know about SQL, utilizing QuestDB would be simpler as it depends on SQL with its very time-series alterations.

Performance

Execution is one of the fundamental rules that ought to be thought of while picking the database.
QuestDB offers better execution while managing time-series data like data from IoT gadgets, stock value information and DevOps measurements.
For a database that is new in the TSDB field, it gives seemingly preferable execution over other entrenched time-series databases.

Why QuestDB is not a good choice?

New to the TSDB ecosystem

QuestDB is a less mature product than other time-series databases like TimescaleDB and InfluxDB, which implies that there are a ton of integrations and features that are yet to be worked from the developers' side.
So if you are searching for a well established time-series database with an extraordinary help network and discussion, QuestDB isn't there yet.

OLTP Support

OLTP databases require frequent inserts, updates, and deletes with user-defined constraints to be kept up.
If you are intending to utilize a database that likewise supports Online Transaction Processing, QuestDB is not a solid match.

How does it compare with other databases?

QuestDB vs SQL Databases

QuestDB is like a SQL database (MySQL, Oracle, PostgreSQL) however unique from multiple points of view.

Other relational databases can also handle time-series data but they tend to offer worse performance while dealing with common time-series operations like real-time aggregations of data.

On the other hand, QuestDB is purpose-built for fast storage and processing of time series data.
The data model followed by QuestDB is a relational data model which means that data is stored in a table and each row in a table represents a collection of related values.
It also follows the ACID property and supports joins like relational databases but does not allow user-defined constraints and triggers.

Alternatives to QuestDB

Some of the best alternatives to QuestDB are as follows

InfluxDB

InfluxDB is an open-source time-series database designed for high-availability storage and retrieval of data.

It is a part of the TICK (Telegraf, InfluxDB, Chronograf, Kapacitor) stack and has a non-relational, NoSQL data model. It has SQL-like query language to handle interactions with the data

TimescaleDB

Built on the top of the Postgres database, TImescaleDB is optimized for fast ingest and complex queries with full SQL support like traditional relational databases.

KDB+

KDB+ is also a high performance column-store relational time-series database with in-memory abilities.

It is commonly used to store, analyze and process high-frequency financial time series data.

I hope you enjoyed, and of course as always, feel free to comment or share.

Thank you for reading.

Securing Your Electron App

asaf g — Tue, 18 Aug 2020 10:33:53 +0000

Written by Andres Acevedo

In our previous article, we explored how to create a useful Electron app focusing on functionality, but leaving aside some aspects like security and platform specific features.

In this article, we will continue from where we left, to progress in the path from a Prototype to a real world Application!

I’d like to begin with an analogy: in healthcare studies, there's an important Latin maxim: Primum non nocere, it means:

"first, do no harm". Another way to state it is that, "given an existing problem, it may be better not to do something, or even to do nothing, than to risk causing more harm than good."

If we are introducing security problems to our clients, it could be best to go back a bit (pun intended) and solve these vulnerabilities before adding new features to our pretty piece of software.

Working according to the warnings

So, is our app secure? If we run it with npx electron . and open the Developer Tools (View>Toggle Developer Tools), we can see that the console is giving us several security warnings:

One easy way to improve our app's security is to add a special meta header to our index.html file:


<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'">

This instructs the Chromium rendering engine to only run local scripts.

You can also add other allowed domains in the following way:


Content-Security-Policy: script-src 'self' https://apis.example.com

If you want to learn more about CSP, check:

https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP

Once we add that line to the header section of our website and refresh our app (View > Reload), we are left with just one security warning:

Getting rid of this security warning requires that we modify our main.js file and add the following webPreference to our BrowserWindow call:


webPreferences: {
            worldSafeExecuteJavaScript: true,
            contextIsolation: true
}

What this does is to enforce isolation between the Renderer World (Chromium renderer) and Node World. This sandboxes any malicious code run from the renderer, reducing the possible damage of an exploited vulnerability.

Once we add the code, reloading the view is not enough - as we did changes in our main.js file, so we have to terminate our Electron App and relaunch it with: npx electron .

Good news: we got rid of the warning.

Bad news: now we have an actual error and our app does not work anymore

Fixing the code

Require is a node function, and because we configured electron to separate renderer and node worlds, we can not use require() on our renderer.js file.
Even if we have the nodeIntegration: true setting.

In fact, let's remove that setting, so our BrowserWindow call in main.js now looks like this:


const win = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      worldSafeExecuteJavaScript: true,
      contextIsolation: true,
    }
  })

Electron provides us with a way to use node functions in the renderer world: Preload scripts.

Add the following line to webPreferences in our BrowserWindow call on main.js.

preload: path.join(app.getAppPath(), 'preload.js')

It should look like this:


const win = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      worldSafeExecuteJavaScript: true,
      contextIsolation: true,
      preload: path.join(app.getAppPath(), 'preload.js')
    }
  })

Don't forget to include the path module at the beginning of main.js:

const path = require('path');

Now let's create the preload.js file and write the following on it:


const { contextBridge, ipcRenderer } = require("electron");

contextBridge.exposeInMainWorld(
    "electron",
    {
        ipcRenderer: ipcRenderer
    }
);

As you can see on the first line, we can use the require() function in preload.js without problems in order to import the ipcRenderer.

Now, we will remove the require line that is causing the error on the renderer.js file:


const { ipcRenderer } = require('electron');

As we are not requiring ipcRenderer anymore, we can not continue using it in the same way as before.
But that's when the contextBridge.exposeInMainWorld function we added on preload.js comes in handy.
Our ipcRenderer can be accessed on the rendered.js file by using window.electron.ipcRenderer.

So it now should look like this:


async function fileSelected(e){    
    const loadedFilePath = e.target.files[0]?.path
    let data = await window.electron.ipcRenderer.invoke('read-file', loadedFilePath)
    document.getElementById("loadedText").value = data
}
document.getElementById("fileLoader").addEventListener("change", fileSelected);

Result:

We've made it! Our app works again and without any security warnings!

You can find the code of our Secure Text Loader on the following repo:
https://github.com/mran3/Secure-Text-File-Loader/

Multi Platform Desktop App With Electron

asaf g — Fri, 14 Aug 2020 10:17:15 +0000

Written by Andres Acevedo

Slack, Visual Studio Code, and Skype among others have shown us that writing performant eye-catching multi-platform apps with web technologies is not only possible but that they can compete with apps made using different frameworks as native libraries or Java.

Today we will create a basic app that allows reading a text file into a text area.

This functionality is at the core of several apps like word processors, and code editors among others.

Pre Requisites

A solid understanding of how HTML, CSS, and Javascript work together on a web page.

Ability to code on Javascript.

Node and Npm (or Yarn installed).

Know how to install node packages.

Installing Electron

Run

npm init

to initialize a new node application and fill the information requested in the terminal.

Then run

npm install --save-dev electron

Hello Electron

Now we will create a basic app that displays "Hello Electron" just to see the electron framework working.

Create a new main.js file and write the following on it:

// We need to use require instead of import because this code runs on a node environment. 
const { app, BrowserWindow } = require('electron')

function createWindow () {
  // Create the browser window.
  const win = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: { nodeIntegration: true }
  })

  // and load the index.html of the app.
  win.loadFile('index.html')
}

app.whenReady().then(createWindow)

// In this file you can include the rest of your app's specific main process
// code. You can also put them in separate files and require them here.

Now create a very simple index.html file:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <title>Hello Hello Electron!</title>
  </head>
  <body>
    <h1>Hello Electron!</h1>
  </body>
</html>

Now, let's run the npx electron command on your shell to see our first electron app running.

This is very nice, but can give us the impression that Electron is just a wrapper that we use on our websites to convert them to desktop applications.

Once we add some more functionality to our app, we will see that there are specific Electron topics that we need to take care of in order to create a useful piece of software.

Electron File Loader

Let's change our HTML so we have an input file control and a textarea element:

<!DOCTYPE html>
<html>

<head>
  <meta charset="UTF-8">
  <link rel="stylesheet" href="styles.css">
  <title>Text File Loader</title>
</head>

<body>
  <main>
    <h1>Text File Loader</h1>
    <form action="">
      <input type="file" name="fileLoader" id="fileLoader">
      <textarea name="loadedText" id="loadedText" cols="30" rows="20"></textarea>
    </form>
  </main>
</body>

</html>

We will also need some simple CSS style, so create a new file called styles.css and put the following content on it:

main {
    margin: 2rem;;
}
textarea {
    width: 100%;
    margin: auto;
    display: block;
}

At this moment, when you run your app it should look something like this:

We have the visual elements, now we need to make them interactive with Javascript. For this, we need to call a script inside our HTML using the script tag. Put it at the end of the file, just after closing the body tag:


</body>
<script type="module" src="./renderer.js"></script>

Of course, we need to create this new renderer.js file. There we will add an event listener to detect that a file has been selected by the user:

document.getElementById("fileLoader").addEventListener("change", fileSelected);

Now, the code for the event handler:

const { ipcRenderer } = require('electron');

async function  fileSelected(e){    
    const loadedFilePath = e.target.files[0]?.path
    let data = await window.electron.ipcRenderer.invoke('read-file', loadedFilePath)
}

This kind of function should be familiar if you are a web Javascript developer, except for this line:


let data = await window.electron.ipcRenderer.invoke('read-file', loadedFilePath)

Usually, the code of Electron apps runs in two different isolated contexts:

The rendered context is the JavaScript code that is interpreted by a WebView provided by Electron and the main context is a node environment where you can make use of some native desktop functionalities (like loading files from the filesystem).

ipcRenderer is an EventEmitter. It provides a few methods so you can send synchronous and asynchronous messages from the render process (web page) to the main process.

We are sending the 'read-file' message, but we need another piece of code to receive this message, right?

Of course! That is what the next code is about, write it into your app.js file:


ipcMain.handle('read-file', async (ipcEvent, path) => {
    if (path != undefined) {
        console.log('Load: ' + path)        
        //Here we will add the necessary code for loading the file.
    }
})

Don't forget to require ipcMain from electron at the beginning of the main.js file

const { app, BrowserWindow, ipcMain } = require("electron");

If you check in the console (your terminal as the handle is executed in the main context).

You will see that it prints out the name of the file that we have selected.

Loading and returning the file contents is a two-liner using promises along with async and await:


ipcMain.handle('read-file', async (ipcEvent, path) => {
    if (path != undefined) {
        console.log('Load: ' + path)        
        let loadedText = await fs.readFile(path, 'utf-8')
        return loadedText
    }
})

Don't forget to require the promise ready version of the fs module at the beginning of main.js .

Now that we have our data, let's go back to our renderer.js file and show them in the text area adding the following line to our fileSelected function:


async function fi
<a href="/electron-1/text-file.png" target="_blank"><img src="/electron-1/text-file.png" alt="electron text file loader" class="screenshot"></a>leSelected(e){    
    const loadedFilePath = e.target.files[0]?.path
    let data = await ipcRenderer.invoke('read-file', loadedFilePath)
    document.getElementById("loadedText").value = data
}

Voilà! You have coded a simple but functional app with Electron.

You can check out the source code of this example on:

https://github.com/mran3/Text-File-Loader

Evolution of Serverless Technologies

asaf g — Sat, 25 Jul 2020 17:01:18 +0000

Written by Nipuna Dilhara

Where it all started - Datacenters

When it comes to the modern era, deploying web services has become a common requirement that needs minimal effort. However, it wasn’t the case a few decades ago.

In the late 90s, the internet was a trending topic among people due to the emergence of web applications. It was a fascinating idea to make the data available online and available from anywhere from the world.

During this period, companies started to move their services as monolithic web applications and deployed and hosted in bulky and massive physical servers with local data centers. However, deploying monolithic applications and maintaining physical servers required a lot of work. They needed a considerable amount of hardware components. Delivering, assembling, and configuring them required a lot of time and expertise.

The hardware components were needed to be delivered, assembled, and configured which was time-consuming and needed specific domain expertise that wasn't common in those days. It was the opposite of what we are experiencing today.

As time goes, the amount of data to be processed gradually increased due to the advancements of new technologies and more importantly due to the improvements of businesses and their use cases. More complex data types such as relational data, graphical representations, and document-oriented data came into play. Hence, using traditional approaches based on local servers was not an option anymore. Due to these reasons, businesses started to move towards running several environments on single hosting services to utilize the hardware and maintenance efforts.

The concept of Network-Attached Storage (NAS) was the first such approach towards shared hosting services. It was simply a file storage server attached to a computer network that facilitates access of data to a large and diverse group of clients.

Even though the NAS was an effective solution for the problem, the deployment, and required resources were highly expensive. As a result, the small and medium-sized companies could not afford to move into NAS based solutions.

Introduction of Cloud Computing

With the further advancements of technologies, the concept of using Virtual Machines (VM) surfaced as a much affordable and flexible approach. As the name indicates, the VM is a computer application that acts as a separate virtual computer inside the same computer. Hence it allows users to run different operating systems with the required applications and configurations inside the same physical machine. During the 1990s, these concepts provided the fundamental of developing necessary computer infrastructure for the initiation of cloud computing concepts.

When the required technological background was being built, Amazon launched the Amazon Web Services - Elastic Computer Cloud (AWS-EC2) in the year 2006, making an evolutionary change to the direction of cloud computing. This allowed people and businesses to hire VMs to run their web services with minimal configuration cost and effort. The EC2 provided a secure and scalable computer space to run user applications through virtual servers hosted in the cloud.

The EC2 consists of five key advantages over the known limitations of the traditional approach of data centers which became a common feature of later cloud computing services.

Reduced labor cost
Reduced risk
Reduced infrastructure cost
Scaling
Lead time

Hosting monolithic applications in local servers required extensive maintenance and management. With the arrival of cloud computing, the corresponding responsibilities were transferred to the cloud service providers which was a huge relief and advantage for businesses.

Also, cloud computing freed businesses from having the burden of any hardware failure and the expertise of handling such situations. The cloud computing services provided the required infrastructure and their repair and maintenance for less cost.

In the early days, businesses had to go through the troublesome processes of buying more servers and related infrastructures when expanding their businesses with the growth of customer base and service. With the arrival of cloud computing, it became so much easier and cost-effective to increase and decrease the number of owning instances according to the requirement.

The significantly less deployment time of applications made cloud computing a good bargain for businesses. Also, it gave the edge for businesses over those who were still using traditional data centers based hostings.

IaaS, Saas, and PaaS

There are three popular paradigms of cloud computing which changed the way of using the infrastructure.

Infrastructure as a Service (Iaas)
Software as a Service (SaaS)
Platform as a Service (PaaS)

IaaS

Iaas was the first one to be introduced. The AWS Simple Storage Service was one of such kind which was soon followed by Google, Microsoft, and other leading IT names of the industry.

The term ‘Cloud’ was originally referred to as a public cloud which was a set of computer resources delivered by a provider and stored in a service provider’s data centers. However, due to the rise of cloud infrastructures, businesses became able to have their own data centers utilizing software such as Open Stack. Such self-hosted systems that use on-premise hardware were known as private clouds.

PaaS

The PasS came into play after the IaaS. As IaaS provided all the infrastructure necessities, PaaS brought an operating system and kept on top of IaaS. This freed businesses from the burden of installing operating systems and setting up relevant configurations. The cloud service provider took the responsibility of:

Operating system installation
Upgrading patches
Monitoring the system

Microsoft Azure, Google App Engine, Heroku, and Elastic Beanstalk are some examples of such PaaS providers.

If a business wants to have their own PaaS, they can either host it on-premise servers, on top of a public IaaS cloud service, or use a mix of both private and public clouds which was generally referred to as a hybrid cloud.

SaaS

The Software as a Service was a successor of PaaS. As the IaaS provided the necessary infrastructure and PaaS provided an operating system to run on, SaaS was a software distribution model where the businesses can host their software applications and make them available for customers. If a customer requested a published software, the business provided the customer with access to a copy of the software with which the customer can work. The customer received all the new features and updates promptly and customer-specific details were saved on the customer’s device, on the cloud, or both.

Parallel to these cloud computing advancements, the application architectures have also evolved massively. As the monolithic applications were needed to be scaled with the time, the amount of data processing was also increased with that. This made it’s difficult to run monolithic applications within a single server. So they were moved into a more fine-grained approach where the application was broken into small manageable parts that will run in dedicated machines. This leads the path to the arrival of containers.

Containers

A container consists of a complete run time environment that includes an application with all its dependencies, libraries, and other configuration files that are required for the successful and smooth run of the application.

The first commercial container was introduced as a feature of the Solaris 10 Unix operating system in the name “Zones”. There are some similarities between Containers and VMs as both of them provide secure and independent space for the software application to execute. They give the impression of an individual system with its own set of administrators and users. However, unlike with virtual machines, a stack of containers can be put together on a single hosted operating system with significantly less time compared to VMs.

In the early days of containers, the community believed that the containerizing was hard to do. The arrival of Docker in the year 2013 was able to change this upside down. Docker fastly got the attention of the industry experts due to its easiness to bundle applications with everything that it requires to run. As the containers gathered attention, cloud computing service providers had to host and manage containers on behalf of businesses which eventually got the name Container as a Service (CaaS). Kubernetes, Google container engines, AWS-ECS are some well-known examples of CaaS.

Containers offered a simplified interface for developers that can be used to directly implement the code which reduced the burden of what developers needed to worry about while deploying applications. However, it was not enough. the industry experts were looking for a more simplified and convenient approach to further reduce the complexity for developers. It was this era, that the concept of serverless emerged making it possible for developers to focus only on their code and corresponding service configurations instead of worrying about everything else. With the arrival of serverless, all the burden of deploying an application into the final hosted location was transferred to the service provider.

Serverless

The serverless architecture facilitates the developers with both Backend as a Service (BaaS) and Function as a Service (FaaS) features.

The BaaS allows programmers to concentrate on the front end of the application and manage backend infrastructure without building or maintaining them. The FaaS is a serverless backend service that enables developers to create modular pieces of code on the move that can be implemented in response to certain events. This approach took off the requirement of maintaining physical infrastructure and the procedure that should be followed when deploying an application to server-side spaces. Hence it further facilitated programmers to only focus on the application logic. The programmers only had to write the programming logic and upload them to the FaaS platform which would invoke the function as an executable object once an corresponding event occurred. This made the serverless to be an event-based approach where the flow of the program was determined by different events or user inputs.

Businesses got most out of these evolutionary advancements of serverless technologies. The arrival and development of serverless technologies reduced the cost and the effort required for the deployment and maintenance of business applications. When the businesses wanted to scale their operations or even change the directions of their services, they were able to do it with considerably less burden compared to monolithic applications hosted on server-side spaces. Tasks that took years to complete became less time consuming as the evolutions of serverless concepts.

The release of AWS Lambda marked a great milestone in the serverless, reducing the speed of deployment to milliseconds with greater scalability and availability. The AWS Lambda facilitated programmers to completely focus on the logical aspect of the application.

Soon the competition for the dominance of the serverless domain grew rapidly. Even though it seems like Amazon is still in lead, other services such as Microsoft and Google came into the play threatening the dominance of Amazon. However, even to the current date of 2020, Amazon is still leading the Cloud infrastructure services launching new concepts and technologies regularly.

But the race hasn't ended yet. The evolution of new serverless concepts and technologies hasn't ended yet. It’s guaranteed that there are a lot more to come. The focus on serverless technologies in the IT industry is greater than never before. Even at the university level, serverless technologies have become a trending topic that created opportunities for new projects and researches. It’s already clear that serverless technologies will become a key factor of the future IT and business world. For the moment, let’s keep our heads high and see where these technological evolutions take us.

Analyzing SSL Handshake Logs

asaf g — Sat, 11 Jul 2020 20:15:41 +0000

Analyzing SSL handshake logs

SSL 101

Currently, we are in an era of online businesses and for good or bad, we are in a path of becoming an online life form. Meanwhile, cyber crimes and attacks are becoming more of a threat to businesses at each passing day.

If you are an online business owner or stakeholder, one of your main goals is to create a trusted environment for your consumers. You have to win the faith of customers that their money and sensitive details are secured with you. Consumers won’t purchase unless they feel safe with their money and have the trust of receiving the purchased product or service.

In the modern era, the SSL is one of the key elements that create the foundations of trust by establishing a secure connection.

The SSL ensures that the internet connection between two systems and the sensitive data transferring between those two systems is secure which prevents cyber criminals from compromising any information transferred. If you visit an SSL secured website, you will receive a visual indication as below.

So what is this SSL?

The SSL stands for Secure Socket Layer.

Let me give you some insightful details on where this SSL protocol runs.

You might have heard that TCP/IP (Transmission Control Protocol/Internet Protocol) governs the transport and routing of data over the internet. The HTTP protocol runs on top of TCP/IP to support application tasks such as presenting web pages and running relevant functionalities.

The SSL protocol runs above the TCP/IP and below the HTTP. The SSL uses TCP/IP on behalf of HTTP and establishes an encrypted secure connection between two communicating parties (client and server). With the SSL protocol in place, an eavesdropper can only see the connection endpoints but cannot read or modify any of the actual data. Hence it protects users' sensitive data.

Let’s see how the SSL does that?

That is where the SSL-Handshake comes into play.

Deep Dive into SSL Handshake

The SSL-Handshake allows an SSL-enabled server to authenticate itself to an SSL-enabled client and also allows the client to authenticate itself to the remote server. During the handshake, both parties exchange important and unique information that establishes the aforementioned secured connection.

There are two types of SSL-Handshakes:

One-way SSL
Two-way SSL (Mutual SSL)

There is a major difference between these two types of handshakes.

In a one-way SSL, the client validates the identity of the server whereas, in two-way SSL, both the server and the client validate the identity of each other.

Let’s discuss the steps of SSL-Handshake in more detail.

The SSL-Handshake mainly consists of the following steps.

Client Hello
Server Hello
Certificate Chain
Server Key Exchange
Certificate Request (only in two-way SSL)
Server Hello Done
Client Key Exchange
Client Change Cipher Spec
Client Finished
Server Change Cipher Spec
Server Finished

The following diagram illustrates more on the flow.

Let me explain what each step means.

1. Client Hello

The first message is called ‘ClientHello’ which is sent from the client to the server. ClientHello is responsible for sending the fundamental details required by the server to establish a secure connection.
The ClientHello message carries the details such as:

SSL or TLS version
Cryptographic algorithms
Data compression methods supported by the client
Client random

The given handshake logs show that the ClientHello has been sent with TLS v1.2 which tells the server of the supported and preferred TLS version. Also, it shows the list of ciphers that are supported by the client.

Forgot to mention.

The TLS stands for Transport Layer Security who is the successor of SSL. The later versions of SSL come under the TLS.

*** ClientHello, TLSv1.2
RandomCookie:  GMT: 1593979669 bytes = { 106, 130, 143, 152, 153, 26, 144, 152, 168, 252, 70, 107, 162, 143, 98, 21, 191, 157, 185, 58, 130, 231, 165, 65, 33, 110, 224, 91 }
Session ID:  {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, secp384r1, secp521r1, sect283k1, sect283r1, sect409k1, sect409r1, sect571k1, sect571r1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA256withDSA, SHA224withECDSA, SHA224withRSA, SHA224withDSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA
Extension extended_master_secret
Extension renegotiation_info, renegotiated_connection: <empty>
***

Once the server receives this message, it selects a suitable cipher suite that it supports from the given cipher suite list by the client. The server will ignore other ciphers that are not recognized or supported. The selection process that happens here leads to the second step, ServerHello.

2. Server Hello

As the name indicates, the ServerHello is sent by the server. The ServerHello message tells the client what are the connection parameters that have been selected by the server from the provided list.

The server considers the received TLS version with the ClientHello and selects the highest supported version which is equal or lower to the client's suggestion. Also, it will select a cipher suite from the presented list of ciphers.

Generally, the ServerHello consists of:

Agreed SSL/TLS version
Session ID
Agreed cipher suite and other relevant details
Server random

*** ServerHello, TLSv1.2
RandomCookie:  GMT: 1593979669 bytes = { 204, 192, 95, 44, 21, 64, 253, 87, 10, 239, 52, 190, 190, 61, 186, 51, 125, 16, 68, 141, 170, 91, 232, 205, 148, 128, 78, 185 }
Session ID:  {95, 2, 51, 21, 21, 21, 188, 149, 248, 189, 8, 38, 152, 164, 142, 100, 130, 187, 177, 23, 237, 154, 136, 93, 124, 112, 147, 126, 43, 86, 219, 51}
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
Extension extended_master_secret
***

If the client and server don’t have any common attributes the SSL handshake fails at this point.

3. Server Certificate Chain

The Certificate Chain message is sent by the server along with the ServerHello. This particular message consists of the server's SSL certificate and it’s the intermediate certificate chain.

To establish the trust between both parties, the server should have a valid SSL certificate signed by a Certificate Authority (CA).

Once the certificate is received, the client contacts the server's CA and verifies the server's digital certificate thus confirming the authenticity of the server. The verification process might consist of:

Validating the certificate’s digital signature
Verifying the certificate chain
Checking for certificate details such as domain name, expiration date, etc.

*** Certificate chain
chain [0] = [
[
  Version: V3
  Subject: CN=ActiveMQ Artemis Server, OU=Artemis, O=ActiveMQ, L=AMQ, ST=AMQ, C=AMQ
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

  Key:  Sun RSA public key, 2048 bits
  modulus: 17909370440277738952478021743432069490529460393959126050805622971590660765692586361919607268411957926196894593987779661515762234267984346200511739419991266042217515340629051394016939807213747552584949772262005902231184424962174373631762755583097509054166650866644125680436982880019781047492697566821940066615518316518157033101020228431401518000223024317668304190076399662045036184270398732773568597958342161459714296835776954926256734783011948914891588288449919569310898632115411415768934785779564943363123191032516179643589537811762899108471543653043311642283642184737727344704476726058553680310548513037215719011553
  public exponent: 65537
  Validity: [From: Sun Jul 05 22:36:37 IST 2020,
               To: Sat Oct 03 22:36:37 IST 2020]
  Issuer: CN=ActiveMQ Artemis Server, OU=Artemis, O=ActiveMQ, L=AMQ, ST=AMQ, C=AMQ
  SerialNumber: [    341a9018]

Certificate Extensions: 1
[1]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 19 D8 C4 B1 B1 85 FA E1   65 97 80 90 EF 6C AE 56  ........e....l.V
0010: 14 6E DA E1                                        .n..
]
]

]
  Algorithm: [SHA256withRSA]
  Signature:
0000: 44 E5 D4 3F E8 7C 15 E7   B1 E8 F0 45 AB A3 EA 77  D..?.......E...w
0010: 1D 67 3C FC 5B 95 D0 98   F1 B9 28 6C E4 FC D1 6B  .g<.[.....(l...k
0020: CD 7D 9F C3 4C D8 8F CB   E1 A1 63 24 EC E5 A3 5B  ....L.....c$...[
0030: E6 18 12 46 7D 0F E4 14   25 08 B8 3B AB AF E2 CE  ...F....%..;....
0040: A3 FE 93 8E 88 EB 97 62   59 71 3D 5D 92 D0 99 55  .......bYq=]...U
0050: 68 E3 13 1A 7E AD 41 9E   32 DD C0 58 C7 08 81 E4  h.....A.2..X....
0060: C7 8B CF 57 69 A8 0C EA   74 5F 45 8C 6E D9 E2 D1  ...Wi...t_E.n...
0070: B6 E4 B4 D2 5F D0 BD FA   BA CB 98 7B B6 1D 9D E4  ...._...........
0080: 45 DA C9 09 19 71 BD B4   7C 44 CE 6A 48 13 75 A1  E....q...D.jH.u.
0090: 6D DD DC 48 65 0E 05 3B   BA AF 63 42 2E 9F 3B 29  m..He..;..cB..;)
00A0: FE 8E 3D 07 03 04 C4 D7   65 DB 7A FA 45 C1 62 6B  ..=.....e.z.E.bk
00B0: E8 E7 D1 A7 8E 6F 0D 1D   7F 3E 83 66 75 5B 60 45  .....o...>.fu[`E
00C0: 0E BC A1 8B B3 2D 50 1A   E5 3D 02 A5 37 D6 76 DB  .....-P..=..7.v.
00D0: 5A A5 EE 26 B1 06 DC 57   F4 68 3F DA 0F C9 D4 30  Z..&...W.h?....0
00E0: BA B0 DA AD 94 24 B4 15   FF E5 DC 6E 1D 5D F9 58  .....$.....n.].X
00F0: D4 ED 31 9C 11 C0 C9 E8   29 5D F0 45 AF 93 9B CB  ..1.....)].E....

]
***

If the client finds a valid certificate, you will receive a ‘Found trusted certificate’ message followed by the ‘Certificate chain’ at the client side logs.

Found trusted certificate:
[
[
  Version: V3
  Subject: CN=ActiveMQ Artemis Server, OU=Artemis, O=ActiveMQ, L=AMQ, ST=AMQ, C=AMQ
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

  Key:  Sun RSA public key, 2048 bits
  modulus: 17909370440277738952478021743432069490529460393959126050805622971590660765692586361919607268411957926196894593987779661515762234267984346200511739419991266042217515340629051394016939807213747552584949772262005902231184424962174373631762755583097509054166650866644125680436982880019781047492697566821940066615518316518157033101020228431401518000223024317668304190076399662045036184270398732773568597958342161459714296835776954926256734783011948914891588288449919569310898632115411415768934785779564943363123191032516179643589537811762899108471543653043311642283642184737727344704476726058553680310548513037215719011553
  public exponent: 65537
  Validity: [From: Sun Jul 05 22:36:37 IST 2020,
               To: Sat Oct 03 22:36:37 IST 2020]
  Issuer: CN=ActiveMQ Artemis Server, OU=Artemis, O=ActiveMQ, L=AMQ, ST=AMQ, C=AMQ
  SerialNumber: [    341a9018]

Certificate Extensions: 1
[1]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 19 D8 C4 B1 B1 85 FA E1   65 97 80 90 EF 6C AE 56  ........e....l.V
0010: 14 6E DA E1                                        .n..
]
]

]
  Algorithm: [SHA256withRSA]
  Signature:
0000: 44 E5 D4 3F E8 7C 15 E7   B1 E8 F0 45 AB A3 EA 77  D..?.......E...w
0010: 1D 67 3C FC 5B 95 D0 98   F1 B9 28 6C E4 FC D1 6B  .g<.[.....(l...k
0020: CD 7D 9F C3 4C D8 8F CB   E1 A1 63 24 EC E5 A3 5B  ....L.....c$...[
0030: E6 18 12 46 7D 0F E4 14   25 08 B8 3B AB AF E2 CE  ...F....%..;....
0040: A3 FE 93 8E 88 EB 97 62   59 71 3D 5D 92 D0 99 55  .......bYq=]...U
0050: 68 E3 13 1A 7E AD 41 9E   32 DD C0 58 C7 08 81 E4  h.....A.2..X....
0060: C7 8B CF 57 69 A8 0C EA   74 5F 45 8C 6E D9 E2 D1  ...Wi...t_E.n...
0070: B6 E4 B4 D2 5F D0 BD FA   BA CB 98 7B B6 1D 9D E4  ...._...........
0080: 45 DA C9 09 19 71 BD B4   7C 44 CE 6A 48 13 75 A1  E....q...D.jH.u.
0090: 6D DD DC 48 65 0E 05 3B   BA AF 63 42 2E 9F 3B 29  m..He..;..cB..;)
00A0: FE 8E 3D 07 03 04 C4 D7   65 DB 7A FA 45 C1 62 6B  ..=.....e.z.E.bk
00B0: E8 E7 D1 A7 8E 6F 0D 1D   7F 3E 83 66 75 5B 60 45  .....o...>.fu[`E
00C0: 0E BC A1 8B B3 2D 50 1A   E5 3D 02 A5 37 D6 76 DB  .....-P..=..7.v.
00D0: 5A A5 EE 26 B1 06 DC 57   F4 68 3F DA 0F C9 D4 30  Z..&...W.h?....0
00E0: BA B0 DA AD 94 24 B4 15   FF E5 DC 6E 1D 5D F9 58  .....$.....n.].X
00F0: D4 ED 31 9C 11 C0 C9 E8   29 5D F0 45 AF 93 9B CB  ..1.....)].E....

]

4. Server Key Exchange

The ‘ServerKeyExchange’ message is an optional message which is sent by the server to the client. The client uses the details received from the Server Key Exchange message to generate the pre-master secret.

This message is only required in the Elliptic Curve Diffie-Hellman (ECDH) key exchange algorithm and won’t be seen in other mechanisms such as RSA key exchange, which don't require the information from the server to generate the pre-master secret. The ECDH key exchange algorithm sends the following public key details to the client.

*** ECDH ServerKeyExchange
Signature Algorithm SHA512withRSA
Server key: Sun EC public key, 256 bits
  public x coord: 46057439580226412250665263866147684562887286117348620148026818665523810505319
  public y coord: 25930488906832570201561097323128777249979681591509199039675514975479400287814
  parameters: secp256r1 [NIST P-256, X9.62 prime256v1] (1.2.840.10045.3.1.7)

Let’s discuss the differences between these algorithms and handshakes based on them later.

5. Certificate Request

The ‘CertificateRequest’ message is the first indication we get to differentiate the one-way SSL from the two-way SSL. During the one-way SSL, the client validates the authenticity of the server. The authenticity of the client is not checked. However, in two-way SSL, both parties validate the authenticity of each other.

So how does the server validate the client?

It’s kind of the same as how the client does it.

At this step, the server sends the CertificateRequest with the details of expected certificate type, certificate signature algorithms, and certificate authorities. It’s the client's responsibility to send the certificate from its arsenal to the server matching the requirement. However, there is a wild card at this step. If the server sends an empty list of certificate authorities, depending on the client's implementation, it can decide whether to send the certificate or not.

Here is what a Certificate Request looks like.

*** CertificateRequest
Cert Types: RSA, DSS, ECDSA
Supported Signature Algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA256withDSA, SHA224withECDSA, SHA224withRSA, SHA224withDSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA
Cert Authorities:
<CN=EI Server, OU=EI, O=EI, L=EI, ST=EI, C=EI>

The client sends its certificate chain matching to the requirements sent in the Certificate Request.

*** Certificate chain
chain [0] = [
[
  Version: V3
  Subject: CN=EI Server, OU=EI, O=EI, L=EI, ST=EI, C=EI
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

  Key:  Sun RSA public key, 2048 bits
  modulus: 20633040407737278567218634330232059050042226479044722496122452505410479273023117031844484746505155564073294892298093888208868315188456270185531088712664441886338425740008444649208646876917816694847967760308554699673706712637838857175549988647511729487971524875773472297556010021422655352423126217625806863759012805189832944776623955038735096318822704111503458955521102855569127337450070103240845157800531714174178998060010010408282365398717630638484524135245861883501920626423778221475087217057744850665905219857063710661918813533021515797130163876841737728794450828239304431761997536237765605147373975769055502409267
  public exponent: 65537
  Validity: [From: Sun Jul 05 23:32:01 IST 2020,
               To: Sat Oct 03 23:32:01 IST 2020]
  Issuer: CN=EI Server, OU=EI, O=EI, L=EI, ST=EI, C=EI
  SerialNumber: [    3ded1c63]

Certificate Extensions: 1
[1]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 9C 15 A6 4E AC 34 89 2A   8D AD 4E 9A 3B 56 81 73  ...N.4.*..N.;V.s
0010: 05 A3 40 4E                                        ..@N
]
]

]
  Algorithm: [SHA256withRSA]
  Signature:
0000: 5A E1 C2 5B 71 AC 72 50   D8 DE 0E 80 10 10 A6 B4  Z..[q.rP........
0010: D6 EC 39 2E DF 24 1D 81   5E E2 39 0F C0 C2 49 CC  ..9..$..^.9...I.
0020: EE D9 5C 44 0A 07 26 23   EE 7E FC C0 32 01 7C 48  ..\D..&#....2..H
0030: 4C CB 7D 56 CB F9 42 1D   75 06 3B 92 1B 8D DE 45  L..V..B.u.;....E
0040: 4E 66 B6 5C 36 C2 07 1F   1C 7D D9 BC 09 FE F1 91  Nf.\6...........
0050: BB 6B 3D 3B F9 5F 86 FB   B1 27 33 CD D0 5C 4F 5F  .k=;._...'3..\O_
0060: 7A 9D 18 41 E3 32 BA 17   58 23 26 29 F2 A9 AE 00  z..A.2..X#&)....
0070: F7 39 64 37 89 D1 B9 D3   90 94 63 68 99 85 D7 22  .9d7......ch..."
0080: 39 E2 E8 BE BB EA 56 17   AF FC 24 14 3A 6C 23 8F  9.....V...$.:l#.
0090: E6 EC 97 A1 CA 8B 1F 7A   D7 E1 30 57 EE 92 F8 9D  .......z..0W....
00A0: 34 2E D4 CD 86 4C 19 B7   B8 3D 5B 0B 28 32 28 4D  4....L...=[.(2(M
00B0: FA AA 40 38 7D 1B 19 24   67 74 CC AD 82 CA 9E A5  ..@8...$gt......
00C0: 3B A5 36 A7 D7 B0 74 AC   96 8B 49 0B E8 5F E1 EB  ;.6...t...I.._..
00D0: 97 BB A1 DF B8 A6 8C 04   C0 1B 83 02 D2 5E F7 91  .............^..
00E0: 62 01 FC F9 35 81 4A BE   0A DF EE 78 8B 6D 9D 8E  b...5.J....x.m..
00F0: F9 3D 61 3E 93 F0 19 42   A3 7A 7F F8 B5 DF 42 F1  .=a>...B.z....B.

]
***

If the server finds a valid certificate, you will receive a ‘Found trusted certificate’ message followed by the ‘Certificate chain’ message at the server side logs.

Found trusted certificate:
[
[
  Version: V3
  Subject: CN=EI Server, OU=EI, O=EI, L=EI, ST=EI, C=EI
  Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11

  Key:  Sun RSA public key, 2048 bits
  modulus: 20633040407737278567218634330232059050042226479044722496122452505410479273023117031844484746505155564073294892298093888208868315188456270185531088712664441886338425740008444649208646876917816694847967760308554699673706712637838857175549988647511729487971524875773472297556010021422655352423126217625806863759012805189832944776623955038735096318822704111503458955521102855569127337450070103240845157800531714174178998060010010408282365398717630638484524135245861883501920626423778221475087217057744850665905219857063710661918813533021515797130163876841737728794450828239304431761997536237765605147373975769055502409267
  public exponent: 65537
  Validity: [From: Sun Jul 05 23:32:01 IST 2020,
               To: Sat Oct 03 23:32:01 IST 2020]
  Issuer: CN=EI Server, OU=EI, O=EI, L=EI, ST=EI, C=EI
  SerialNumber: [    3ded1c63]

Certificate Extensions: 1
[1]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 9C 15 A6 4E AC 34 89 2A   8D AD 4E 9A 3B 56 81 73  ...N.4.*..N.;V.s
0010: 05 A3 40 4E                                        ..@N
]
]

]
  Algorithm: [SHA256withRSA]
  Signature:
0000: 5A E1 C2 5B 71 AC 72 50   D8 DE 0E 80 10 10 A6 B4  Z..[q.rP........
0010: D6 EC 39 2E DF 24 1D 81   5E E2 39 0F C0 C2 49 CC  ..9..$..^.9...I.
0020: EE D9 5C 44 0A 07 26 23   EE 7E FC C0 32 01 7C 48  ..\D..&#....2..H
0030: 4C CB 7D 56 CB F9 42 1D   75 06 3B 92 1B 8D DE 45  L..V..B.u.;....E
0040: 4E 66 B6 5C 36 C2 07 1F   1C 7D D9 BC 09 FE F1 91  Nf.\6...........
0050: BB 6B 3D 3B F9 5F 86 FB   B1 27 33 CD D0 5C 4F 5F  .k=;._...'3..\O_
0060: 7A 9D 18 41 E3 32 BA 17   58 23 26 29 F2 A9 AE 00  z..A.2..X#&)....
0070: F7 39 64 37 89 D1 B9 D3   90 94 63 68 99 85 D7 22  .9d7......ch..."
0080: 39 E2 E8 BE BB EA 56 17   AF FC 24 14 3A 6C 23 8F  9.....V...$.:l#.
0090: E6 EC 97 A1 CA 8B 1F 7A   D7 E1 30 57 EE 92 F8 9D  .......z..0W....
00A0: 34 2E D4 CD 86 4C 19 B7   B8 3D 5B 0B 28 32 28 4D  4....L...=[.(2(M
00B0: FA AA 40 38 7D 1B 19 24   67 74 CC AD 82 CA 9E A5  ..@8...$gt......
00C0: 3B A5 36 A7 D7 B0 74 AC   96 8B 49 0B E8 5F E1 EB  ;.6...t...I.._..
00D0: 97 BB A1 DF B8 A6 8C 04   C0 1B 83 02 D2 5E F7 91  .............^..
00E0: 62 01 FC F9 35 81 4A BE   0A DF EE 78 8B 6D 9D 8E  b...5.J....x.m..
00F0: F9 3D 61 3E 93 F0 19 42   A3 7A 7F F8 B5 DF 42 F1  .=a>...B.z....B.

]
check handshake state: client_key_exchange[16]
update handshake state: client_key_exchange[16]
upcoming handshake states: certificate_verify[15](optional)
upcoming handshake states: client change_cipher_spec[-1]
upcoming handshake states: client finished[20]
upcoming handshake states: server change_cipher_spec[-1]
upcoming handshake states: server finished[20]

6. Server Hello Done

The ‘Server Hello Done’ message indicates the end of ‘Server Hello’. By this point, the server has sent over all the details required to the client and is waiting for the client's response. You will see the after the Certificate Request message.

*** ServerHelloDone

7. Client Key Exchange

At this point, the client generates a random string of bytes called ‘pre-master secret’ based on the selected key exchange method selected during the initial ‘Hello’ messages.

If RSA is used, the client generates the pre-master secret. Then the client encrypts the pre-master secret with the server's public key and transmits it to the server. Only the server has the matching private key to decrypt the message and get the pre-master secret generated by the client.

If the Diffie-Hellman is used, the required Diffie-Hellman parameters are transmitted to allow both client and server to generate the same pre-master secret.

Please be noted that the public keys are stored on digital certificates for secure transport and sharing. Hence the client receives the server's public certificate along with a Certificate Chain message.

After having the pre-master key, both sides generate the master-key, or else we can say the symmetric session key, based on it.

*** ECDHClientKeyExchange
ECDH Public value:  { 4, 155, 208, 236, 141, 18, 15, 41, 223, 46, 75, 222, 216, 4, 182, 247, 86, 18, 14, 248, 5, 121, 129, 250, 130, 84, 247, 239, 96, 115, 63, 158, 23, 130, 248, 178, 245, 86, 154, 207, 194, 82, 194, 158, 138, 4, 216, 191, 74, 185, 66, 167, 181, 231, 6, 44, 189, 8, 85, 154, 134, 217, 118, 220, 9 }
SESSION KEYGEN:
PreMaster Secret:
0000: CA 05 DD 3E 70 28 0C 66   33 52 7C CA 3C 09 80 22  ...>p(.f3R..<.."
0010: BC 09 31 C8 D4 30 0B C9   C0 0D 7E 69 0C 4E A9 BE  ..1..0.....i.N..
CONNECTION KEYGEN:
Client Nonce:
0000: 5F 02 33 15 6A 82 8F 98   99 1A 90 98 A8 FC 46 6B  _.3.j.........Fk
0010: A2 8F 62 15 BF 9D B9 3A   82 E7 A5 41 21 6E E0 5B  ..b....:...A!n.[
Server Nonce:
0000: 5F 02 33 15 CC C0 5F 2C   15 40 FD 57 0A EF 34 BE  _.3..._,.@.W..4.
0010: BE 3D BA 33 7D 10 44 8D   AA 5B E8 CD 94 80 4E B9  .=.3..D..[....N.
Master Secret:
0000: 0F 72 A0 5D A8 38 1E 2B   1C C9 09 92 15 AA CE 40  .r.].8.+.......@
0010: 22 AD DA 52 9B B9 C1 50   18 6E 0A 0E 3B C7 5D D5  "..R...P.n..;.].
0020: CE A6 6E F0 9F 07 4D 60   8D E8 6C 49 E3 DD 57 6F  ..n...M`..lI..Wo
Client MAC write Secret:
0000: 6B 3C 42 38 37 85 FE 92   CB 72 62 10 EA 78 22 33  k<B87....rb..x"3
0010: 76 D2 1D 33 A6 D3 C7 8A   83 78 F5 13 13 83 27 61  v..3.....x....'a
0020: 30 12 1D 1C E9 01 75 5D   FA 6B CE 1C 72 EE FB 9A  0.....u].k..r...
Server MAC write Secret:
0000: 85 49 6C 4D F1 9A A7 B6   C4 33 A2 6C 5D EB D0 39  .IlM.....3.l]..9
0010: 35 A7 AF 75 48 40 6E 59   F5 22 A0 37 EB B6 E9 AB  5..uH@nY.".7....
0020: A5 F5 00 29 37 82 5C 87   72 6D CD AC C8 38 D8 EF  ...)7.\.rm...8..
Client write key:
0000: 09 1D A3 90 C1 08 57 80   B5 58 16 69 F8 9E DC AB  ......W..X.i....
0010: 23 B2 47 CD 98 96 E1 27   69 64 E2 6E 68 07 EE E4  #.G....'id.nh...
Server write key:
0000: 77 76 89 F8 93 EE 40 FA   1B 8B AA 31 03 34 D1 16  wv....@....1.4..
0010: 03 33 0A F3 81 65 24 20   D7 EC 8A BE 31 7F B0 8B  .3...e$ ....1...

8. Client Change Cipher Spec

Then the client generates a ‘Change Cipher Spec’ message letting the server know that it’s going to switch to an encrypted communication based on the session key.

Thread-1 (activemq-netty-threads), READ: TLSv1.2 Change Cipher Spec, length = 1

9. Client Finished

The ‘Finished’ message is sent by the client to the server indicating that the client’s side of the handshake has been completed. The client’s ‘Finished’ message is encrypted by the shared session key and it's the first message protected by it.

*** Finished
verify_data:  { 236, 172, 15, 118, 186, 177, 150, 21, 7, 82, 236, 97 }
***

This message consists of required data that provide the guarantee for the server that the handshake hasn’t been compromised.

10. Server Change Cipher Spec

The server decrypts the received message and finds the pre-master secret sent by the client. Then it computes the session key based on the pre-master key, which will be used in the following communication.

At this point, the server generates its own ‘Change Cipher Spec’ message to indicate that it’s also switching to encrypted communication.

Thread-1 (activemq-netty-threads), WRITE: TLSv1.2 Change Cipher Spec, length = 1

11. Server Finished

At last, the server also generates a ‘Finished’ message encrypted by the session key. This marks the end of the server part of the communication as well as a successful SSL handshake.

*** Finished
verify_data:  { 183, 200, 213, 124, 127, 251, 136, 195, 116, 238, 43, 54 }
***

Once this handshake is done, the server and client now can exchange messages that are symmetrically encrypted with the shared session key.

Now you know about the general flow of SSL handshakes. But there can be some slight variations for it based on the used key exchange algorithm. There are mainly two key exchange algorithms used in TLS.

RSA key exchange
Diffie-Hellman key exchange

Let’s have a close look at both.

RSA vs Diffie-Hellman

There are some fundamental differences between Diffie-Hellman and RSA. If we put in simple words, RSA can handle both the authentication based on digital signature and the secure key exchange whereas Diffie-Hellman only handles the key exchange. Hence the RSA can be named as the more capable method from the two.

Often you will see that the Diffie-Hellman is often combined with Elliptic Curves making the Elliptic Curve Diffie-Hellman (ECDH). Including ECDH, there are four main branches of Diffie-Hellman.

Diffie-Hellman (DH)
Diffie-Hellman Ephemeral (DHE)
Elliptic Curve Diffie-Hellman (ECDH)
Elliptic Curve Diffie-Hellman Ephemeral (ECDHE)

Since the Diffie-Hellman is lacking the authentication part, it needs to be paired with a digital signature algorithm. Nowadays you will often see few modern variants of Diffie-Hellman such as ECDH and ECDHE are paired with Elliptic Curve Digital Signature Algorithm (ECDSA) or RSA.

RSA based Authentication and Key Exchange

The authentication and key exchange processes of RSA have been entangled with each other. The key exchange is performed as a part of the RSA’s authentication process.

If the RSA is used and the client receives the server certificate, a set of actions are performed to verify the validity of it.

Validates the digital signature using the public key
Check the certificate chain and verify that the server certificate is available in the client trust store or it descends from a root certificate in the trust store.
Check the validity period of the certificate. If the certificate has been expired the handshake fails.

If the certificate passes all the verifications, the client uses the public key received with the certificate to encrypt the pre-master key and send it over to the server.

What if the client receives a different certificate which is yet to be valid than the actual server certificate.

Since it’s a valid certificate, it passes the client’s verification of the certificate signature. But how does the client know that the server owns the private key that signed the certificate?

The client sends back the pre-master key encrypted with the server’s public key. Hence, it only can be decrypted using the server’s private key. If the server can decrypt and get the correct pre-master key, it validates that the server owns the private and public key pair used in the process.

That explains how the RSA based authentication happens, which lets us move for the key-exchange.

Here is how it happens,

During the Client Hello and Server Hello, both parties exchange two prime numbers called ‘client random’ and ‘server random’ respectively.
The client generates the pre-master key, encrypts it using the server’s public key, and sends it over to the server.
The server decrypts the received pre-master key using the corresponding private key.
At this point, both parties should have client random, server random, and pre-master key.
Using the aforementioned parameters, both client and server generate matching session keys.

It seems to be a good time to know how DH acts to accomplish these tasks.

Diffie-Hellman based Authentication and Key Exchange

As we have mentioned earlier, the DH cannot perform the authentication process by itself. Hence it associates with digital signature algorithms such as ECDSA or RSA.

However, please be noted that, even though RSA uses the key pair for both authentication and key exchange, the DH uses the key pair only for the authentication.

If the DH is in use, similar actions are performed as in the RSA by the client upon receiving the server certificate.

Validate the certificate signature
Check the certificate chain
Check the validity period and other relevant details

How does the client validate the key pair?

During the Client Hello and Server Hello, both parties exchange two prime numbers called ‘client random’ and ‘server random’ respectively. As we have mentioned earlier, the Server Key Exchange step is only specific to the DH handshake. Before the Server Key Exchange, the server takes the server randomly and the client random and the server's DH specific parameters and then encrypts them using the server’s private key which will work as the digital signature. Then the client uses the corresponding public key to verify that the server owns the private and public key pair used in the process.

So how does the DH key exchange happen?

Not like in RSA, the required parameters are transmitted to both parties to generate the same pre-master secret.

Here is what happens,

Both parties exchange two prime numbers called client random (x) and server random (y) respectively during the client and server ‘Hello’ messages.
The client selects a secret number called pre-master secret (a) and applies a modulo function as xa mod y.
Similarly, the server selects its pre-master secret (b) and applies a modulo function as xb mod y.
Let’s name the client’s result as A and server’s result as B.
Both parties then exchange the value and apply another modular function. The client applies Ba mod y and the server applies Ab mod y.

If the functions and values are applied correctly, both parties should have the same value which we called the session key. As we already know, the later communication between these two parties will be encrypted using the retrieved session key.

That’s it then.

Now we know the end to end details of how communication happens between client and server. We have discussed how the one-way SSL differs from the two-way SSL and how to identify it from SSL logs. Also, we have covered how the handshake varies based on the used key exchange method.

The entire process is guaranteed to establish trust between both communicating parties. In the current era, the SSL is considered to be an essential part of secure communication. It’s fascinating to see how users put their faith in this protocol about the safety of their sensitive details that are being transferred through the internet.

Stay safe! Be secure!

Sliding Pages With React.js

asaf g — Tue, 03 Jul 2018 13:39:20 +0000

Make Them Slide!

The popular way to achieve routing in React is using React Router v4. React Router works great, but it doesn't make the pages slide. Of course, you can use a package like React Router Page Transition to make that happen.

We are not going to use either of that packages in this tutorial. Instead, we will roll our own and create a sliding pages router.

This tutorial is for learning purposes, If you want to use the final code in production it would require some more work.

The complete solution is on github and you can find it here

Prerequisites

You should be comfortable with HTML, Javascript, css and React.

Step 1 - Setup

we'll use create-react-app. If you don't have React installed, install it.

Let's create our new app:

$ create-react-app react-sliding-pages

Alright! First step complete!
Feel free to delete logo.svg, we won't be using it.

You can cd into the project's directory and run npm start, so you can watch your progress.

Step 2 - Creating The Page Element

Create a new file called page.js under the src directory.

page.js


import React, { Component } from 'react';


class Page extends Component {

    constructor(props) {
        super(props);
    }


    render() {
        return (
            <div style={{
                width: '100vw', 
                height: '100%',
                position: 'fixed',
                top: 0,
                left: 0,
                transform: 'translateX('+ this.props.left + 'px)',
                animationTimingFunction: 'ease-in',
                zIndex: -20,
                transition: 'transform .8s ease-in-out'
            }}>
                {this.props.children}
            </div>
        );

    }

}

export default Page;

There's a few things to pay attention to in this component.

First of all, this is a container component — we use this.props.children to render all child components.

The other two are in the css:

The transformX property is determined by the left prop.

The transition and animationTimingFunction controls the animation and makes the sliding happen.

Now that we have our page component, we should try to use it.

Step 3 - Sliding The Page

Let's take care of some crucial styling. Change the contents of App.css to this:

App.css


.App {
  text-align: center;
}

.page {
  width: 300px;
  height: 100%;
  min-height: 5em;
  margin-right: auto;
  margin-left: auto;
  margin-top: 5em;
  padding: 2em;
  z-index: 15000;
}

Change the contents of App.js to:

App.js


import React, { Component } from 'react';
import './App.css';
import Page from './page';

class App extends Component {

  constructor(props) {
    super(props);
    this.state = {left: 0}
  }


  render () {
    return (
      <div className='App'>
        <div>
          <input type="text" onChange={(e) => this.setState({left: parseInt(e.target.value)})} />
        </div>
        <Page left={this.state.left}>
          <div className='page' style={{ backgroundColor: 'green' }}>
            bla1
          </div>
        </Page>
      </div>
    )
  }
}

export default App

Play around with the input, try typing 1000 or -500 and see what happens. If everything went well, the page should slide across the screen.

So what did we do here?

We created a state variable called left and passed it to our page component. When the input changes, it triggers the setState method and changes the page's left property.

That's nice, but it's only one page. We need an element that will wrap our page components and switch them.

Step 4 - Creating The Nav Element

This is the main part of our program and where the most logic is.
Let's create our Nav element in small parts.

We'll start with an empty component. Create a new file called nav.js under the src directory.

nav.js


import React, { Component } from 'react';


class Nav extends Component {

}

export default Nav;

The plan is to place the current page in the middle of the screen and hide the other pages on the sides of the screen. To achieve that, we'll need to know the screen width.
The left prop worked well before, so we'll use it here as well.

Let's create the constructor for the Nav component:

nav.js


...

    constructor(props) {
        super(props);
        this.width = window.innerWidth || document.documentElement.clientWidth || document.body.clientWidth;
        let left = props.children.map(o => this.width);
        if (left.length > 0 ) {
            left[0] = 0;
        }
        this.state = {page: 0, left: left};        
    }

...

We now have the screen's width in this.width, an array of integers that represents the pages' left property and a page that represents the current page number.

Now let's add the move method:

nav.js

...

    move(page) {
        const left = this.state.left.slice();
        if (page >= left.length) {
          page = 0;
        }
        for (let i = 0; i < left.length; i++ ) {
          if (i < page) {
            left[i] = -this.width;
          } else if (i === page) {
            left[i] = 0;
          } else {
            left[i] = this.width;
          }
        }    
        this.setState({left: left, page: page})
      }

...

Take a moment to read the code and understand what it does.

Basically, the code iterates on a copy of the left array and sets each page's left location according to the page number.

Let's continue to the render method.

nav.js

...

    render() {
        const pageElements = React.Children.map(this.props.children, (page, idx) =>
            React.cloneElement(page, { left: this.state.left[idx] })); 
        return (
            <div>
                {pageElements}
            </div>
        );
    }

...

Here, we use React.Children.map to iterate over the page elements and add the left prop.

Now let's use our new Nav element in App.js.

App.js


import React, { Component } from 'react';
import './App.css';
import Page from './page';
import Nav from './nav';

class App extends Component {

  constructor(props) {
    super(props);
  }



  render() {
    return (
      <div className="App">
        <Nav>
          <Page>
            <div className="page" style={{ backgroundColor: 'green' }}>
              bla1
            </div>
          </Page>
          <Page>
            <div className="page" style={{ backgroundColor: 'red' }}>
              bla1
            </div>
          </Page>
          <Page>
            <div className="page" style={{ backgroundColor: 'blue' }}>
              bla1
            </div>
          </Page>
          <Page>
            <div className="page" style={{ backgroundColor: 'brown' }}>
              bla1
            </div>
          </Page>
        </Nav>

      </div>
    );
  }
}

export default App;

In case everything worked well, you should see the first page rendered in your browser. You can check the html source (using the browser's dev tools) and see that all the child elements are indeed rendered.

Alright, so we have all the pages but no access to the move method. Let's add some buttons temporarily just to see that the slide works.

nav.js

...

    render() {
        const pageElements = React.Children.map(this.props.children, (page, idx) =>
            React.cloneElement(page, { left: this.state.left[idx] })); 
        const buttonElements = React.Children.map(this.props.buttons, (button, idx) => {
            let newButton = React.cloneElement(button , { onClick: () => this.move(idx), ...button.props});
            return newButton;
        });
        return (
            <div>
                <div>
                    {buttonElements}
                </div>
                <div>
                    {pageElements}
                </div>
            </div>
        );
    }

...

App.js

...

  render() {
    let buttons = [];
    for (let i = 0; i < 4; i++) {
      let button = <button key={i.toString()}>{i}</button>;
      buttons.push(button);
    }
    return (
      <div className="App">
        <Nav buttons={buttons}>
          <Page>
            <div className="page" style={{ backgroundColor: 'green' }}>
              bla1
            </div>
          </Page>
          <Page>
            <div className="page" style={{ backgroundColor: 'red' }}>
              bla1
            </div>
          </Page>
          <Page>
            <div className="page" style={{ backgroundColor: 'blue' }}>
              bla1
            </div>
          </Page>
          <Page>
            <div className="page" style={{ backgroundColor: 'brown' }}>
              bla1
            </div>
          </Page>
        </Nav>

      </div>
    );
  }

...

Try the buttons in your browser, the pages should slide.

Now that the basic functionality works, let's make our Nav element more router like.

Step 5 - Implement Routing

First, remove the buttons we used for testing. We don't need them.
With that out of the way, create a new file called link.js under the src directory.

link.js


import React, { Component } from 'react';


class Link extends Component {

    go() {
        window.location.hash = this.props.to.slice(1);
    }

    render() {
        return (
            <i onClick={() => this.go()}>
                {this.props.children}
            </i>

        );
    }

}

export default Link;

This component is pretty straight forward, it renders the component's children and changes the hash on a click according to the to prop.

Let's make our Nav component react to hash changes.

nav.js

...

    constructor(props) {
        super(props);
        this.width = window.innerWidth || document.documentElement.clientWidth || document.body.clientWidth;
        let left = props.children.map(o => this.width);
        if (left.length > 0 ) {
            left[0] = 0;
        }
        this.state = {page: 0, left: left, from: left};
        this.pages = {};
    }

...

    componentDidMount() {
        this.move(this.pages[window.location.hash.slice(1)]);
    }

    componentWillMount() {
        window.onhashchange = (e) => {
            const i = e.newURL.indexOf('#');
            const hash = e.newURL.slice(i + 1);
            this.move(this.pages[hash]);
        };
    }


    render() {
        const pageElements = React.Children.map(this.props.children, (page, idx) => {
            const element = React.cloneElement(page, { left: this.state.left[idx], from: this.state.from[idx], eventer:this.eventer }); 
            this.pages[element.props.path.slice(1)] = idx;
            return element;
        });
        return (
            <div>
                {pageElements}
            </div>
        );
    }

...

As you can see we, are registering the pages to the pages prop, so we can know which route belongs to which page.

And finally use our new links in App.js.

App.js


import React, { Component } from 'react';
import './App.css';
import Page from './page';
import Nav from './nav';
import Link from './link';

class App extends Component {

  render() {
    return (
      <div className="App">
        <div>
          <Link to="/green" >
            <button>green</button>
          </Link>
          <Link to="/red" >
            <button>red</button>
          </Link>
          <Link to="/blue" >
            <button>blue</button>
          </Link>
          <Link to="/brown" >
            <button>brown</button>
          </Link>
        </div>
        <Nav>
          <Page path="/green">
            <div className="page" style={{ backgroundColor: 'green' }}>
              bla1
            </div>
          </Page>
          <Page path="/red">
            <div className="page" style={{ backgroundColor: 'red' }}>
              bla1
            </div>
          </Page>
          <Page path="/blue">
            <div className="page" style={{ backgroundColor: 'blue' }}>
              bla1
            </div>
          </Page>
          <Page path="/brown">
            <div className="page" style={{ backgroundColor: 'brown' }}>
              bla1
            </div>
          </Page>
        </Nav>

      </div>
    );
  }
}

export default App;

Alright! We have a sliding router! But wait...

The pages that are off screen are still rendered. This might be good for you, if your app is pretty much static. But as we know, React components might do a whole lot of stuff.. Use the the network, do some calculations, animations... etc. We need some way to disable the component that are off screen.

Step 6 - Only Render What Is Visible

This part is a bit tricky, we need the pages visible for the sliding effect but we don't want them to be rendered when they are off screen. For example, if the user goes from page 5 to page 1, we need to render all the pages for the sliding effect and then only render page 1.

We'll use an event mechanism, that will fire every time the page changes.

Add another class to nav.js .

nav.js

...

class MoveEvent {

    constructor() {
        this.listeners = [];
    }

    register(f) {
        this.listeners.push(f);
    }

    unregister(f) {
        this.listeners.splice(this.listeners.indexOf(f), 1);
    }

    fire(params) {
        this.listeners.forEach((listener) => {
            listener(params);
        });

    }
}


...

This is a pretty standard event class. We will also add a from prop, so that the page element will be able to tell if it's in transition or not.
Let's pass the props to our page components from our Nav component.

nav.js

...

class Nav extends Component {

    constructor(props) {
        super(props);
        this.width = window.innerWidth || document.documentElement.clientWidth || document.body.clientWidth;
        let left = props.children.map(o => this.width);
        if (left.length > 0 ) {
            left[0] = 0;
        }
        this.state = {page: 0, left: left, from: left};
        this.pages = {};
        this.eventer = new MoveEvent();
    }

    move(page) {
        const left = this.state.left.slice();
        const from = left.slice();
        if (page >= left.length) {
          page = 0;
        }
        for (let i = 0; i < left.length; i++ ) {
          if (i < page) {
            left[i] = -this.width;
          } else if (i === page) {
            left[i] = 0;
          } else {
            left[i] = this.width;
          }
        }    
        this.setState({left: left, page: page, from: from});
        this.eventer.fire();
      }

    componentDidMount() {
        this.move(this.pages[window.location.hash.slice(1)]);
    }

    componentWillMount() {
        window.onhashchange = (e) => {
            const i = e.newURL.indexOf('#');
            const hash = e.newURL.slice(i + 1);
            this.move(this.pages[hash]);
            this.eventer.fire();
        };
    }


    render() {
        const pageElements = React.Children.map(this.props.children, (page, idx) => {
            const element = React.cloneElement(page, { left: this.state.left[idx], from: this.state.from[idx], eventer:this.eventer }); 
            this.pages[element.props.path.slice(1)] = idx;
            return element;
        });
        return (
            <div>
                {pageElements}
            </div>
        );
    }

}


...

Great, Now let's make the page understand if it needs to render children or not.

page.js


import React, { Component } from 'react';

function sleep(ms) { // just a sleep util
    return new Promise(resolve => setTimeout(resolve, ms));
}

class Page extends Component {

    constructor(props) {
        super(props);
        this.state = {shouldRenderChildren: true};
    }

    isVisible() {
        if (this.props.left < 0) {
            return false;
        }
        if (this.props.left >= window.innerWidth) {
            return false;
        }
        return true;
    }

    move() {
        if (this.props.left !== this.props.from) {
            this.setState({ shouldRenderChildren: true });
        }
        this.reRender();
    }

    async reRender() {
        await sleep(1000);
        this.setState({shouldRenderChildren: this.isVisible()});
    }

    render() {
        return (
            <div
             style={{
                width: '100vw', 
                height: '100%',
                position: 'fixed',
                top: 0,
                left: 0,
                transform: 'translateX('+ this.props.left + 'px)',
                animationTimingFunction: 'ease-in',
                zIndex: -20,
                transition: 'transform .8s ease-in-out'
            }}> {this.state.shouldRenderChildren &&
                    this.props.children
                 }
            </div>
        );

    }

}

export default Page;

...

The isVisible method returns whether or not the page is off screen. The reRender method sleeps for 1 second and updates the shouldRenderChildren state variable. When move is called the element will be fully rendered for 1 second and then it will disappear if it's off screen.

Now we just need to register to the move event.

page.js

...

    componentDidMount() {
        this.props.eventer.register(() => this.move());
    }

    componentWillUnmount() {
        this.props.eventer.unregister(() => this.move());
    }

...

Awesome! Check your browser dev tools and see that off screen pages are not rendering any children.

Step 7 (optional) - Customizing And Extending The Code

There's a chance that your app needs the pages to behave differently.

For example, you can render each page's child components and pass them a prop telling them if they are visible or not. Whatever comes to mind really.

That part is totally up to you!

If you are extending or customizing the code, I would love to hear about your ideas and implementations.

Summary

We created our own router with sliding pages, that's pretty cool. Of course our router is no match for React Router, which is much more fancy and also battle tested. Rolling your own can be a huge pain sometimes(no one likes reinventing the wheel), but it has a big advantage - you can code exactly what your app needs.

I hope you enjoyed, and of course, feel free to comment or share.

Thank you for reading.