DEV Community: MrSrv7

Serverless Computing

MrSrv7 — Sun, 05 Mar 2023 18:21:01 +0000

Imagine running an e-commerce platform like Amazon, Flipkart, etc., that needs to handle massive traffic during a holiday shopping event. You must ensure your platform can handle the traffic surge while maintaining high performance and reliability. This might seem daunting, but with the power of Serverless Computing, you can easily scale your application to meet the demands of your customers. Let's explore the same.

Intro

So, what is Serverless Computing anyway? Serverless computing is a cloud computing model in which the cloud provider manages the infrastructure and automatically allocates resources to support applications as needed. In other words, you don't have to worry about provisioning servers, maintaining the operating system, or handling any underlying infrastructure. You only need to focus on writing code for your application's business logic.

Imagine you run an e-commerce website that sells handmade crafts. During the holiday season, there's a sudden spike in traffic and sales, which means you need more computing power to handle the increase in traffic. However, setting up and managing servers to handle this increased demand can be challenging and time-consuming.

With serverless computing, you don't have to worry about managing servers or infrastructure. You can focus on building your e-commerce site, leaving the server management and scaling to a serverless platform like AWS Lambda or Google Cloud Functions. This means your site can handle sudden spikes in traffic without worrying about server crashes or slowdowns.

Working Model of Serverless Computing

In a traditional web hosting environment, a server constantly runs, waiting for requests. With serverless computing, the cloud provider automatically starts and stops instances of your application based on incoming requests. When a request comes in, the cloud provider spins up a new instance of your application and runs it until it is fulfilled. Once the request is complete, the cloud provider shuts down the instance, freeing up resources for other applications.

Serverless computing works on the principle of "Functions as a Service" (FaaS), where the application is broken down into smaller functions executed independently. HTTP requests, file uploads, database updates, and other events trigger these functions.

When an event occurs, the provider spins up a new function instance to handle the event. The provider then allocates the required resources to the function and runs it. Once the function has completed its task, the resources are released back to the provider, ensuring that the resources are not wasted when they are not in use.

Benefits of Serverless Computing

Scalability

One of the key benefits of serverless computing is scalability. Because the cloud provider automatically allocates resources, your application can easily scale up or down based on demand. This means you can handle spikes in traffic without worrying about provisioning additional servers or capacity.

Reduced Costs

Serverless computing can result in significant cost savings compared to traditional hosting models. With serverless computing, you only pay for the resources you use, so you don't have to pay for idle server time. Additionally, you don't have to worry about maintaining and updating servers, which can be a significant time and cost investment.

Faster Development and Deployment

With serverless computing, developers can focus on coding the application's business logic rather than worrying about the underlying infrastructure's maintenance. This allows for faster development and deployment of applications.

Flexibility

Serverless computing allows developers to use their chosen programming languages and tools. This provides greater flexibility and can help developers write code more efficiently.

Performance

Serverless computing can improve the performance of applications, as the serverless platform can automatically scale resources up or down based on demand. This ensures that the application always has the necessary resources to handle user requests, leading to improved performance. Additionally, serverless computing platforms are designed to handle failures and automatically recover from them. This can help ensure that applications are always available to end users.

Use Cases for Serverless Computing

Backend processing: Serverless functions can process data and perform backend tasks such as file processing, image resizing, and data analytics.
API development: Serverless functions can be used to develop APIs for web applications, providing a scalable and cost-effective way to handle user requests.
Real-time data processing: Serverless functions can process real-time data streams from web applications like chat applications or IoT devices.
Event-driven architectures: Serverless computing is well-suited for event-driven architectures, where functions are triggered by specific events such as user actions or system events.
DevOps automation: Serverless computing can automate DevOps tasks such as continuous integration and deployment, reducing the need for manual intervention and improving overall efficiency.

Limitations

Like any technology, serverless computing has challenges and drawbacks that developers should know before adopting for their applications.

Cold start issues: Serverless functions can have a "cold start" delay when invoked, resulting in slower response times for the first user request.
Debugging challenges: Debugging serverless functions can be more challenging than traditional applications, as there is less visibility into the underlying infrastructure.
Vendor lock-in: Moving away from a serverless provider can be challenging due to proprietary APIs and services, leading to vendor lock-in.
Performance limitations: Serverless computing may not be suitable for high-performance applications that require dedicated hardware or resources.
Cost uncertainty: The cost of serverless computing can be difficult to predict, as it is based on usage and can vary significantly depending on application demands.

Usage

Consider if you want to send a welcome email to anyone who just signed up for your site. Configuring SendGrid or any other E-Mail delivery service from scratch with a custom server application and accessing it programmatically might be challenging for beginners. This can be solved by using a serverless function to automate and access the function programmatically.

Please check out this post to learn more about how to setup AWS's SES and Lambda functions to send emails technically

This is one of the vast options/functionalities we can achieve using serverless computing. We didn't create our custom server application to set up email-delivering functionality and configuration. Instead, we made a function that will trigger the email deliveries, and we are using that function in our application directly.

Conclusion

Serverless computing is a powerful paradigm that allows developers to build and run applications without worrying about server management. AWS Lambda, Azure Functions, and Google Cloud Functions are popular serverless platforms that you can use to get started with serverless computing.

AWS SES with NodeJS - Send E-Mails with Amazon AWS's SES utility and Lambda functions.

MrSrv7 — Sun, 05 Mar 2023 18:15:00 +0000

If you are a beginner, it might not be easy to set up an E-Mail delivery service like SendGrid from scratch and access it programmatically in your NodeJS Server application or even a ReactJS frontend application. Let's discuss how to set up the same with AWS's SES and Lambda functions to deliver a welcome email to new users signing up for our application.

SES

If you are unaware of SES, Amazon's Simple Email Service (SES) is a cloud-based email service provided by Amazon Web Services (AWS) that enables businesses to send and receive emails. SES allows developers to send bulk emails to their users with high deliverability. It provides a reliable, cost-effective solution for managing transactional emails such as welcome emails, password reset emails, order confirmations, and other automated emails.

AWS Setup

If you are new to AWS, you should create an Access Key (like an API key) to use AWS SDK's functions from your NodeJS functions after creating an account. Go to the Security Credentials tab in AWS and create an Access Key for the IAM user or root user.

Note: AWS doesn't recommend creating root user access keys. Because you can't specify the root user in a permissions policy, you can't limit its permissions, which is a best practice. Instead, use alternatives such as an IAM role or a user in IAM Identity Center, which provide temporary rather than long-term credentials. If your use case requires an access key, create an IAM user with an access key and apply the least privilege permissions for that user.

As this is for educational purposes, I will create an Access Key for the root user and proceed with that.

AWS SES Setup

After creating the Access Key, search for SES in the AWS search bar and select Amazon Simple Email Service from the results (I cannot provide the direct link as it will vary according to the region of your account).

In SES, select create identity. Enter the from address (from which the emails will be sent to the users). You can verify the domain if you plan to use SES for production. For the sake of this article, I will create an identity with my email address and verify it.

After creating the identity, we will receive a verification email from AWS to authenticate the sender's identity. Click on the link in the email and authenticate the same.

Serverless Lambda function

Now that we have authenticated the from identity, we can set up our serverless lambda function that will trigger and act as an API through which we can programmatically send emails to recipients.

We must create a custom role for our lambda function to access AWS's SES to send emails. Please follow AWS SES Setup for the same.

Click on Create Function. Select the Author from Scratch option, give the function a name, and select the architecture and run time (in our case, NodeJS). Scroll down to the code section and enter the following code by selecting index.js



const AWS = require('aws-sdk');
const ses = new AWS.SES({ region: 'ap-south-1' });

exports.handler = async (event) => {
  const params = {
    Source: '', // replace with your verified email address
    Destination: {
      ToAddresses: [] // replace with the recipient's email address
    },
    Message: {
      Subject: {
        Data: 'Test email from Lambda' // replace with your email subject
      },
      Body: {
        Text: {
          Data: 'This is a test email sent from a Lambda function.' // replace with your email body
        }
      }
    }
  };

  try {
    const result = await ses.sendEmail(params).promise();
    console.log(result);
    return result;
  } catch (err) {
    console.log(err);
    throw err;
  }
};

After entering the code, click on Deploy. Once the changes are deployed successfully, click on Test to verify the functionality.

Check out this for GIF version for setting up the Lambda function

Now we can programmatically access this AWS Lambda function even from your ReactJS application or our own NodeJS server application like



const AWS = require("aws-sdk");

AWS.config.update({
  accessKeyId: "accessKey",
  secretAccessKey: "secretAccessKey",
  region: 'region'
});

const lambda = new AWS.Lambda();

const functionName = "welcomeEmail";

const payload = {};

const params = {
  FunctionName: functionName,
  Payload: JSON.stringify(payload),
};

lambda.invoke(params, function (err, data) {
  if (err) {
    console.error(err);
  } else {
    console.log(data);
  }
});

Once we run this code with node filename.js, we see no errors, and the email has been delivered to our inbox.

Output

Mail Inbox

View the GitHub repository here: MrSrv7/aws-ses-lambda

Conclusion

This way, we can make a Serverless AWS Lambda function to send emails via AWS's SES programmatically. This is just one of the many advantages of Serverless functions. With the help of serverless functions, we can have our custom functions for custom functionalities without creating a server application from scratch and configuring and maintaining the same. To know more about Serverless functions, checkout this post.

Frontend Development - How do the UI Applications work?

MrSrv7 — Fri, 17 Feb 2023 12:38:07 +0000

So when I was surfing Reddit, I came across what a framework like Vue does behind the scene. that intrigued me. As there is a lot of information on how a framework/library handles the Dynamic UI, I thought to write up. This will serve as a starting point for anyone relatively new and trying to understand what is happening behind the code they type and a memory refresher for season veterans.

Intro

Have you ever wondered how websites become interactive and dynamic, responding to user actions and displaying information seamlessly? A lot has changed since the early days of static HTML pages. Today, building a website requires a deep understanding of the design and the functionality that powers the user experience. Even though there are drag-and-drop website builders like WordPress, SquareSpace, etc., for people from non-IT backgrounds, yet building web applications by profession requires the developer to have profound knowledge of the same. To build dynamic and interactive websites that are more accessible and more efficient, frontend frameworks such as Angular and VueJS have been developed.

If you are wondering about ReactJS, it is not a full-blown framework. It is a JavaScript library to create UI. Still, it is equivalent to a framework as it gets the job done.

Frontend Development

Frontend development refers to creating and implementing a website or web application's User Interface (UI) and User Experience (UX). It involves working with technologies such as HTML, CSS, and JavaScript to create visually appealing and interactive web pages. The primary focus of front-end development is to enhance the user experience by making web pages visually appealing, interactive, and user-friendly.

A front-end developer creates a website or web application's look, feel, and behavior. They work closely with designers to bring the design to life and with backend developers to ensure that the web application doesn’t provide outdated content by integrating it appropriately.

There are a lot of frontend frameworks like Angular, VueJS, SvelteJS, etc., and Libraries like ReactJS, etc.,

Frontend Frameworks

As web applications grow in complexity, the need for efficient and scalable front-end development solutions has become increasingly apparent. Frontend frameworks provide tools and components that make developing complex and dynamic web applications easier. They offer a simplified and consistent way of working with HTML, CSS, and JavaScript and provide a range of functionalities, including state management, DOM manipulation, and component-based architecture.

These frameworks/libraries handle many of the complex and repetitive tasks involved in front-end development, freeing up the developer to focus on the unique and creative aspects of the project. They define both the HTML template and JavaScript behavior of UI elements, with templates providing the HTML structure and including variables, bindings, and expressions that are dynamically evaluated at runtime. Reactive data binding allows developers to bind the application state to the UI in a declarative manner, ensuring that changes in the state are automatically reflected in the UI and vice versa. State management is responsible for keeping track of the values of variables, user interactions, and other data affecting the application's behavior. Event handling handles user interactions and other events, while routing manages navigation between different pages or views.

Working model of the Frontend frameworks

So, now that we know what a frontend library or a framework is, let’s discuss the working model of the same. At its core, a frontend framework like VueJS is an intermediary between the user interface and the underlying data and logic. It provides a set of abstractions and tools that make it easier for developers to manage the DOM (Document Object Model), which is the hierarchical representation of the content and structure of a web page.

One can argue that a web application can be built with HTML, CSS, and JavaScript without external libraries or frameworks. Yes, it is possible. But, maintaining the codebase for an extensive application, better understanding, and debugging will become tedious. Consider a simple to-do list application that allows users to add, edit, and delete items. With a frontend framework like VueJS, instead of manipulating the DOM to update the list of items manually, the developer can update a JavaScript object that represents the state of the application, and the framework will take care of automatically update the DOM.

Herein after, VanillaJS will refer to plain/pure JavaScript

VanillaJS implementation for rendering a list of todos and adding a todo item

<!DOCTYPE html>
<html>
  <head>
    <script type="text/javascript">
      var todoList = [];

      function addTodo() {
        var todoInput = document.getElementById("todoInput").value;
        todoList.push(todoInput);
        renderTodos();
      }

      function renderTodos() {
        var todoListElem = document.getElementById("todoList");
        todoListElem.innerHTML = "";
        for (var i = 0; i < todoList.length; i++) {
          todoListElem.innerHTML += "<li>" + todoList[i] + "</li>";
        }
      }
    </script>
  </head>
  <body>
    <input type="text" id="todoInput" />
    <button onclick="addTodo()">Add Todo</button>
    <ul id="todoList"></ul>
  </body>
</html>

VueJS implementation of rendering a list of todos and adding a todo item

<div id="app">
  <input type="text" v-model="newTodo" />
  <button @click="addTodo">Add Todo</button>
  <ul>
    <li v-for="todo in todos">{{ todo }}</li>
  </ul>
</div>

<script src="https://cdn.jsdelivr.net/npm/vue/dist/vue.js"></script>
<script>
var app = new Vue({
  el: "#app",
  data: {
    newTodo: "",
    todos: []
  },
  methods: {
    addTodo: function() {
      this.todos.push(this.newTodo);
      this.newTodo = "";
    }
  }
});
</script>

We can see from the above code the VueJS code is more concise and organized, making it easier to understand and maintain. With VueJS, we can bind data to the view using directives like v-model and v-for, making it easier to keep the view and data in sync. The code also uses an object-oriented approach to manage the state and behavior of the application, as opposed to using imperative code to manipulate the DOM in the Vanilla HTML and Vanilla JS examples.

The DOM API is a low-level API that allows developers to interact with and manipulate the content and structure of a web page. It provides a set of methods for accessing and modifying the elements of a web page, such as adding and removing elements, changing the styles, and more. While the DOM API is powerful, it can also be complex and cumbersome, especially for more complex applications. This is where front-end frameworks like VueJS come in. They encapsulate the DOM API and provide a higher-level API that is more developer-friendly and abstracts away much of the complexity of working with the DOM. Let’s discuss some of the key concepts of the working model compared with the Vanilla JS approach.

Templating:

In VueJS, a template is a string or HTML-like syntax representing your application's structure. The template is compiled into a render function that generates a Virtual DOM. The Virtual DOM is a lightweight representation of the actual DOM, which VueJS uses to optimize updates and improve performance.

<!-- VueJS template -->
<div id="app">
  <h1>{{ message }}</h1>
</div>

In VanillaJS, we manually create and manipulate the actual DOM using APIs like document.createElement() and appendChild(). Here's an example of creating an element and appending it to the document using VanillaJS:

// Vanilla JS code
const appDiv = document.createElement('div');
const h1 = document.createElement('h1');
const text = document.createTextNode('Hello, Vanilla JS!');
h1.appendChild(text);
appDiv.appendChild(h1);
document.body.appendChild(appDiv);

As you can see, the VanillaJS code is more verbose and requires more code to achieve the same result.

Reactive Data binding:

In VueJS, you can define reactive data properties in the data object. VueJS automatically updates the virtual DOM when reactive property changes and re-renders the affected components.

// VueJS app
const app = new Vue({
  el: '#app',
  data: {
    message: 'Hello, VueJS!',
  },
});

In VanillaJS, we manually create event listeners and update the DOM when a data property changes. Here's an example of creating an event listener and updating the DOM using VanillaJS:

// Vanilla JS code
const h1 = document.querySelector('h1');
let message = 'Hello, Vanilla JS!';
h1.textContent = message;

const button = document.querySelector('button');
button.addEventListener('click', () => {
  message = 'Hello, Vanilla JS! (updated)';
  h1.textContent = message;
});

As you can see, the vanilla JS code is more complex and requires more code to achieve the same result.

Components

In VueJS, you can define components as reusable building blocks for your application. Each component encapsulates its state, props, and behavior, making it easier to reason and test.

<!-- VueJS component -->
<template>
  <div>
    <h1>{{ title }}</h1>
    <p>{{ message }}</p>
  </div>
</template>

<script>
export default {
  data() {
    return {
      title: 'Hello, VueJS!',
      message: 'This is a VueJS component',
    };
  },
};
</script>

In VanillaJS, we manually define and organize our code into modules and use techniques like the Revealing Module Pattern to encapsulate state and behavior. Here's an example of defining a module using vanilla JS:

// Vanilla JS code
const module = (function() {
  let title = 'Hello, Vanilla JS!';
  let message = 'This is a Vanilla JS module';

  function setTitle(newTitle) {
    title = newTitle;
    update();
  }

  function setMessage(newMessage) {
    message = newMessage;
    update();
  }

  function update() {
    const h1 = document.querySelector('h1');
    const p = document.querySelector('p');
    h1.textContent = title;
    p.textContent = message;
  }

  return {
    setTitle,
    setMessage,
  };
})();

Lifecycle Methods (Hooks)

In VueJS, you can define lifecycle hooks to perform actions at specific stages of the component's lifecycle, such as when it is created, mounted, updated, or destroyed.

// VueJS component with lifecycle hooks
export default {
  data() {
    return {
      message: 'Hello, VueJS!',
    };
  },
  created() {
    console.log('Component created');
  },
  mounted() {
    console.log('Component mounted');
  },
  updated() {
    console.log('Component updated');
  },
  destroyed() {
    console.log('Component destroyed');
  },
};

To achieve this functionality, we can follow the below approach in VanillaJS.

class MyComponent {
  constructor() {
    console.log('Component is constructed');
  }

  connectedCallback() {
    console.log('Component is connected');
    this.innerHTML = '<div>Hello, Vanilla JS!</div>';
  }

  disconnectedCallback() {
    console.log('Component is disconnected');
  }

  attributeChangedCallback(name, oldValue, newValue) {
    console.log(`Attribute "${name}" has changed from "${oldValue}" to "${newValue}"`);
  }

  adoptedCallback() {
    console.log('Component is adopted');
  }
}

// Define the new custom element
customElements.define('my-component', MyComponent);

We're creating a custom element using the customElements API, which allows us to define a new HTML element that can be used in our markup. The MyComponent class represents the implementation of the new custom element, and it defines various methods that are part of the custom element's lifecycle. To know more about the customElements, refer to https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_custom_elements)

Here are the VanillaJS lifecycle methods that are defined in the example above:

constructor: This method is called when a new instance of the custom element is created. It's similar to the VueJS beforeCreate hook, allowing you to perform initialization and setup before the element is connected to the DOM.
connectedCallback: This method is called when the custom element is added to the DOM. It's similar to the VueJS mounted hook, allowing you to perform a setup that requires DOM access, such as adding event listeners or updating the component's content.
disconnectedCallback: This method is called when the custom element is removed from the DOM. It's similar to the VueJS beforeDestroy hook, allowing you to perform cleanup tasks, such as removing event listeners or freeing up resources.
attributeChangedCallback: This method is called when an attribute of the custom element is changed. It allows you to respond to changes in the element's properties and update its content accordingly. This method is similar to the VueJS updated hook, but it only applies to changes in attributes rather than changes in the component's data.
adoptedCallback: This method is called when the custom element is moved to a new document. It's similar to the VueJS mounted hook but only relevant in rare cases where the custom element moves between documents.

These are a few important topics/concepts that help us understand what happens behind the code we write when we use a framework like VueJS and how the respective framework/library handles the dynamic nature of a front-end application. And I just took VueJS for a quick comparison. Generally, any framework or library might follow this approach to provide an easy, manageable, and scalable developer experience and architecture.

Conclusion

In conclusion, utilizing a front-end framework like VueJS can significantly simplify the development process and provide advanced features compared to writing everything in Vanilla HTML and VanillaJS. However, it is essential to understand the fundamental concepts and principles of front-end development to make the most out of these tools. Whether for personal or professional purposes, learning front-end development can lead to exciting and creative opportunities in the digital world.

Generate a Non Repetitive One Time Password (OTP) in Vanilla JS

MrSrv7 — Fri, 10 Feb 2023 00:38:56 +0000

Have you wondered how to write a simple JavaScript (Vanilla JS) function to generate text or numbers? In this case, a JavaScript function that generates a nonrepetitive OTP. There might be libraries and frameworks for your backend services to make use of, for the functionality of verifying the users or similar. Yet, you can be asked in an interview to write a code for the same, or your project's bundle size might be a critical factor, so you might not be able to include a separate package just for generating text or numbers. So, let's discuss how to achieve the same via Vanilla JS.

Code

const generateOtp = (otpLength = 6) => {
  let otpCode = "";
  const possibleDigits = "0123456789";
  while (otpCode.length < otpLength) {
    let currentDigit = possibleDigits.charAt(
      Math.floor(Math.random() * possibleDigits.length)
    );
    if (!otpCode.includes(currentDigit)) {
      otpCode += currentDigit;
    }
  }
  return otpCode;
};

console.log(`Your OTP is ${generateOtp()}`); // As the function is called without any parameters, the default value of 6 will be considered as the length of the OTP

console.log(`Your OTP is ${generateOtp(4)}`); // For four digit OTP

Preview in CodeSandBox

Explanation

First, create a function named generateOTP and pass in a required parameter otpLength.
A default value of 6 is assigned for OTP length if no parameter is passed.
Declare a variable otpCode that will be returned when the function is called.
Declare a variable possibleDigits that holds the digits/characters to be considered for generating the OTP.
The while loop is executed until otpCode.length is less than otpLength. That is, the loop will execute till it generates the otpLength of numbers (in this case, its six digits)
Within the loop, generate a random character from the possibleDigits string using possibleDigits.charAt(Math.floor(Math.random() * possibleDigits.length)) and store the index in a temporary variable currentDigit
Check if the generated character exists in otpCode using if (!otpCode.includes(currentDigit)).
If it doesn't exist, add the generated character to the otpCode string using otpCode += currentDigit.
The function returns the generated OTP using the return otpCode statement.;

Output

You can make use of such functions and make multiple generators for your use case, especially with the help of regex (regular expressions) to have ultimate customized generator functions without any 3rd party library. This approach helps you to understand the underlying concept and will help you to come up with similar solutions to similar questions in interviews, coding rounds, etc. when attending interviews.

Do suggest me for any such topics to cover in future. Do let me know in the comments if this helped you at any situation.

Securing Your Data with Queryable Encryption in MongoDB 6.0.3

MrSrv7 — Fri, 20 Jan 2023 03:37:38 +0000

Recently, I got a mail from the MongoDB team mentioning the 6.0.3 release notes. In the mail, they mentioned Queryable Encryption, which quickly attracted my attention. Even though this preview version was released on June 7, 2022, I learned about the same only via the mail. As a Fullstack Developer, primarily working with MERN (MongoDB, ExpressJS, ReactJS, NodeJS) Stack in most projects, this interests me as this helps protect the data from potential attacks and, as they claim,

MongoDB is the only database provider that allows customers to run expressive queries, such as equality (available now in preview) and range, prefix, suffix, substring, and more (coming soon) on fully randomized encrypted data.

Hence, I would like to share my views of the same.

TL; DR

Unlocking Powerful Queries on Encrypted Data: Run a range of query expressions like range, equality, prefix, suffix, and more with MongoDB's Queryable Encryption, making it the only database provider that offers this capability.

Total Data Protection: Queryable Encryption protects sensitive data throughout its lifecycle, from transit to storage, with added server-side randomized encryption for added security.

Meeting Strict Data Privacy Standards: Queryable Encryption enables compliance with strict data privacy regulations such as HIPAA, GDPR, CCPA, etc., through strong technical controls like client-side encryption/decryption and key management for complete control over data confidentiality and integrity.

Efficient Application Development: Queryable Encryption simplifies data protection for developers with built-in, standard-based cryptography and key management, eliminating the need for a separate SDK and supporting popular MongoDB drivers.

Lowering Institutional Risk: Queryable Encryption in MongoDB Atlas allows for secure storage of sensitive data in the cloud with rich querying capabilities on fully randomized encrypted data, reducing institutional risk.

Intro

With the increasing number of data breaches and cyber-attacks, securing sensitive data has become a top priority for organizations of all sizes. MongoDB 6.0.3 offers a powerful new feature to help organizations protect their data: Queryable Encryption. In this post, we'll take a closer look at Queryable Encryption and how it can help organizations secure their data stored in MongoDB.

What is Queryable Encryption?

Queryable Encryption is a feature that allows for the encryption of data stored in MongoDB while still allowing for the execution of queries on that data. This means that data is encrypted using a specific encryption key, and the encrypted data is stored in the database. When a query is executed, the query is executed against the encrypted data, and the results are returned in an encrypted format. This allows organizations to maintain the confidentiality of their data while still performing meaningful analysis and gaining insights from it.

Why do we need Queryable Encryption in the first place?

Before the introduction of Queryable Encryption, organizations faced several challenges regarding protecting sensitive data. One of the main issues was that traditional encryption methods made it difficult or impossible to query or search encrypted data. This meant that organizations had to trade between the security of their data and their ability to access and use it.

Another issue was that traditional encryption methods did not provide enough control over the encryption keys, increasing the risk of data breaches. And often, implementing the encryption required expertise in cryptography and a separate SDK, which made it difficult for developers to protect data properly.

Additionally, due to the large amount of data generated and stored, it's become increasingly important to store data in the cloud, but this can raise concerns about the security of the data.

How does Queryable Encryption address the issues?

Queryable Encryption addresses the issues faced before its introduction by allowing for data encryption so that it can still be queried, searched, and indexed without compromising the security of the data. It achieves this by using standard-based cryptography and strong key management, which eliminates the need for a separate SDK and allows developers to protect data without needing to be experts in cryptography.

Queryable Encryption also provides added layers of security, such as client-side encryption/decryption and key management, which gives organizations complete control over their data's confidentiality and integrity and enables them to comply with strict data privacy regulations.

Organizations can store sensitive data in the cloud without worrying about security using Queryable Encryption with MongoDB Atlas. This feature allows organizations to maintain control over their data, perform rich, expressive querying capabilities on fully randomized encrypted data and reduce institutional risk.

Additionally, the built-in encryption solution supports popular MongoDB drivers, making it easy for developers to implement.

Benefits of Queryable Encryption

Compliance and regulatory standards: Queryable Encryption enables organizations to meet compliance requirements and regulatory standards, such as HIPAA or PCI-DSS, that mandate the encryption of sensitive data.
Protection against data breaches and unauthorized access: Queryable Encryption can also protect against data breaches and unauthorized access to the data, ensuring that sensitive information is protected at rest and in transit.
Continued use of existing tools and workflows: Queryable Encryption allows organizations to continue using their existing tools and workflows for data analysis without making any changes to their existing infrastructure.

Use Case Scenario

Let’s understand better with a real-life use case: Queryable Encryption in MongoDB could be a healthcare application that stores and processes sensitive patient information, such as personal identification numbers, medical history, and treatment plans. The application may need to comply with regulations such as HIPAA, which require that sensitive patient information be protected from unauthorized access.

Without Queryable Encryption, the healthcare application would need to store patient information in plaintext, making it vulnerable to data breaches. With Queryable Encryption, sensitive information can be encrypted at rest, making it much more difficult for unauthorized parties to access the data. The healthcare application can use MongoDB's built-in queryable encryption feature to encrypt the patient information using a key management system, such as AWS Key Management Service (KMS) or Azure Key Vault.

This encryption allows the healthcare application to perform queries on the encrypted data, such as searching for patients with specific medical conditions or treatment plans, without having to decrypt the data first. This provides an additional layer of security, as sensitive patient information is not exposed during the query process.

Tradeoffs

Even though Queryable Encryption sounds fascinating and has a decisive advantage, it does have some tradeoffs to be kept in mind while trying to implement the same.

One issue is that encryption can introduce performance overhead, as the data needs to be encrypted and decrypted when it is read from or written to the database. This may result in slower query performance, especially for large datasets. Encryption can also introduce complexity to the application, as the application needs to handle the encryption and decryption of data and the management of encryption keys.

Another potential issue is the difficulty in querying the encrypted data directly. Since the data is encrypted, traditional queries will not work on the encrypted data, so the application needs to be able to handle the encryption and decryption of the data on-the-fly, to be able to query the data. This can add complexity to the application and may also impact performance.

Conclusion

In summary, Queryable encryption in MongoDB provides a secure solution for storing and processing sensitive data while maintaining compliance with regulations. This feature allows for data encryption at rest and the execution of queries on the encrypted data without the need for decryption, thereby providing an additional layer of security.

However, it should be noted that the implementation of this feature may introduce performance and complexity considerations

For a technical deep dive, refer to MongoDB Releases Queryable Encryption Preview | MongoDB.

flatMap() - JavaScript Array method - ES2021

MrSrv7 — Wed, 18 Jan 2023 13:27:10 +0000

Have you ever wondered if there is any simple way to create an array from an existing nested array of objects' property?

Consider this example. You have an array of objects that contain the product information

const productsArray = [
    {name: 'Shirt', variants: [{color: 'red', size: 'S'}, {color: 'blue', size: 'M'}]},
    {name: 'Pants', variants: [{color: 'green', size: 'L'}, {color: 'black', size: 'XL'}]},
    {name: 'Dress', variants: [{color: 'pink', size: 'S'}, {color: 'purple', size: 'M'}]}
];

So from this example, you need to create an array which has all the variants array values. In this case, traditionally, map() method might be used.

const variantsArrays = products.map(product => product.variants);
console.log(variantsArrays);

Output will be

[ [ { color: 'red', size: 'S' }, { color: 'blue', size: 'M' } ],
  [ { color: 'green', size: 'L' }, { color: 'black', size: 'XL' } ],
  [ { color: 'pink', size: 'S' }, { color: 'purple', size: 'M' } ] ]

If you see this carefully, this is just an array, that is containing the arrays of variants from each object in the productsArray.

Now, to make this more readable and as a single array of objects, we will apply flat() method to the same.

const allVariants = variantsArrays.flat();
console.log(allVariants);

Output will be

[ { color: 'red', size: 'S' },
  { color: 'blue', size: 'M' },
  { color: 'green', size: 'L' },
  { color: 'black', size: 'XL' },
  { color: 'pink', size: 'S' },
  { color: 'purple', size: 'M' } ]

Issue

As you can see, using map() and flat() separately requires two separate steps, and also the result is the same, but the code is more verbose and harder to read. So, developers had to use nested loops and chaining multiple array methods to flatten and map nested arrays.

Solution

To overcome this issue, flatMap() method has been introduced with ES2021 update.

Definition (from MDN)

The flatMap() method returns a new array formed by applying a given callback function to each element of the array, and then flattening the result by one level. It is identical to a map() followed by a flat() of depth 1 (arr.map(...args).flat()), but slightly more efficient than calling those two methods separately.

Syntax (from MDN)

// Arrow function
flatMap((element) => { /* … */ })
flatMap((element, index) => { /* … */ })
flatMap((element, index, array) => { /* … */ })

// Callback function
flatMap(callbackFn)
flatMap(callbackFn, thisArg)

// Inline callback function
flatMap(function (element) { /* … */ })
flatMap(function (element, index) { /* … */ })
flatMap(function (element, index, array) { /* … */ })
flatMap(function (element, index, array) { /* … */ }, thisArg)
});

So, the above issue can be solved by flatMap() as

const variantsArray = productsArray.flatMap(product => product.variants))
console.log(variantsArray)

Output will be

[
  { color: 'red', size: 'S' },
  { color: 'blue', size: 'M' },
  { color: 'green', size: 'L' },
  { color: 'black', size: 'XL' },
  { color: 'pink', size: 'S' },
  { color: 'purple', size: 'M' }
]

Instead of returning required properties from the array of objects in a map() method and then applying flat() to get a simple array of objects (in the above usecase), we made use of flatMap() and extracted only the variants array in each object to a single array of objects.

Browser Compatibility

Browser	Version
Google Chrome	69 and above
Edge	79 and above
Firefox	62 and above (Note: This function is available in Firefox Nightly only.)
Opera	56 and above
Safari	12 and above

Conclusion

The flatMap() method is a powerful addition to JavaScript, allowing developers to simplify their code and perform complex operations on arrays and iterable objects with ease.

(Please do let me know on what topics to cover in the comments. Thanks)