DEV Community: Rajdeep Das

Build a simple fileserver in 1 line using go

Rajdeep Das — Sun, 21 Jan 2024 13:14:51 +0000

Photo by imgix on Unsplash

You have probably heard that Go is fantastic for building web applications of all shapes and sizes. This is partly due to the fantastic work that has been put into making the standard library clean, consistent, and easy to use.

If you’re just starting with web development in Go, the net/http package is super important. It helps you create servers for the web using simple and powerful building blocks.

The http.Handler Interface

As you become more familiar with Go, you will notice how much of an impact interfaces make in the design of your programs. The net/http interface encapsulates the request-response pattern in one method:

type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}

Implementors of this interface are expected to inspect and process data coming from the http.Request object and write out a response to the http.ResponseWriter object.

The http.ResponseWriter interface looks like this:

type ResponseWriter interface {
    Header() Header
    Write([]byte) (int, error)
    WriteHeader(int)
}

Often, you just want to share simple web pages with pictures and styles. Imagine you have a basic webpage with text, images, and design, and you just want to show it to others on the internet. That’s where serving static files comes in handy.

package main

import (
 "fmt"
 "net/http"
)

func main() {
 fmt.Println("FileServer Stared on Port : 8080")
 // "." - current directory where the progam is running
 http.ListenAndServe(":8080", http.FileServer(http.Dir("."))) 
}

The http.ListenAndServe function is used to start the server, it will bind to the address we gave it (:8080) and when it receives an HTTP request, it will hand it off to the http.Handler that we supply as the second argument. In our case, it is the built-in http.FileServer.

The http.FileServer function builds an http.Handler that will serve an entire directory of files and figure out which file to serve based on the request path. We told the FileServer to serve the current working directory with http.Dir(".").

If we go to localhost:8080/main.go in our web browser, we’ll see what’s inside our main.go file. With just one line of Go code, we can run this program from any folder and turn our computer into a simple server that shares files, like a tree of files, with others.

Checkout the project — Rajdeep-Das/go-networking: Go Lang Networking Stand Lib & Cloud Native Networking Projects (github.com)

Happy reading 📖. 😃

Vite 4.0 Takes Development to a Whole New Level: Faster, Smarter, and More Efficient

Rajdeep Das — Wed, 11 Jan 2023 08:26:35 +0000

Experience Lightning-Fast Development with Vite 4.0 and React: The Perfect Match

Photo by Lautaro Andreani on Unsplash

Vite is a new front-end development tool that addresses the performance issues associated with current JavaScript-based toolings like webpack, Rollup, and Parcel. By leveraging the availability of native ES modules in the browser and utilizing JavaScript tools written in compile-to-native languages.

Vite aims to improve developers’ productivity and happiness by reducing the wait times associated with spinning up a dev server and reflecting file edits in the browser, often seen in large-scale projects with thousands of modules.

Vite, the front-end development tool, saw tremendous growth and adoption over the past five months. The number of npm downloads per week has doubled from 1 million to 2.5 million and usage among the community in the Jamstack Conf survey jumped from 14% to 32% with a high satisfaction score of 9.7.

Vite-powered frameworks such as Astro 1.0 , Nuxt 3 , SvelteKit , Solid Start , and Qwik are also growing and collaborating. Storybook, Deno, and Nx also announced first-class support and investment in the Vite ecosystem. This upward trend is set to continue with Vitest adoption also rising, it’s expected to represent half of Vite’s npm downloads soon.

Let's Start experimenting with Vite 4.

pnpm create vite

React Vite Template

New React plugin using SWC during development#

SWC is now a mature replacement for Babel, especially in the context of React projects. SWC’s React Fast Refresh implementation is a lot faster than Babel, and for some projects, it is now a better alternative. From Vite 4, two plugins are available for React projects with different tradeoffs. We believe that both approaches are worth supporting at this point, and we’ll continue to explore improvements to both plugins in the future.

Learn more about SWC at https://swc.rs/

@vitejs/plugin-react#

@vitejs/plugin-react is a plugin that uses esbuild and Babel, achieving fast HMR with a small package footprint and the flexibility of being able to use the Babel transform pipeline.

@vitejs/plugin-react-swc (new)#

@vitejs/plugin-react-swc is a new plugin that uses esbuild during build but replaces Babel with SWC during development. For big projects that don’t require non-standard React extensions, cold start and Hot Module Replacement (HMR) can be significantly faster.

Let’s switch our project to use SWC (plugin-react-swc)

pnpm i -D @vitejs/plugin-react-swc

Change import in vite.config.js

vite.config.js

import { defineConfig } from 'vite'
//import react from '@vitejs/plugin-react'
import react from "@vitejs/plugin-react-swc";

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [react()],
})

React Vite SWC Project

Official Caveats (From Github)

https://github.com/vitejs/vite-plugin-react-swc

This plugin has limited options to enable good performances and be transpiler agnostic. Here is the list of non-configurable options that impact runtime behavior:

useDefineForClassFields is always activated, as this matches the current ECMAScript spec
jsx runtime is always automatic
In development:
esbuild is disabled, so the esbuild configuration has no effect
target is es2020
JS files are not transformed
tsconfig is not resolved, so properties other than the ones listed above behave like TS defaults

Importing CSS as a String (From Official Website)

In Vite 3, importing the default export of a .css file could introduce a double loading of CSS.

import cssString from './global.css'

This double loading could occur since a .css file will be emitted and it's likely that the CSS string will also be used by the application code — for example, injected by the framework runtime. From Vite 4, the .css default export has been deprecated. The ?inline query suffix modifier needs to be used in this case, as that doesn't emit the imported .css styles.

import stuff from './global.css?inline'

Environment Variables (From Official Website)

Vite now uses dotenv 16 and dotenv-expand 9 (previously dotenv 14 and dotenv-expand 5). If you have a value including # or `, you will need to wrap them with quotes.

-VITE_APP=ab#cdef
+VITE_APP="ab#cdef"

For more details, see the dotenv and dotenv-expand changelog.

Summary

Vite is a lightweight development server that is designed to be fast and easy to use. The latest version, Vite 4.0, brings a number of new features and improvements that make it an even more powerful tool for building web applications. Here are a few reasons why developers might want to give Vite 4.0 a try:

Speed : Vite is incredibly fast, thanks to its native ES module support and its ability to perform hot module replacement (HMR) without a webpack configuration. This makes development faster and more efficient, as you don’t have to wait for long build times or manually refresh the page to see your changes.
Simplicity : Vite is easy to set up and use, with minimal configuration required. This means developers can get up and running quickly, without having to spend a lot of time on setup. Additionally, since it’s built on top of native ES modules, it requires minimal configuration and dependencies.
Flexibility : Vite is highly customizable, with a range of options and plugins available. This allows developers to tailor their workflow to their specific needs. Additionally, Vite 4.0 has introduced a rollup and webpack plugin for even more flexibility.
Great Community : Vite has a growing community of developers who contribute to the project and also provides a lot of plugins that you can use to enhance your development experience.

Given these benefits, developers who are looking for a fast, easy-to-use, and flexible development server should definitely give Vite 4.0 a try.

Reference:

Rajdeep Das | LinkedIn

Software Developer | Rajdeep Software Engineering & Consulting Services

Serverless Stack for Cost-Effective Architecture on AWS

Rajdeep Das — Mon, 02 Jan 2023 09:36:39 +0000

Considering serverless stack for simplified cost-effective app development

Serverless computing is a cloud computing model that allows developers to build and deploy applications more quickly and cost-effectively than traditional approaches. Serverless Stack is a combination of cloud-native technologies and best practices that enable developers to take advantage of the scalability, security, and cost benefits offered by cloud platforms such as Amazon Web Services (AWS). In this blog, we’ll discuss the benefits of using Serverless Stack on AWS and offer tips for SMEs and entrepreneurs building cloud-native applications.

Introduction

Serverless Stack is a combination of cloud-native technologies and best practices that enable developers to build and deploy applications more quickly and cost-effectively than traditional approaches. It leverages serverless computing, which is a cloud computing model that allows developers to build and deploy applications without managing servers or infrastructure. With serverless computing, developers only pay for the resources they use, making it a more cost-effective approach than traditional server-based architectures.

The benefits of using a Serverless Stack include:

Low cost — Serverless Stack can significantly reduce the cost of deploying and running applications as compared to traditional approaches.
Easy to scale — Serverless Stack makes it easy to scale applications quickly and easily without the need for additional hardware or software.
Flexible — Serverless Stack allows developers to quickly adjust their architecture to meet changing needs without having to make costly changes to their existing code.
Automation — Serverless Stack automates many of the tedious tasks involved in deploying and maintaining applications, allowing developers to focus on building new features instead.

Solution Architecture

AWS provides a variety of services, including computing, storage, databases, analytics, and machine learning, which are all available to leverage when creating a cost-effective architecture on AWS. To take advantage of these services, developers can utilize AWS services such as Lambda, DynamoDB, and API Gateway to create a serverless architecture that is cost-effective and can scale with demand. Additionally, developers can leverage AWS services such as CloudFormation to automate the deployment of their applications. Finally, developers can also utilize AWS services such as CloudWatch for monitoring and alerting.

Development

When building cloud-native applications, it’s important to understand the advantages of using microservices. Microservices are small, independently deployable components that can be combined to create larger, more complex applications. The advantages of using microservices include:

Faster development — Microservices allow teams to develop and deploy code more quickly than traditional monolithic architectures.
Easier maintenance — Microservices also make it easier to maintain and update code over time, since each microservice can be updated independently of other services.

To ensure secure access to APIs hosted on AWS, developers can utilize Amazon API Gateway. API Gateway is a service that provides secure access to APIs hosted on AWS. It allows developers to control who has access to their APIs and how they can access them.

Conclusion

In conclusion, Serverless Stack is a combination of cloud-native technologies and best practices that enable developers to build and deploy applications more quickly and cost-effectively than traditional approaches. The benefits of using a Serverless Stack include low cost, easy scalability, flexibility, and automation. To create a cost-effective architecture on AWS, developers should utilize AWS services such as Lambda, DynamoDB, API Gateway, CloudFormation, and CloudWatch; use microservices; take advantage of the scalability, security, and cost benefits offered by cloud platforms; and automate the deployment of their application with tools such as CloudFormation. SMEs and entrepreneurs building cloud-native applications should keep these tips in mind when designing their architecture.

Originally published at https://www.linkedin.com.

Filters in ASP.NET Core

Rajdeep Das — Wed, 14 Dec 2022 19:18:26 +0000

How To Create Custom Action Filter In ASP.NET 6 & Above

Filters in ASP.NET Core allow code to run before or after specific stages in the request processing pipeline.

Inbuild Filters

Authorization prevents access to resources a user isn’t authorized for.
Response caching, short-circuiting the request pipeline to return a cached response.

Custom Filters

Custom filters can be created to handle cross-cutting concerns. Examples of cross-cutting concerns include error handling, caching, configuration, authorization, and logging. Filters avoid duplicating code. For example, an error-handling exception filter could consolidate error handling.

There are many filters in ASP.NET Core, but we will focus on Action Filters

Action filters:

Run immediately before and after an action method is called.
Can change the arguments passed into action.
Can change the result returned from the action.

How Filters Works

Filters run within the ASP.NET Core action invocation pipeline, sometimes referred to as the filter pipeline. The filter pipeline runs after ASP.NET Core selects the action to execute:

ASP.NET Filters

We can implement Action Filters as synchronous and asynchronous through different interface definitions

We are going to build a SingleDeviceSessionCheck Filter for example

using System;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Filters;
using Microsoft.AspNetCore.Mvc.Infrastructure;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;

namespace MyService.UserApi.Filters
{
  public class SingleDeviceCheck : IActionFilter
  {
      private readonly IDeviceAccessRepository deviceAccessRepository;
      private readonly ILogger<SingleDeviceCheck> logger;
      private readonly IConfiguration configuration;

      /* Constructor for DI */ 
      public SingleDeviceCheck(
      /* This Respository Containts logic for checking session in DB */
          IDeviceAccessRepository deviceAccessRepository,
          ILogger<SingleDeviceCheck> logger,
          IConfiguration configuration)
      {
          this.deviceAccessRepository = deviceAccessRepository;
          this.logger = logger;
          this.configuration = configuration;
      }
  }
   // our code before action executes
   public void OnActionExecuting(ActionExecutingContext context){}
   // our code after action executes
   public void OnActionExecuted(ActionExecutedContext context){}
}

We need to implement our logic in these two methods, for my use case I required the onActionExecuting method.

public void OnActionExecuting(ActionExecutingContext context)
{
  // Checking single device access ON or OFF (Feature Flag For Dev/Prod Test
  if (this.configuration["SD_ON_OFF"].Equals("OFF")) { return; }

  // our code before action executes
  var deviceId = context.HttpContext.Request?.Headers["Device-Id"];
  if (string.IsNullOrEmpty(deviceId.ToString()))
  {
      context.Result = new BadRequestObjectResult("Device-Id is empty");
      return;
  }
  // if same device login then permission granted
  // Read USER INFO From JWT Claims
  var email = context.HttpContext.User.FindFirst("https://xyz.com/email")?.Value;
  var deviceAccessData = deviceAccessRepository.GetActiveDeviceByEmailAsync(email).Result;
  if (deviceAccessData == null)
  {
      context.Result = new UnauthorizedObjectResult(new { status = false, message = "Couldn't Find Your Device. Please Logout & Login Again." })
      {
          StatusCode = 403
      };
      return;
  }
  if (!(deviceAccessData.DeviceId == deviceId.ToString() && deviceAccessData?.IsActive == true))
  {
      logger.LogInformation("Device Failed To Pass {device} and {email}", deviceId, email);
      context.Result = new UnauthorizedObjectResult(new { status = false, message = "You have already active in another device.\n Access Denied." })
      {
          StatusCode = 403
      };
      return;
  }
  else
  {
      logger.LogInformation("Device Pass {device} and {email}", deviceId, email);
  }
}

Now we will register our Action Filter in Program. cs or Startup.cs

public void ConfigureServices(IServiceCollection services)
{
  services.AddDbContext<MovieContext>(options =>
   options.UseSqlServer(Configuration.GetConnectionString("sqlConString")));
  //Filters 
  services.AddScoped<SingleDeviceCheck>();
  services.AddControllers();
}

Now we will use This Filter In Our Controller

//Dev Test Only
[HttpGet("device")]
//Filter Code Run in Request Pipeline
[ServiceFilter(typeof(SingleDeviceCheck))]
public IActionResult CheckDevieID() {
    return Ok("Single Device ID Endpint Check");
}

Summary

Action Filter helps us to reuse code.
Run Some Common Logic In SPecific Controller or Methods In the Request Pipeline
A middleware can run for all requests while filters will only run for requests that reach the EndpointMiddleware
filters will only run on specified actions and controllers unless you register the filter globally in the startup. Since you have full access to the context you can also access the controller and action itself.

I always recommend using Action Filters because they give us reusability in our code and cleaner code in our actions as well.

Software Developer | Rajdeep Software Engineering & Consulting Services

Rajdeep Das | LinkedIn

Thank You, happy reading 📖 , 😊

Create Logger for Microservices Powered By RedisSearch & RedisJSON

Rajdeep Das — Thu, 15 Sep 2022 12:19:49 +0000

A Lightweight Dot Net Realtime Message, Event, HTTP (Request & Response) and Exception Logger For Web (Trying to make centralised logger for Microservice and K8s environment)

I’m searching for a logger and a simple UI to view my HTTP request & response logs in microservice environments. I found something called elastic-search, where all logs are stored & it gives a nice UI to view.

But for my simple APP & Low User base, Elastic search setup & operation cost are so high for my requirements. so I decided to build my simple logger, nothing fancy or complicated.

The idea is to build a simple Middleware that logs all the HTTP requests & responses. Then store the logs in DB for View Later. Create an Extension Method to register this middleware in HTTP Pipeline & Publish this middleware as Nuget Package.

My Requirements are

I want a simple temporary log storage max 7–14 Days Logs.
Very Lightweight to set up, and operate and has low memory usage.
It just does two things record all HTTP request & response data, and exception logs for my microservices which are written in ASP.NET Core (.NET 6).
A Simple UI where I can search & view logs by service name, request methods, timestamp, etc filter.
I don’t want to use my primary application, DB, to store temporary logs.

How I build

When searching on the internet I found something called RedisSearch . RediSearch is a Full-Text Search Engine for NoSQL Database.

According to Redis Website, RediSearch is powerful indexing, querying, and full-text search engine for Redis, available on-premises and as a managed service in the cloud.

Everyone knows what is Redis (At least for those who are in the web development field for the last few years). Redis is a very popular cache but Redis also can be used as Primary Database, it supports JSON Documents like MongoDB (No-SQL Database). We can enable these features by installing Redis JSON Module.

This is the perfect choice for my requirements because I want nice search capabilities, I want search data in the HTTP request & response body and I don’t want a strong schema because HTTP request and response bodies vary from service to service but at the same time I also want strong search functionality.

Let’s Get Started

Let’s set up Redis, we are going to use the Redis cloud for easy setup, and quickly get started with RedisJSON & RedisFullText Search Module.
Redis Enterprise Cloud provides performance, operational simplicity, data management, and security in the cloud. https://redis.com/redis-enterprise-cloud/overview/. Don’t worry it provides a free tier for quickly building & prototype your solution. You can choose your favorite cloud provider & region. I’m Going to choose AWS and AP-South-1.

Redis Enterprise Cloud

After creating Account & Choosing Your RedisStack Deployment, You will get a public endpoint.

Redis Public Endpoint

We will use RedisInsight . A Redis GUI tool to connect to Redis cloud instance.RedisInsight enables the use of Redis data types including JSON and time series by providing a new user interface, a browser tool, a sophisticated CLI, bespoke data visualization, and built-in tutorials.

Redisinsight

Redis Enterprise enables you to create backups across all database shards quickly and consistently. It achieves quick auto-cluster recovery by rebuilding the cluster from the configuration file and using the same endpoints and database configurations.

Instant disaster recovery is achieved through Active-Active deployment, which allows reading and writing to each replica at any time and is backed up by an academically proven conflict-resolution mechanism.

Quick Overview of Redis JSON & Get Started

We can interact with Redis JSON documents using Redis CLI or Redis CLI available on RedisInsight GUI Tool.

To interact with Redis JSON we most often use JSON.GET & JSON.SET Commands from CLI

Redis Insight CLI

According to Redis DOCS & Website Redis.JSON GET & SET Commands specification.

Redis JSON Supported Data Structure

Scalar
Objects (including nested objects)
Arrays of JSON objects
JSON nested objects

JSON.GET

Syntax

JSON.GET key [INDENT indent] [NEWLINE newline] [SPACE space] [paths
[paths ...]]

Available in:

Redis Stack / JSON 1.0.0

Time complexity:

O(N) when the path is evaluated to a single value where N is the size of the value, O(N) when the path is evaluated to multiple values, where N is the size of the key

This command accepts multiple path arguments. If no path is given, it defaults to the value's root.

The following subcommands change the reply’s format (all are empty strings by default):

INDENT sets the indentation string for nested levels
NEWLINE sets the string that's printed at the end of each line
SPACE sets the string that's put between a key and a value

JSON.GET Result

JSON.SET

JSON.SET key path value [NX | XX]

Available in: Redis Stack / JSON 1.0.0

Time complexity: O(M+N) when the path is evaluated to a single value where M is the size of the original value (if it exists) and N is the size of the new value, O(M+N) when the path is evaluated to multiple values where M is the size of the key and N is the size of the new value

JSON.SET Command

How to get all keys in Redis JSON

To list the keys in the Redis data store, use the KEYS command followed by a specific pattern. Redis will search the keys for all the keys matching the specified pattern. In our example, we can use an asterisk (*) to match all the keys in the data store to get the keys.

List Keys

To connect Redis from .NET Required Some sort of DB Driver & ORM. StackExchange Redis is popular amongst .NET Developers to connect Redis instances. But for easy use of Redis search & Redis JSON features, we will use Redis OM. NET.

Why Redis OM?

Redis OM provides high-level abstractions for using Redis in .NET, making it easy to model and query your Redis domain objects.

Declarative object mapping for Redis objects
Declarative secondary-index generation
Fluent APIs for querying Redis
Fluent APIs for performing Redis aggregations

Let’s try to use RedisOM in Our Dotnet Application. It’s simple to use RedisJSON with ASP.NET.

Create the Logs model. Here you will notice that the fields are decorated with the Indexed attribute. These attributes ( Searchable and Indexed ) tell Redis OM that you want to be able to use those fields in queries when querying your documents in Redis Stack.

[Document(StorageType = StorageType.Json, Prefixes = new []{“WLog”})] Indicates that the data type that Redis OM will use to store the document in Redis is JSON and that the prefix for the keys for the WLog class will be WLog.

The Id property is marked as a RedisIdField. This denotes the field as one that will be used to generate the document’s key name when it’s stored in Redis. e,g Id:”01GC92PD10R0DFVST9CV6Q7SBS”

To insert your object into Redis, use Set on the RedisConnection or Insert in the RedisCollection. Redis OM will automatically set the id for you and you will be able to access it in the object. If the Id type is a string and no IdGenerationStrategy is explicitly overridden on the thing, the ULID will bind to the string.

WLog.cs to RedisJSON

After you’ve finished building the model, you’ll need to construct the index in Redis. The best method to handle this is to spin the index creation out into a Hosted Service that will execute when the app is launched. Make a directory called ‘HostedServices’ and add IndexCreationService.cs to it. Add the following to that file to construct the index on startup.

Redis OM handles connections to Redis and offers the classes you may use to communicate with Redis via the RedisConnectionProvider class. To utilize it, just inject a RedisConnectionProvider object into your application.

Let’s Create a Helper Class That will contain DB Access Logic. One Of the great features I like about Redis OM, we can easily query using LINQ expression. I used Object Mapper to Convert Domain Model to Redis DB Models. In the future, if we want to support other DB, we can easily do that.

public IEnumerable<WatchLog> FilterByMethod(string method){ var logs = _redisWatchLog.Where(x => x.Method == method).ToList(); List<WatchLog> logList = CustomMapper.Mapper.Map<List<WLog>, List<WatchLog>>(logs);            
return logList;                        
}

Import Note About Redis OM .NET

Redis OM may be used in conjunction with normal Redis for object mapping and retrieving objects by ID. Redis OM is dependent on the Redis Stack platform for more sophisticated functionality like indexing, querying, and aggregation. The Redis Stack platform is a collection of modules that expand Redis.

Why is this significant?

You can still use Redis OM to develop declarative models supported by Redis if you don’t have Redis Stack.

We’ll store your model data as Hashes in Redis, and you’ll be able to retrieve models using their main keys.

So, what won’t operate in the absence of Redis Stack?

Models cannot be nested inside each other.
You will not be able to discover objects using our expressive queries; you will only be able to query by primary key.

As We are Building A Logger Libray as Middleware, the Injection code is written in Service Collection Extension Method.

We Register the RedisConnection provider as a singleton.

services.AddSingleton(new RedisConnectionProvider(WatchLogExternalDbConfig.ConnectionString));

This will retrieve your connection string from the configuration and initialize the provider. The provider is now accessible for usage in your controllers/services.

Now we will create the leading Middleware , That will intercept the HTTP Request & Response and save it to DB.

Next, To Get Logs in UI, We have to create an API, from which we will fetch logs. Let’s Create an API Controller That looks like the one below.

This is a very basic simple API Idea, We will later extend it with robust Redis FullTextSearch Methods, RealTime Streaming, etc fancy things. Keep in mind this is a Proof of concept type of Project, a very early stage in development.

Now we can use This Logger in the dotnet Web API project.

// Configure DB Connection In API Service
builder.Services.AddWatchLoggerServices(opt =>{ opt.SetExternalDbConnString = builder.Configuration["RedisConnectionString"];});

Now Register WatchLogger In Middleware

// Custom Logger Middleware
app.UseWatchLogger();

Next, We will Build A Sample UI In Angular or a Similar Framework, That will somewhat look like this.

WatchLogger UI

For UI, I am writing a separate second Part. Where we will extend Redis Stack Features, Fully Use RedisFullText Search, and RedisStream also adds Exception Logger Middleware, WebSocket, SignalR, etc.

Don’t forget to try Redis, and we’ll continue the series as a Redis Stack. In forthcoming blogs, we will look at Redis Search, Redis Stream, pub-sub, and many more topics. To learn more about Redis, go to https://redis.io/.

Early Stage Project Available On Github: WatchLogger

GitHub - Rajdeep-Das/WatchLogger: A Lightweight Dot Net Realtime Message, Event, HTTP (Request & Response) and Exception Logger For Web (Trying to make centralized logger for Microservices and K8s)

This post is in collaboration with Redis.

Learn more about Redis Cool 😎 Features:

JavaScript Path Documentation of Handy Methods

Rajdeep Das — Mon, 11 Jul 2022 01:35:56 +0000

The find() method returns the value of the first element in the provided array that satisfies the provided testing function.

const array1 = [{id:'1',name:'test'}, {id:'2',name:'test2'}];
const found = array1.find(element => element.id === '1');
console.log(found);

OutPut:

Object { id: "1", name: "test" }

Design Restful WebServices — Part 1

Rajdeep Das — Fri, 17 Jun 2022 17:05:37 +0000

Design Restful WebServices — Part 1

Design API like end user will consume It

Photo by Fahim Muntashir on Unsplash

HTTP is an application-level protocol that defines the operation between client and server. There is a difference between designing Restful API & Designing API, which also uses HTTP protocols and JSON/XML for data transfer.

REST is an acronym for R epresentational S tate T ransfer, an architectural style for distributed hypermedia systems.

In this protocol, methods such as GET , POST , PUT , and DELETE operations that act on resources.

This protocol eliminates the need for you to invent application-specific operations such as createOrder , getStatus , updateStatus , getUserInfoById, etc.

A Web API or Web Service that follows this type of architecture is known as a RESTful API

Problem 1:

You want to know how to apply those protocols GET, POST, PUT, PATCH, and DELETE on resources to achieve Restful Architecture Style.

Solution:

When you define the resource like Customer, Order, User, etc, use GET to get a resource representation, PUT to update a resource, DELETE to delete a resource, and POST to create a resource.

Role of Idempotent & Safe HTTP Methods in Restful Web Services & Implementation

What is idempotent in the HTTP Method?

We call an HTTP method idempotent if that method does not create any side effects or alter the state of the server even if we send the same request many times.

GET — if we call get request on a resource many times it will not change the state of the resource, it will return the result of how many times we call it without doing any side effects

POST — If we call a post request multiple times it will create a new resource every time. For Example, if we send create customer resource multiple times it will every time generate a new customer and save to DB.

If we Implement correctly, the GET, HEAD, PUT, and DELETE methods should be idempotent in Restful API Design. If we violate that rule then we can’t call it RESTful Standard API Design.

In an idempotent check, only Backend Server State is considered. Not the Status Code.

For Example

The first Call of DELETE Return 200 but the Second call will return 404.

HTTP Method Table

Happy reading… 😁😃 Rajdeep Das | LinkedIn

Relational Database At A Glance With PostgreSQL

Rajdeep Das — Mon, 31 May 2021 16:53:51 +0000

According to Wikipedia A relational database is a digital database based on the relational model of data, as proposed by E. F. Codd in 1970. A software system used to maintain relational databases is a relational database management system (RDBMS). Many relational database systems have an option of using the SQL (Structured Query Language) for querying and maintaining the database

Database 101

A database stores data

Plain text files have limited functionality for basic processes, including:

Searches
Simultaneous updates and reads
Access control

Access the text file via a management system to:

Traffic management for the data
Manage concurrency
Provide other features

ACID compliance

Atomicity — Store data in an all-or-nothing approach.

Consistency — Give me a consistent picture of the data.

Isolation — Prevent concurrent data updates from incorrect reads/writes.

Durability — When User say ‘ COMMIT ;’ the data, make sure it is safe until User explicitly destroy it.

Database transactions

A transaction is a unit of work.

A transaction is all or nothing:

Beginning (BEGIN;)
Work (INSERT/UPDATE/DELETE/SELECT)
Ending (END;) results in one of the following:
COMMIT; (save everything)
ROLLBACK; (undo all changes, and save nothing)

Once the transaction ends, it either makes ALL of the changes between BEGIN; and COMMIT; or NONE of them (if there is an error, for example).

PostgreSQL 101

PostgreSQL meets all the requirements to be a fully ACID-compliant, transactional database. To accomplish this, many database and computer science concepts are implemented. The focus of the course is to illustrate PostgreSQL internals and the effects they have on the end user.

PostgreSQL relational database management system (RDBMS) serves an instance that:

Serves one (and only one) TCP/IP port

Has a dedicated data directory

Contains at least one database

PostgreSQL features

ACID compliant

Transactional (uses WAL/REDO)

Partitioning

Multi version concurrency control (readers don’t block writers)

Online maintenance operations

Hot/warm Standby

Full-text search

Rich geospatial (PostGIS)

Procedural languages

Postgres Database limitations

Limitations

PostgreSQL terminology

Terminology

Summary

PostgreSQL is an enterprise class, open source relational database management system.

PostgreSQL has a rich history and was created at UC Berkeley. PostgreSQL is flexible, and can scale into the future.

The PostgreSQL engine adds new features and functionality to appeal to new use cases.

Node Awesome Filesystem Magic

Rajdeep Das — Fri, 16 Oct 2020 12:59:03 +0000

Build a simple file watcher👀 without external dependencies or packages from NPM

Photo by Sean Lim on Unsplash

Let’s get started by developing a couple of simple programs that watch files

for changes and read arguments from the command line. Even though they’re

simple toy program, these applications offer insights into Node.js’s event-based architecture.

The Power of asynchronous programming

The power of asynchronous coding in Node.js. Taking action whenever a file changes is just plain useful in a number of cases, ranging from automated deployments to running unit tests.

Our File Watcher 👀 in Action

**const** fs = require('fs');

**const** filename = process.argv[2];

if(!filename) {
    throw Error('A file must specified');
}

fs.watch(filename, () **=>** console.log(`File ${filename} changed!` ));

console.log(`Now Watching Your Awesome ${filename} for change `);

File Watcher In Action

The Great Event Loop ⚡

To run the program, Node.js does the following:

• It loads the script, running all the way through to the last line, which

produces the Now watching message in the console.

• It sees that there’s more to do because of the call to fs.watch().

• It waits for something to happen — namely, for the fs module to observe a

change to the file.

• It executes our callback function when the change is detected.

• It determines that the program still has not finished, and resumes waiting.

The Node.js Process

Any unhandled exception thrown in Node.js will halt the process. The exception output shows the offending file and the line number and position of the exception. It’s pretty common (maybe or mayn't be🙄 ) in Node.js development to spawn separate processes as a way of breaking up work, rather than putting everything into one big Node.js program. let’s spawn a process in Node.

Spawning a Child Process

'use strict'
**const** fs = require('fs');
**const** spawn = require('child_process').spawn;
**const** filename = process.argv[2];

if(!filename) {
  throw Error('A file must specified');
}

fs.watch(filename ,() **=>** {
**const** ls = spawn('ls', ['-l', '-h', filename]);
ls.stdout.pipe(process.stdout);

});

console.log(`Now Watching Your Awesome ${filename} for change `);

The object returned by spawn() is a Child Process. Its stdin , stdout , and stderr properties are Streams that can be used to read or write data. We want to send the standard output from the child process directly to our own standard output stream. This is what the pipe() method does.

The New Child Process In Action

Capture The Data From Stream or Event Emitter

EventEmitter is a very important class in Node.js. It provides a channel for events to be dispatched and listeners to be notified. Many objects you’ll encounter in Node.js inherit from EventEmitter , like the Streams we saw in the last section.

'use strict'

**const** fs = require('fs');
**const** spawn = require('child_process').spawn;
**const** filename = process.argv[2];

if(!filename) {

   throw Error('A file must specified');
}

fs.watch(filename ,() **=>** {

**const** ls = spawn('ls', ['-l', '-h', filename]);

**let** outpt = '';

_// Added event listener like 'data', 'close'_

ls.stdout.on('data', chunk **=>** outpt += chunk);

ls.on('close', () **=>** {

**const** parts = outpt.split(/|s+/);

console.log(parts[0],parts[4],parts[8]);

});

ls.stdout.pipe(process.stdout);

});

console.log(`Now Watching Your Awesome ${filename} for change `);

EventEmitter Stream in action

The on() method adds a listener for the specified event type. We listen for data events because we’re interested in data coming out of the stream.

A Buffer is Node.js’s way of representing binary data.It points to a blob of memory allocated by Node.js’s native core, outside of the JavaScript engine.

Any time you add a non-string to a string in JavaScript (like we’re doing here

with chunk), the runtime will implicitly call the object’s toString() method.

For a Buffer, this means copying the content into Node.js’s heap using the default encoding (UTF-8).

Like Stream, the ChildProcess class extends EventEmitter, so we can add listeners to it, as well.

Child Process

After a child process has exited and all its streams have been flushed, it emits a close event.

Reading and Writing Files In Node.js Asynchronously

we wrote a series of Node.js programs that could watch files for changes. Now let’s explore Node.js’s methods for reading and writing files. Along the way we’ll see two common error-handling patterns in Node.js error events on EventEmitters and err callback arguments.

There are a few approaches to reading and writing files in Node. The simplest is to read in or write out the entire file at once. This technique works well for small files. Other approaches read and write by creating Streams or staging content in a Buffer

'use strict';
**const** fs = require('fs');
fs.readFile('target.txt', (err, data) **=>** {

if (err) {
   throw err;
}
console.log(data.toString());
});

Notice how the first parameter to the readFile() callback handler is err. If readFile() is successful, then err will be null. Otherwise the err parameter will contain an Error object. This is a common error-reporting pattern in Node.js, especially for built-in modules.

'use strict';
**const** fs = require('fs');

fs.writeFile('target.txt', 'This is file conntent', (err) **=>** {
if (err) {
   throw err;
}
console.log('File saved!');
});

This program writes This is file content to target.txt (creating it if it doesn’t exist, or overwriting it if it does). If for any reason the file can’t be written, then the err parameter will contain an Error object.

Creating Read and Write Streams

You create a read stream or a write stream by using fs.createReadStream() and fs.createWriteStream(), respectively.

Rebuild Cat program in node

_#!/usr/bin/env node_  
'use strict'
require('fs').createReadStream(process.argv[2]).pipe(process.stdout)

Because the first line starts with #!, you can execute this program directly in Unix-like systems. Use chmod to make it executable:

$ chmod +x cat.js

Then, to run it, send the name of the chosen file as an additional argument:

$ ./cat.js target.txt

We have learned how to watch files for changes and to read and write files. we also learned how to spawn child processes and access command-line arguments. Node.js also support synchronous file access but it’s better not to block event-loop unless you know what you are doing.

Thank you. Happy Reading 😀😀

Software Engineer

Short Intro about serverless applications

Rajdeep Das — Sun, 08 Mar 2020 16:23:38 +0000

Photo by Christina @ wocintechchat.com on Unsplash

What is serverless?

Serverless workloads are “event-driven workloads that aren’t concerned with aspects normally handled by server infrastructure.” Concerns like “how many instances to run” and “what operating system to use” are all managed by a Function as a Service platform (or FaaS), leaving developers free to focus on business logic.

The above definition by the spring.io group.

Generally when we talk about serverless architecture first things comes to mind is most popular Function as a service ( FaaS ) model on cloud.

What the heck is FaaS?

According to Wikipedia Function as a service (FaaS) is a category of cloud computing services that provides a platform allowing customers to develop, run, and manage application functionalities without the complexity of building and maintaining the infrastructure typically associated with developing and launching an app. Building an application following this model is one way of achieving a “serverless” architecture, and is typically used when building microservices applications.

AWS Lambda was the first FaaS offering by a large public cloud vendor, followed by Google Cloud Functions, Microsoft Azure Functions, IBM/Apache’s OpenWhisk (open source) in 2016 and Oracle Cloud Fn (open source) in 2017.

Use cases for FaaS are associated with “on-demand” functionality that enables the supporting infrastructure to be powered down and not incurring charges when not in use. Examples include data processing (e.g., batch processing, stream processing, extract-transform-load (ETL)), Internet of things (IoT) services for Internet-connected devices, mobile applications, and web applications.

Let’s looks at a sample code example of firebase cloud function which supports node.js function as services. In the example, we build a REST endpoint with node.js and express framework. The below code currently in production as a backend service for a mobile app.

https://medium.com/media/7ed20259e5f6f21788941b81ebee8b62/href

Serverless characteristics?

Serverless applications have a number of specific characteristics, including:

Event-driven code execution with triggers
The platform handles all the starting, stopping, and scaling chores
Scales to zero, with low to no cost when idle
Stateless

The Image from spring.io

List of Resources for Serverless platform for getting started

Thank you.

Rajdeep Das

Modern days Identity management for your next project

Rajdeep Das — Fri, 18 Oct 2019 14:28:26 +0000

Custom Build vs Paid Identity Management Services

What is Identity Management?

Identity management, refers to a service or platform that identifies individuals and controls their access to system resources through user rights and restrictions.Identity management is important for security and increases the productivity of users by implementing a central directory: users don’t need to remember and keep track of several different usernames and passwords.

According to Wikipedia Identity management, also known as identity and access management, is a framework of policies and technologies for ensuring that the proper people in an enterprise have the appropriate access to technology resources. IdM systems fall under the overarching umbrella of IT security and Data Management.

Use cases

Identity management solutions provide benefits for all types of businesses. IAM can also provide distinct and specialized features to serve B2B, B2C, and B2E use cases.

B2B : A business provides federated identity management to another business, such as Trello allowing another business to log into Trello with their enterprise credentials.

B2C : A business provides social authentication to consumers through Facebook, Google, or other social media identity providers.

B2E : A business provides single sign-on to its own employees.

Different types of identity management solution

Federated Identity : Federated Identity Management is a method of transferring authentication data without violating the same origin policy, generally by using an external authorization server.

Single Sign On (SSO): SSO is a type of Federated Identity Management. SSO occurs when a user logs into one client and is then signed in to other clients automatically, regardless of differences in platform, technology, or domain. A token or cookie is generated to authenticate the user across domains.

Enterprise Federation : Enterprise Federation is Federated Identity Management with enterprise connections such as Active Directory, LDAP, ADFS, SAML, Google Apps, etc.

Signs You Need to Move From Custom Build to an Identity Management Solution

You need a standards-based solution, such as OpenID Connect, SAML, WS-Federation, and/or OAuth
You have users that authenticate with various identity providers but lack a way to link their accounts.
You have applications on different domains and require users to log in separately for each.
Your best developers spend their time building and maintaining identity management and authentication instead of building core business applications.
Your company has experienced any type of data breach or you are concerned with a data breach.

Here Specifically I will talk about B2C(Business to Customers) Identity Management Use Cases.

Why we want a 3rd Party identity management Solution for B2C Products specially for Startups,Small to medium business

Your main source of user data comes from directly asking users on forms or surveys. Being able to easily extract third party data about your users would help you better understand your customers and drive more revenue through upsells and targeted marketing.
If you sell to consumers, you don’t offer an easy 1-click signup option through social identity providers.
You’ve faced performance concerns as you increased your user base.
Last but not the least as developer i faced this issue of managing own identity management solution really really slow down your core business and product development and also increase the security threats.

List of popular 3rd party identity management solution

Auth0[https://auth0.com/]
Okta[https://www.okta.com/]

I’m not going to compare which is better both are good and I have used both , you can try this freely for your upcoming projects

Others Identity Management Solution are

Published By

Originally published at https://www.linkedin.com.

SSE(Server-Sent Events) VS WebSocket

Rajdeep Das — Tue, 20 Aug 2019 15:43:29 +0000

https://github.com/manucorporat/sse

For beginners, both WebSockets and SSEs make use of HTTP connections, although in case of WebSockets a TCP handshake effectively upgrades from HTTP protocol to WebSocket protocol, allowing WebSocket applications to more easily fit into existing infrastructures. However, the main similarity between both is their functionality: both push data from the client to server, a process also known as “server push”. With this in mind, let´s now have a look at some differences between the two before we can talk about best use cases for both technologies.

Before we get into the differences between both technologies, we should state they’re not competing technologies, nor is one better than the other. The popularity of WebSockets over SSEs can be explained by the fac that WebSockets have received more attention (and appreciation) than SSEs, and it’s a fact that more browsers support it natively than they support SSEs.

The differences between both technologies are not that big, and if you want to do “server-push” only, both are good choices.

By far the biggest difference between both technologies is that WebSockets are full-duplex, bidirectional communication between client and server, whereas SSEs are mono-directional.

SSEs come with a set of features that WebSockets lack by design, such as automatic reconnection , event IDs and sending arbitrary events. On the other hand, WebSockets are able to detect a dropped client connection, whereas SSEs first need to send a message before detecting the same issue. In terms of browser support, both Internet Explorer and Edge do not yet support SSEs, although polyfills that simulate SSE functionality are available to solve this issue.

Although WebSockets use an initial HTTP connection, this connection is updated after a TCP handshake after which data is sent through the WebSocket protocol. This is a more complex protocol than the SSE protocol. Because they’re bidirectional , WebSocktets require more development effort than SSEs, that only need to send an HTTP message with a specific header, whereas a WebSocket needs to establish and maintain a TCP socket communication, as well as a listener socket on the server side.

It´s good to know that SSE suffers from a limitation to the maximum number of open connections, which can be specially painful when opening various tabs as the limit is per browser is six. Only WebSockets can transmit both binary data and UTF-8, whereas SSE is limited to UTF-8.

While WebSockets are more complex and demanding than SSEs, a full-duplex TCP connection makes it useful for a wider range of application scenarios. SSE is a simpler and faster solution, but it isn’t extensible: if your web application requirements were to change, it would need to be refactored using WebSocket. Although WebSocket technology presents more upfront work, it´s a more versatile and extensible framework, so a better option for complex applications that will add new features over time.

WS and SSE Best Use Cases

In the end, whether you should be using WebSockets or SSEs depends on your use case. As stated before, SSEs cannot provide bidirectional client-server communication as opposed to WebSockets. Use cases that require such communication are real-time multi-player games and messaging and chat apps. When there´s no need for sending data from a client, SSEs might be a better option than WebSockets. Examples of such use cases are status updates , news feeds and other automated data push mechanisms.

How Server Sence( A Upcoming Project by RajdeepDev’s) Use SSE

ServerSence is RealTime Server , Web App monitoring Solution with instant alert for server or Web app failure.You don’t have to develop or maintain a monitoring solution for your appwe take care of that, so you can deliver amazing User Experience to client.

Server Sence Dashboard Use SSEs for realtime visulization of Server and Web App Status and Sending Realtime Alert.(For more details contact rajdeepdas.india@gmail.com , if you want to be part of that project).

Conclusion

In this post we compared WebSockets and SSEs. We discussed the differences and similarities between the two technologies. While both can be used for “server push”, there are many subtle differences between the two. Because WebSockets have received more attention from developers and bloggers, SSEs are lesser-known but no less potent.

Published By

Originally published at https://www.linkedin.com.