DEV Community: Anuj Singh

Building a Lightning-Fast NSE Stock Data Scraper: From API Calls to Full-Stack Web App

Anuj Singh — Wed, 27 Aug 2025 19:00:38 +0000

Introduction

Ever tried to download historical stock data from the National Stock Exchange (NSE) of India and found yourself waiting hours for selenium-based scrapers to crawl through web pages? I built a solution that's 10x faster using direct API calls and wrapped it in a beautiful full-stack web application.

🔗 Live Demo: https://nse-scrap.onrender.com

📁 GitHub Repository: https://github.com/singhanuj620/nse_scrap

🚀 What We're Building

A complete stock data scraping solution that includes:

Backend API scraper using Node.js with direct NSE API calls
React frontend with real-time progress tracking
Session management for concurrent scraping jobs
Downloadable ZIP exports of all scraped data
Responsive UI with modern design

🎯 The Problem with Traditional Scraping

Most stock data scrapers rely on browser automation tools like Selenium or Puppeteer. While functional, they have significant drawbacks:

Slow: Loading full web pages for each request
Unreliable: Breaking when UI changes
Resource-heavy: Requires browser instances
Limited scalability: Can't handle many concurrent requests

💡 The Solution: Direct API Approach

Instead of scraping web pages, I discovered that NSE provides direct API endpoints for historical data. Here's the core approach:

class NSEAPIClient {
    constructor() {
        this.baseUrl = 'https://www.nseindia.com';
        this.headers = {
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
            'Accept': 'application/json, text/plain, */*',
            'Referer': 'https://www.nseindia.com/report-detail/eq_security',
            // ... other headers for authentication
        };
    }

    async initializeSession() {
        // Get session cookies from NSE
        const response = await https.get(`${this.baseUrl}/report-detail/eq_security`);
        this.cookies = extractCookies(response);
    }

    async fetchData(symbol, year) {
        const fromDate = `01-01-${year}`;
        const toDate = `31-12-${year}`;

        const url = `${this.baseUrl}/api/historicalOR/generateSecurityWiseHistoricalData?from=${fromDate}&to=${toDate}&symbol=${symbol}&type=priceVolumeDeliverable&series=EQ&csv=true`;

        return await this.makeRequest(url);
    }
}

🏗️ Architecture Overview

Backend (Node.js + Express)

The backend handles three main responsibilities:

API Scraping Engine: Direct calls to NSE APIs
Session Management: Track multiple scraping jobs
File Management: Organize and zip downloaded data

// Session tracking for concurrent scraping jobs
const activeSessions = new Map();

app.post('/api/start-scraping', async (req, res) => {
    const { stocks, startYear, endYear, sessionId } = req.body;

    // Initialize session tracking
    const session = {
        sessionId,
        total: stocks.length * (endYear - startYear + 1),
        completed: 0,
        failed: 0,
        currentStock: null,
        currentYear: null,
        status: 'starting',
        results: [],
        startTime: new Date()
    };

    activeSessions.set(sessionId, session);

    // Start scraping in background
    scrapingProcess(stocks, startYear, endYear, sessionId);

    res.json({ 
        message: 'Scraping started successfully', 
        sessionId,
        total: session.total 
    });
});

Frontend (React + Vite)

The frontend provides a clean interface with:

Stock management: Add/remove stocks to scrape
Real-time progress: Live updates during scraping
Result visualization: Success/failure status for each stock
Download functionality: One-click ZIP download

const NSEScraper = () => {
  const [stocks, setStocks] = useState(['RELIANCE', 'TCS', 'HDFCBANK']);
  const [progress, setProgress] = useState(0);
  const [isRunning, setIsRunning] = useState(false);

  // Real-time progress polling
  useEffect(() => {
    if (sessionId && isRunning) {
      const interval = setInterval(async () => {
        const response = await axios.get(`/api/progress/${sessionId}`);
        const data = response.data;

        setProgress(data.progress);
        setCurrentStock(data.currentStock);
        setCurrentYear(data.currentYear);
        setCompleted(data.completed);
        setFailed(data.failed);

        if (data.status === 'completed' || data.status === 'error') {
          setIsRunning(false);
        }
      }, 1000);

      return () => clearInterval(interval);
    }
  }, [sessionId, isRunning]);

  return (
    <div className="nse-scraper">
      {/* Beautiful UI components */}
    </div>
  );
};

🔧 Key Technical Features

1. Session Management

Each scraping job gets a unique session ID, allowing multiple users to run concurrent scraping operations without conflicts.

// Progress tracking endpoint
app.get('/api/progress/:sessionId', (req, res) => {
  const sessionId = req.params.sessionId;
  const session = activeSessions.get(sessionId);

  if (!session) {
    return res.status(404).json({ error: 'Session not found' });
  }

  res.json({
    sessionId,
    total: session.total,
    completed: session.completed,
    failed: session.failed,
    currentStock: session.currentStock,
    currentYear: session.currentYear,
    status: session.status,
    progress: session.total > 0 ? (session.completed / session.total) * 100 : 0,
    results: session.results
  });
});

2. Real-Time Progress Updates

The app provides live feedback showing:

Current stock being processed
Current year being scraped
Overall progress percentage
Success/failure counts
Detailed results for each operation

3. Intelligent Rate Limiting

To avoid overwhelming NSE servers:

1-second delay between individual requests
2-second delay between different stocks
Proper session initialization with cookies

// Add delay between requests
await new Promise(resolve => setTimeout(resolve, 1000));

// Longer delay between stocks
if (stockIndex < stocks.length - 1) {
  await new Promise(resolve => setTimeout(resolve, 2000));
}

4. Automated File Organization

Downloaded files are automatically organized and can be downloaded as a ZIP:

app.get('/api/download/:sessionId', async (req, res) => {
  // Create zip file
  const archive = archiver('zip', { zlib: { level: 9 } });

  res.attachment(`nse_data_${sessionId}.zip`);
  res.setHeader('Content-Type', 'application/zip');

  archive.pipe(res);
  archive.directory(dataDir, false);
  await archive.finalize();
});

📊 Performance Comparison

Method	Time for 10 stocks (3 years)	Reliability	Resource Usage
Selenium	~45 minutes	Medium	High
Direct API	~4.5 minutes	High	Low

🎨 UI/UX Highlights

The frontend features a modern, responsive design with:

Clean stock management: Easy add/remove interface with validation
Visual progress indicators: Real-time progress bars and status icons
Live updates: Current stock and year display without page refresh
Responsive design: Works seamlessly on desktop and mobile
Modern styling: Clean, professional interface using Lucide React icons

🚀 Deployment & Architecture

Flexible Deployment Options

The project supports multiple deployment strategies:

Single Deployment (Full-Stack)

// Serve static files only if frontend is not deployed separately
if (!process.env.FRONTEND_URL) {
  app.use(express.static('frontend/dist'));
}

// Configure CORS based on deployment type
const corsOptions = {
  origin: process.env.FRONTEND_URL ? 
    [process.env.FRONTEND_URL, 'http://localhost:5173'] : true,
  credentials: true
};

Separate Deployments

Backend: Railway, Render, or any Node.js hosting
Frontend: Vercel, Netlify, or any static hosting

Live Demo Deployment

The live demo is hosted on Render with:

Automatic builds from GitHub
Environment variable configuration
Zero-downtime deployments

💾 Data Format & Quality

Each CSV file contains comprehensive trading data:

Price Data: Open, High, Low, Close, Previous Close, Last Traded Price, VWAP
Volume Data: Total Traded Quantity, Total Traded Value, Number of Trades
Delivery Data: Delivery Quantity, Delivery Percentage
Metadata: Symbol, Series, Date, Timestamps

Sample data structure:

Date,Symbol,Series,Open,High,Low,Close,Last,Prevclose,TOTTRDQTY,TOTTRDVAL,TIMESTAMP,TOTALTRADES,ISIN,DELIVERYQTY,DELIVERYPER
01-Jan-2024,RELIANCE,EQ,2915.00,2932.00,2901.05,2920.15,2920.15,2918.75,5234567,15234567890,01-JAN-2024,89234,INE002A01018,2617283,50.01

🔮 Future Enhancements

Planned improvements include:

Data visualization: Built-in charts and analytics dashboard
Scheduled scraping: Automated daily/weekly downloads with cron jobs
Database integration: Store data in PostgreSQL/MongoDB for persistence
REST API: Additional endpoints for programmatic data access
Export formats: JSON, Excel, and direct database exports
User authentication: Personal dashboards and saved configurations

📝 Key Lessons Learned

API Discovery: Sometimes the best solution is finding the right API endpoint rather than scraping
Session Management: Proper cookie handling and headers are crucial for API access
User Experience: Real-time feedback transforms the user experience
Error Handling: Always plan for network failures and implement retry mechanisms
Performance: Direct API calls can be orders of magnitude faster than browser automation
Deployment Flexibility: Supporting both monolithic and microservice architectures increases adoption

🛠️ Tech Stack Deep Dive

Backend Technologies:

Node.js: Runtime environment for server-side JavaScript
Express.js: Fast, unopinionated web framework
fs-extra: Enhanced file system operations
archiver: ZIP file creation for bulk downloads
https: Native Node.js module for API calls

Frontend Technologies:

React 18: Modern React with hooks
Vite: Lightning-fast build tool and dev server
Axios: Promise-based HTTP client
Lucide React: Beautiful, customizable icons
CSS3: Modern styling with flexbox and grid

DevOps & Deployment:

Render: Cloud hosting platform
GitHub Actions: CI/CD pipeline (potential future addition)
Environment Variables: Configuration management

🎯 Getting Started

Ready to try it yourself?

# Clone the repository
git clone https://github.com/singhanuj620/nse_scrap.git
cd nse_scrap

# Install all dependencies
npm run install-all

# Start development server (runs both frontend and backend)
npm run dev

# Or run backend only
npm run server

# Build for production
npm run build

Quick Test:

# Test with sample stocks
npm test

# Download all configured stocks
npm start

🌟 Try It Live

Don't want to set up locally? Try the live demo:
👉 https://nse-scrap.onrender.com

Features available in the live demo:

Add/remove stocks from the default list
Set custom date ranges
Real-time progress tracking
Download complete datasets as ZIP files
Mobile-responsive interface

💭 Conclusion

Building this NSE scraper taught me the value of choosing the right approach over the obvious one. By leveraging direct API calls instead of browser automation, we achieved a solution that's:

10x faster than traditional scrapers
More reliable with fewer points of failure
Easier to maintain without UI dependency
Better user experience with real-time feedback

The full-stack implementation with session management and real-time progress tracking creates a professional tool that makes bulk data downloading actually enjoyable.

Whether you're a quantitative analyst, researcher, or developer working with Indian stock market data, this approach can save you hours of waiting time and provide more reliable data access.

🔗 Connect & Contribute

Found this helpful? Let's connect!

💼 LinkedIn: Anuj Singh
🐙 GitHub: @singhanuj620
📁 Project Repository: nse_scrap

Have suggestions, found a bug, or want to contribute? Feel free to:

Open an issue on GitHub
Submit a pull request
Connect with me on LinkedIn

Apollo GraphQL - Setup Guide

Anuj Singh — Sat, 06 Apr 2024 12:41:33 +0000

Welcome to the world of Apollo GraphQL, where you can supercharge your JavaScript applications with efficient data fetching, state management, and more. In this technical blog, we'll dive deep into Apollo GraphQL, covering everything from its core concepts to advanced techniques.

What is Apollo GraphQL?

Apollo GraphQL is a comprehensive GraphQL client that enables you to build robust and scalable applications by simplifying data management. It offers a range of features, including caching, state management, and tools for building UI components seamlessly with GraphQL APIs.

Getting Started

To begin using Apollo GraphQL in your JavaScript project, you'll need to install the necessary packages:

npm install @apollo/client graphql

Once installed, you can import the required modules in your application:

import { ApolloClient, InMemoryCache, createHttpLink } from '@apollo/client';
import { setContext } from '@apollo/client/link/context';
import { gql } from '@apollo/client/core';

Creating a Client

Next, you'll need to create an Apollo Client instance, specifying the URI of your GraphQL server:

const httpLink = createHttpLink({
  uri: 'http://localhost:4000/graphql', // Replace with your GraphQL server URI
});

const authLink = setContext((_, { headers }) => {
  const token = localStorage.getItem('token'); // Assuming you store your token in localStorage
  return {
    headers: {
      ...headers,
      authorization: token ? `Bearer ${token}` : '',
    },
  };
});

const client = new ApolloClient({
  link: authLink.concat(httpLink),
  cache: new InMemoryCache(),
});

Executing Queries

Now that you have set up your Apollo Client, you can start executing GraphQL queries. Here's how you can fetch data using Apollo Client:

client.query({
  query: gql`
    query GetPosts {
      posts {
        id
        title
        body
      }
    }
  `,
})
  .then(result => console.log(result))
  .catch(error => console.error(error));

Mutations

Apollo Client also supports GraphQL mutations for modifying data on the server. Here's an example of how you can perform a mutation:

client.mutate({
  mutation: gql`
    mutation AddPost($title: String!, $body: String!) {
      addPost(title: $title, body: $body) {
        id
        title
        body
      }
    }
  `,
  variables: {
    title: 'New Post',
    body: 'This is the content of the new post.',
  },
})
  .then(result => console.log(result))
  .catch(error => console.error(error));

Advanced Features

Apollo GraphQL provides several advanced features, including:

Caching: Apollo Client automatically caches query results, reducing unnecessary network requests.

Local State Management: You can manage local application state alongside remote data using Apollo Client's built-in capabilities.

Pagination: Apollo Client supports pagination out of the box, allowing you to fetch large datasets efficiently.

Error Handling: Easily handle errors returned from GraphQL operations using Apollo Client's error handling mechanisms.

Conclusion

In this blog post, we've explored the fundamentals of Apollo GraphQL and demonstrated how to integrate it into your JavaScript applications. With its powerful features and intuitive API, Apollo GraphQL empowers developers to build modern, data-driven applications with ease. So why wait? Start leveraging the power of Apollo GraphQL in your projects today!

useReducer () | React Hook with project

Anuj Singh — Wed, 14 Sep 2022 18:52:36 +0000

useReducerr

A inbuilt react hook for mimicking Redux concepts in react itself. If you are comfortable with Redux concepts, you will get the jist.

Don't know about REDUX ? 🙈

➡️➡️ Read it here Learn Redux easily with a project

Note : The code snippets or the code reference is from this Github repo, which I have made for this tutorial. It's the basic shopping cart application, that add, removes products to cart and also you can increase or decrease the product quantity.

Github Repo : useReducer_tutorial
Live Demo of the application : Click Here

Let's get started 😈😈👇👇

An alternative to useState. Accepts a reducer of type (state, action) => newState, and returns the current state paired with a dispatch method. (If you’re familiar with Redux, you already know how this works.)

The official definition of useReducer Official Doc

The syntax of declaration is :

const [state, dispatch] = useReducer(reducer, initialArg);

See the main motive behind using useReducer is to have a common place known as store or state in the application, where we can store the data that we wanted which can be accessible to the components directly. They can fetch that data from that state or update that state as required. ( There is a catch in updating the store, we will see later in the blog )

In our shopping cart application, we have 2 main components :

Product Section
Cart Section

These both will be required to access the same state, so we will have to declare the state in their parent component, which in our case is App.js

import { productReducer } from  './reducers'
const  App = () => {
    const  initialState = {
        products: [],
        cart: []
    }  

    const [state, dispatch] = useReducer(productReducer, initialState);

We have declare the initial state to have 2 key value pair:

products []
cart []

and for the reducer we are importing it from another file ( for better readability ).

And for the child component to access the same state, you need to pass these two i.e. state and dispatch as a prop to those child component which you want to interact with same state.

Note : If you are not comfortable with this code that what's state, action, action.payload, type etc., you need to brush up redux
concepts. More about that here : Learn Redux with
project

we will be using dispatch with specific type to update the state as per requirement. We can only update the state using dispatch, **not directly.

Make sure your reducer function accepts 2 parameter i.e state and action

const  productReducer = (state, action) => { }

How Product component will interact with state :

src/components/product/index.js

We are using a dummy product API for fetching some sample products data. So from the product component, we will store the all product details to the state, so that we have access to the product list.

const fetchProduct = async () => {
        const res = await axios.get('https://dummyjson.com/products');
        setProducts(res.data.products);
        dispatch({
            type: 'ADD_TO_PRODUCT',
            payload: res.data.products
        });
    }

Using the dispatch, I'm updating all the product list into state. And on the basis of data present in cart [] of state, Product component is dynamically changing the button to 'add to cart' or 'remove from cart'.

How Cart component will interact with state :

src/components/cart/index.js

const  Cart = ({ state, dispatch }) => {
const { cart } = state;

we will access the state directly as shown. And if any state updating is required as changing the cart item quantity, we will use dispatch with a specific action type and payload data which will instruct the reducer to change the state as per requirement.

const handleInc = (id) => {
        dispatch({
            type: 'INCREASE_QTY',
            payload: id
        });
    }

    const handleDec = (id) => {
        dispatch({
            type: 'DECREASE_QTY',
            payload: id
        });
    }

That's It, You have now understood what is useReducer. 😎😎

You can play around with the github repo for more practice.
Till then ADIOS 🙌

Don't forgot to like 💗

JWT explained with project

Anuj Singh — Mon, 13 Dec 2021 18:48:33 +0000

My Portfolio || My other blogs || Linkedin Account

What is JWT ?

JSON Web Token (JWT) is an open standard ([RFC 7519]) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA.

When should you use JSON Web Tokens?

Authorization
Information Exchange

What is the JSON Web Token structure?

In its compact form, JSON Web Tokens consist of three parts separated by dots (.), which are:

Header
Payload
Signature

Therefore, a JWT typically looks like the following.

xxxxx.yyyyy.zzzzz

JWT In-Depth Knowledge
JWT Playground

To Run the application :

You should have installed nodeJS and postman and MongoDB in your machine.
We will be using postman to test our application. You can use any front-end for the same.
Also make sure mongoDb is installed correctly.

Clone the project
git clone https://github.com/singhanuj620/probackend_jwt
Install NPM dependencies
npm install
Run the server
npm run start
Register a User first.
http://localhost:4000/register
to register a user. Format will be as follows :

Congratulations !!! User is successfully saved in database.

Congratulations !!! User is successfully logged into application.

You can now access the SECRET ROUTE http://localhost:4000/dashboard

Since we have done with using the application, Now let's understand how JWT is working in background. 🙌🙌🙌

Working of JWT in this application

We are using a popular jwt library for encoding and decoding of token. For more info visit JWT NPMJS
And to encrypt the password we are using bCryptJS

We will be using jwt in this flow :

First user is registered into database
User is going to login, and that's where a new token is generated specific for this user
This genrated token is saved into cookies, so that accessing the SECRET route, this jwt token is checked for the confirmation.
BINGOOOOOOO !!!!

When User is going to login

We are going to find the user from database and validate password using bCryptJS, and is everything is okay, then we are going to generate a new token using the jsonwebtoken library.

It has 3 parameters :

what data you want to encode in this token, here i want to encode an object having 2 key i.e user_id and email.
A secret key which can be any random string
Time when this token is going to expires, in this case i have allocated 2 hour lifespan of this token.

const  token = jwt.sign(
    {
        user_id:  user._id, email
    },
    process.env.SECRET_KEY,
    {
        expiresIn:  "2h"
    }
);

and then pass/save this token into cookies to front-end. We are using cookie-parser npm library to do this work.

const  options = {
    expires:  new  Date(Date.now() + 3 * 24 * 60 * 60 * 1000),
    httpOnly:  true
}

res.status(200).cookie('token', token, options).json({
    success:  true,
    token,
    user
});

Okay so now user is logged in, but how should be validate that the token which is generated is authorized user and he/she is requesting to access the SECRET route is allowed ? 🙄🙄🙄

Validating JWT token 💥💥💥

When logged in user tries to access the SECRET route i.e in our case /dashboard then, a middleware auth is going to validate the token.

For that backend has to fetch the token which is set in cookies when user was login.

Note For those who don't know what's middleware. Middleware is just a check between 2 item. If check is okay, proceed to next operation, otherwise return back from there. Imagine it just like watchman 😂.

Everytime when anyone tries to access the SECRET route, this auth middleware will check that if that request has token in cookie or not (only logged in user will have token in cookie). If not, return back.

If token is there in cookie, it will check that token is authentic or not.
HOW ? 🙄🙄
Remember that secret key which was just a random string, yes that will help to decode the token and fetch the data which was encoded into token.

WHICH DATA? 🙄🙄
Remember user_id and email 😉😉

And if everything is good, proceed to the next operation 💯

try {
    const  decode = jwt.verify(token, process.env.SECRET_KEY);
    req.user = decode;
}
catch (err) {
    return  res.status(403).send("Invalid Token")
}
return  next();

What we have learned ?

What are JWT
How to run this github repo application
How to genrate JWT token
How to validate JWT Token

Thank You 🤗 💥🤩

React Redux | Learn the redux easily with project

Anuj Singh — Mon, 05 Jul 2021 10:23:00 +0000

💥💥 React Redux Example 💥💥

This is a todo project for understanding the concepts of redux in react, this github project is referenced in this project

What is Redux ?

Redux is used for state management for the application. It's main advantage is to overcome the problem of "props drilling" among the components.

For example, Let's assume we have 4 component named A,B,C,D and their representation is like :
A ( Parent Component )
|
B
|
C
|
D ( Leaf Component )

and we have a prop named itemCode which is generated at component A, and only to be used at component D.
Traditional Way was to pass that itemCode as an prop to B, then C, and at last to D.

😎😎 How Redux Solves is that it makes an store for an application, which holds all the states of application at one place. So when itemCode is generated at component A, insted of passing it as an props to all the way down to component D, component A will store that itemCode in the Store, and component D will fetch itemCode value form that Store.

Note, But what's the problem in props 🙄🙄 ?
This todo application is very small scale project, in large projects, this props count be large like 10, 20, 100 anything, so using props everywhere will be make the application very very complex 🤯🤯🤯

Let's drive deep into working of Redux

First let's setup the basic project

Create new react project npx create-react-app todo_redux
Install redux dependencies npm install react-redux redux
Make an folder named action and reducer in /src
Inside action folder, make 2 files, /action-types.js and /todo.js
Inside reducer folder, make file named /todo.js
And in /src folder a file, make a file named store.js

After this your project heirarchy will look like :

But what are these Action, Reducer, and Store ?

💥💥 Actions 💥💥 :

It contains the logic that will tell Reducer what should be done on calling specific commands which developer has made. Like in our case, their are 2 action which is made,

Add Todo
Remove Todo

So, Action will tell Reducer that, Okay so there is an call from component to Add Todo, So Hey Reducer, we now have to Add an todo with this that details.

Syntax for Action :

import { ADD_TODO } from "./action-types";
export const addTodo = (todo) => ({
  type: ADD_TODO,
  payload: todo,
});

addTodo is a name of the package which return object with a compulsory field **"type"** and optional field "payload".

Type : It is used to validate action on reducer side that okay an action of type blah blah 😜 type is recieved and I have to perform logic related to this blah blah 🙈 only
Payload : Sometimes on the reducer side, some data is sent which is used to update the state in the store.

💥💥 Reducer 💥💥 :

So till now we have an action which states what to do, but how to do ? is defined in reducer file. For our case, what to do with the state/store when an addTodo action is recieved == We have to add an todo with details recieved from action package into out store which has some previous todos and return the updated state of todos in out store.

Syntax for Reducer

import { ADD_TODO } from "../action/action-types";
const initialState = [];
export const todos = (state = initialState, action) => {
  switch (action.type) {
    case ADD_TODO:
      return [...state, action.payload];
    default:
      return state;
  }
};

See here the types field of action comes into play, to validate what logic is to be performed when an certain type of action is called. And payload data to update the state/store.

There can be many files containing multiple reducers, but at the end we will combine all the reducers from all the files into one root reducer, because the store expects only one reducer. ( we will see how to do that later in this article )

💥💥 Store 💥💥 :

Store is the place where all the states of an application is kept. Store is readonly , for updating, we can only do it through reducer
And also, for an whole application, there will be only 1 store.

Syntax for Store

import { createStore, combineReducers } from  "redux";
import { todos } from  "./reducer/todo";
const  rootReducer = combineReducers(
    todos
});
const  store = createStore(rootReducer);
export  default  store;

Here we import all the reducer file and with the help of combineReducers we combine them into 1 reducer and with createStore we create an store for the application.

So till now we have made an `Store` with root reducers combining all the reducers, `Action` which tells what to do and `Reducer` which tells how to do. Now the question is how to access store from component ? 🤷‍♀️🤷‍♀️🤷‍♀️

Firstly we have to wrap the main parent component inside App.js with
<Provider store={store}> all the components comes here </Provider >

Provider is react-redux magic which handles all the work and store is passed it, which is imported from store file.

Now connecting component to store ✌✌ :

We have to use 2 function which will fetch and sent the data to store through action then reducers all that flow.

Introducing :

mapStateToProps( ) 🤑🤑:

This function is used to fetch the data from store and pass it as a prop to the component, and from then onwards in the component it is used as traditional way like using value from props.

Syntax for mapStateToProps()

const mapStateToProps = (state) => ({
  todos: state.todos,
});

and in component, it will be accessible as props

const TodoForm = ({ todos }) => {};

mapDispatchToProps ( ) 😁😁:

This function is used to prepare action package which will later on go to reducer to update the state of the application. For example in our case, we have to add todo with todo detail so we have to prepare action of type addtodo

Syntax for mapDispatchToProps ()

const mapDispatchToProps = (dispatch) => ({
  addTodo: (todo) => {
    dispatch(addTodo(todo));
  },
});

and in component, it will be accessible as props

const TodoForm = ({ addTodo }) => {};

All Done 🙌🎉🎉

Thank You for reading it 🤗🤗 Hope you like it.

Mongoose "Populate()" in most simple way | How to import a collection into another schema in Mongodb

Anuj Singh — Sun, 18 Apr 2021 10:55:37 +0000

Hello everyone, I'm going to demenstrate and explain the confusing topic of Mongodb/Mongoose that is "Populate( )" function.

Problem Statement :

Suppose you have 2 schema model named :
1 - Address Model
2 - User Model

Address Model:
There are 3 attributes in this i.e pincode, state, address


 javascript
const  mongoose = require("mongoose");
const  Schema = mongoose.Schema;

const  addressSchema = new  Schema({
    address:  String,
    state:  String,
    pincode : Number
});

const  Address= mongoose.model("address", addressSchema);
module.exports = Address;

User Model:
There are 3 attributes in this i.e name, address, and designation.


 javascript
const  mongoose = require("mongoose");
const  Schema = mongoose.Schema;
const {ObjectId} = mongoose.Schema; 

const  userSchema = new  Schema({
    name:  String,
    designation:  String,
    address: {
        type :  ObjectId,
        ref :  "address"
    }
});

const  User = mongoose.model("user", userSchema);
module.exports = User;

Solution :

Can you see the modification in User Model ?
The addition of :


 javascript
address: {
        type :  ObjectId,
        ref :  "address"
    }

makes all the changes here.

We are here not making the entry in User Model's address section but instead using address model to store the data and uses it's __id_ in User Model.

And at time of fetching the data from User Model we will POPULATE the address attribute from the Address Model.

What is Object Id ?


 javascript
const {ObjectId} = mongoose.Schema;

and


 javascript
type :  ObjectId,

The ObjectId is the one of the data types of Mongoose, that tells the mongoose that this is referenced to another collection in MongoDb Database.

After importing, it is used along with ref.

What is ref ?

Now, ObjectId is used along with ref.
Ref tells the Mongoose that in which Collection the importing data is present. In our case, it's the Address Model_ that is being imported and used in the User Model. So,


 javascript
address: {
        type :  ObjectId,
        ref :  "address"
    }

NOTE : The value in ref the same as the


 javascript
const  Address= mongoose.model("address", addressSchema);

in Address Model_.

At last using POPULATE ( )

So, now it's time to fetch the data from the User Model and at time of fetching, filling the address attribute of User Model with data from the Address Model.

Let's suppose,
the __id_ of data in address model is 100.
So, at time of entering data into User Model, pass the __id_ i.e 100 to the address attribute of User Model.


 javascript
User.find({}).populate("address").exec((err, result) => {
    if(err){
        return  res.json({error :  err})
    }
    res.json({result :  result})
    });

.populate("address") will fill the data coming from User.find({}) and find the __id_ present in (in our case __id_ is 100), and find that in Address Model and take that data and fill into User Model.

Thank You

Thank you, for being patient and reading till last, Hope you find it usefull. 🙂🙂🙂

My Portfolio : https://anujportfolio.herokuapp.com/
My Github : https://github.com/singhanuj620
My Linkedin : https://www.linkedin.com/in/anuj-singh-007/

Feedback is always welcomed. 🤗🤗

How to fix nodemon error- nodemon.ps1 cannot be loaded because running scripts is disabled on this system

Anuj Singh — Fri, 19 Mar 2021 13:18:11 +0000

The problem here is with "Get-ExecutionPolicy" value is Restricted. We have to change it to Unrestricted

Step 1
Open Windows PowerShell with Run As Administrator

Step 2
Execute the following command


 Get-ExecutionPolicy

The result you will see is Restricted

Step 3
Execute the following command


 Set-ExecutionPolicy Unrestricted

Done

and it's fixed 🤗🤗🤗🤗

How to scroll to the top while using react-router-dom

Anuj Singh — Sun, 07 Mar 2021 17:49:11 +0000

Hey ✌✌

PROBLEM TO BE SOLVED 🤷‍♀️🤷‍♀️🤷‍♀️🤷‍♀️🤷‍♀️:

When we use react-router-dom library for routing pages and links in the react project, the problem with it is that when clicked on an Link the next component rendered does not get started from the top of the page, instead it renders the page from the scroll height of the parent component i.e the component which holds the Link tag.

So, what we are going to do is to render the new component/page/route from top of the scroll height, not from in-between.

SOLUTION 😎😎😎😎 :

Make a new File just like other component file - Let us name it ScrollToTop.js

import React, { Component } from "react";
import { withRouter } from "react-router-dom";
class ScrollToTop extends Component {
  componentDidUpdate(prevProps) {
    if (this.props.location !== prevProps.location) {
      window.scrollTo(0, 0);
    }
  }
  render() {
    return <React.Fragment />;
  }
}
export default withRouter(ScrollToTop);

Now add this file as a normal component to just below the tag like this :

How Google is competing against Facebook, Twitter, and Linkedin in marketing space? #PeopleCard

Anuj Singh — Wed, 12 Aug 2020 14:08:22 +0000

In social media advertising, some big giants are Facebook, Twitter, and Linkedin because of their way to collect the user's information.

Facebook and Twitter being somewhat gathering informal data about people and Linkedin on the other hand collecting formal aspects of data about people.

In this whole scenario, Google was missing out somewhere. It tried to do the same with Google+, but failed miserably.

But now it has used the biggest weapon it has i.e 🔥 SEARCH ENGINE 🔥 .

By the announcement of 🔥 People's Card 🔥 , Google is also getting into a social media advertising club.

Who all have made one for themselves or going to make? Do share it in comments. !!!

☺☺☺☺☺

UrlShortner using Nodejs and MongoDb | Node.js Project | Javascript

Anuj Singh — Fri, 03 Jul 2020 05:47:55 +0000

Hello everyone, I'm Anuj Singh.

Description about the project

In this post, I am going to show how you can make a URL shortener web application that shorts the provided full URL into short URL and also keeps a count on how many times that short URL has been clicked.

We will be using the following stack for this :

For FrontEnd - HTML5 and CSS3 + Javascript
For Backend - NodeJs
For Database - MongoDB

Let's start

Prerequisites :

1 : Install NodeJs

2 : Install MongoDb

Making The Folder Directory -

Note - Don't Make the node_modules, package.json and package-lock.jsn file now, it will be addressed afterward.

In .gitignore file, write "node_modules". That's it.

Files Used are -

1 : index.js
Starting Point to the Project, aka app.js
2 : shortUrl.js
MongoDb schema for the project
3 : home.ejs
The HTML code wrapped in ejs for dynamic HTML

Now open VSCode (or your own code editor) and open the terminal in the main folder after creating all those files.

Step 1: Run npm init, and fill the corresponding fields like :
(app.js) - index.js
(author) - your name
and yes yes enter enter in other. Even in License tap enter for default entry.

Update
So by now, you have package.json and package-lock.json file in your directory.

Now in command terminal type -
npm install --save ejs express mongoose shortid nodemon

This will install the following npm package that will be used in creating and running of this project i.e

ExpressJs
ShortId
Ejs
Mongoose
Nodemon

(P.S I'm not going in detail of these. You have Google use it)

`Schema i.e shortUrl.js`

const mongoose = require('mongoose');
const shortId = require('shortid');


const shortUrlSchema = new mongoose.Schema({
    full:{
        type: String,
        required: true
    },
    short:{
        type: String,
        required: true,
        default: shortId.generate
    },
    clicks:{
        type:Number,
        required: true,
        default: 0
    }
});

module.exports = mongoose.model('ShortUrl',shortUrlSchema);

In this file, there are 3 sections.
Full: That stores actual URL
Short: That stores an unique short id generated by shortid library.
Clicks: That stores integer value of number of clicks.

`home.ejs`

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css"
        integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
    <title>URL SHORTNER</title>
</head>
<body>
    <div class="container">
        <h1>Shrink URL</h1>
        <form action="/shortUrl" method="POST" class="my-4 form-inline">
            <label for="fullUrl" class="sr-only">Enter Full Url</label>
            <input type="text" required placeholder="Enter Here" name="fullUrl" id="fullUrl" class="form-control mr-2" style="width: 80%;">
            <button type="submit" class="btn btn-md btn-success">Shrink It !!</button>
        </form>

        <table class="table table-striped">
            <thead>
                <tr>
                    <th>Full Url</th>
                    <th>Shrinked Url</th>
                    <th>Clicked</th>
                </tr>
            </thead>
            <tbody>
                <% shortUrls.forEach(shortUrl => { %>
                    <tr>
                        <td><a href="<%=shortUrl.full%>"><%=shortUrl.full%></a></td>
                        <td><a href="<%=shortUrl.short%>"><%=shortUrl.short%></a></td>
                        <td><%=shortUrl.clicks%></td>
                    </tr>
                <% }) %>
            </tbody>
        </table>
    </div>
</body>
</html>

This is the frontend of the application, where the data is presented visually in table format.

`Index.js`

const express = require('express');
const mongoose = require('mongoose');
const ShortUrl = require('./models/shortUrl');
const app = express();


mongoose.connect("<Enter your database connection url here>", {useNewUrlParser: true , 
useUnifiedTopology: true});

app.set('view engine' , 'ejs');
app.use(express.urlencoded({extended: false}));

app.get('/',async (req,res) => {
    const shortUrls = await ShortUrl.find()
    res.render('home',{shortUrls: shortUrls});
});

app.post('/shortUrl', async (req,res) => {
    await ShortUrl.create({full:req.body.fullUrl})
    res.redirect("/");
});

app.get('/:shortUrl', async (req,res) => {
    const shortUrl = await ShortUrl.findOne({short: req.params.shortUrl})
    if(shortUrl == null){
        res.sendStatus(404)
    }
    shortUrl.clicks++;
    shortUrl.save();
    res.redirect(shortUrl.full)
})

app.listen(process.env.PORT || 3000);

NOTE Write your own DB connection url in mongoose.connect()
Here on the submission of the data from the home.ejs page, the actual URL is stored in DB with a random unique shortid associated with it, and default clicks is 0. When someone clicks on the short URL, the shortUrl id is searched in the DB and coresponding actual id is returned back, also increasing the click by 1.

THANK YOU !!!

`Follow for more such stuff`

Instagram : https://instagram.com/anujcodeop
Portfolio : https://anujportfolio.herokuapp.com

singhanuj620 / urlShortner

Web Application made with nodejs and mongoDb that shorts the full url and tracks the number of times it is clicked.

urlShortner

Web Application made with nodejs and mongoDb that shorts the full url and tracks the number of times it is clicked.

LIVE DEMO

Please give it a start too. Thanks !

How to Use it ->

Note - Make Sure you have nodejs and mongoDb installed in the system.

Step 1 : For running it very first time, run npm install

Step 2 : Open 'cmd' as an administrator, and type net start mongoDb for starting the mongoDb database.

Step 3 : In folder terminal, to start the project run npm run devStart

Step 4 : Now open 'https://localhost:3000' in your web browser to use the project.

Step 5 : After using, open again 'cmd' as an administrator, and type 'net stop mongoDb' for stoping the mongoDb database.

View on GitHub

Can I Use 'Variables' in CSS ? Yes !! Let's see how

Anuj Singh — Fri, 03 Jul 2020 05:10:41 +0000

I am into the web development field for a quite long time now, but I am today year old when discovered that we can use variables in our CSS3.

So let's get started.

Assume that you are applying font-family property to 4 or 5 elements and that's where variable comes into action.

The variable with its property is defined and declared in :root with -- prefixed with the variable name.

:root {
    --varfont: 'Roboto', sans-serif;
}

And use it like this :

h1
{
  font-family: var(--varfont);
}

Hope it helped you !!!

Follow me for more such content :
Instagram : https://instagram.com/anujcodeop
Portfolio : https://anujportfolio.herokuapp.com

Using CSS Module in Latest React 16.12.0

Anuj Singh — Mon, 10 Feb 2020 13:49:39 +0000

[It's my first post, so please ignore any grammatical mistake]

In the newer version of reactJS [as per given date 10-FEB-2020 it's 16.12.0], the older way of using npm run eject will not work.

So, to make it work what we can do is ---

Step 1 Making CSS file
While making a CSS file, name it in following convention.
e.g name i want is "index", so instead of naming it "index.css", name it to "index.module.css"
And write css code as usual you are writing earlier.

Step 2 Importing CSS file
While importing the "index.module.css" file in another file, eg in App.js, use following command;

import style from './index.module.css';

style name is upto you, this is used for refering the classes and id inside of css file

for using CSS codes use className={style.Col}
Col is the css code inside index.module.css

DEV Community: Anuj Singh

Building a Lightning-Fast NSE Stock Data Scraper: From API Calls to Full-Stack Web App

Introduction

🚀 What We're Building

🎯 The Problem with Traditional Scraping

💡 The Solution: Direct API Approach

🏗️ Architecture Overview

Backend (Node.js + Express)

Frontend (React + Vite)

🔧 Key Technical Features

1. Session Management

2. Real-Time Progress Updates

3. Intelligent Rate Limiting

4. Automated File Organization

📊 Performance Comparison

🎨 UI/UX Highlights

🚀 Deployment & Architecture

Flexible Deployment Options

Live Demo Deployment

💾 Data Format & Quality

🔮 Future Enhancements

📝 Key Lessons Learned

🛠️ Tech Stack Deep Dive

🎯 Getting Started

🌟 Try It Live

💭 Conclusion

🔗 Connect & Contribute

Apollo GraphQL - Setup Guide

useReducer () | React Hook with project

useReducerr

Don't know about REDUX ? 🙈

➡️➡️ Read it here Learn Redux easily with a project

Let's get started 😈😈👇👇

How Product component will interact with state :

How Cart component will interact with state :

That's It, You have now understood what is useReducer. 😎😎

Don't forgot to like 💗

JWT explained with project

What is JWT ?

When should you use JSON Web Tokens?

What is the JSON Web Token structure?

To Run the application :

Working of JWT in this application

When User is going to login

Okay so now user is logged in, but how should be validate that the token which is generated is authorized user and he/she is requesting to access the SECRET route is allowed ? 🙄🙄🙄

Validating JWT token 💥💥💥

What we have learned ?

Thank You 🤗 💥🤩

React Redux | Learn the redux easily with project

💥💥 React Redux Example 💥💥

This is a todo project for understanding the concepts of redux in react, this github project is referenced in this project

What is Redux ?

Let's drive deep into working of Redux

But what are these Action, Reducer, and Store ?

💥💥 Actions 💥💥 :

Syntax for Action :

💥💥 Reducer 💥💥 :

Syntax for Reducer

💥💥 Store 💥💥 :

Syntax for Store

So till now we have made an Store with root reducers combining all the reducers, Action which tells what to do and Reducer which tells how to do. Now the question is how to access store from component ? 🤷‍♀️🤷‍♀️🤷‍♀️

Now connecting component to store ✌✌ :

mapStateToProps( ) 🤑🤑:

Syntax for mapStateToProps()

mapDispatchToProps ( ) 😁😁:

Syntax for mapDispatchToProps ()

All Done 🙌🎉🎉

Thank You for reading it 🤗🤗 Hope you like it.

Mongoose "Populate()" in most simple way | How to import a collection into another schema in Mongodb

Problem Statement :

Solution :

What is Object Id ?

What is ref ?

At last using POPULATE ( )

Thank You

How to fix nodemon error- nodemon.ps1 cannot be loaded because running scripts is disabled on this system

How to scroll to the top while using react-router-dom

How Google is competing against Facebook, Twitter, and Linkedin in marketing space? #PeopleCard

UrlShortner using Nodejs and MongoDb | Node.js Project | Javascript

Description about the project

So till now we have made an `Store` with root reducers combining all the reducers, `Action` which tells what to do and `Reducer` which tells how to do. Now the question is how to access store from component ? 🤷‍♀️🤷‍♀️🤷‍♀️

`Schema i.e shortUrl.js`

`home.ejs`

`Index.js`

`Follow for more such stuff`