DEV Community: Tikam Singh Alma

rust

Tikam Singh Alma — Sat, 16 Aug 2025 19:33:25 +0000

Stock Fundamental Analysis Using Python and Streamlit

Tikam Singh Alma — Mon, 10 Feb 2025 09:52:35 +0000

Stock Fundamental Analysis Using Python and Streamlit

In today's dynamic stock market, conducting a thorough fundamental analysis can give investors a competitive edge. This blog post delves into how you can leverage Python, Streamlit, and custom modules like YFinance3 to build an automated tool for analyzing stock fundamentals. We'll break down the code and highlight how each part contributes to efficient stock data collection and processing.

Key Features of the Project

Load and process multiple stock symbols from CSV files.
Fetch fundamental data for each stock using the custom YFinance3 module.
Save and load stock data as JSON files for reusability.
Perform key financial metric computations.
Display clean and structured data insights.

Project Structure

.
├── Data/
│   ├── Datasets/
│   └── sources/
│       └── Index/
│           └── StockList.csv
└── main.py

Step-by-Step Code Breakdown

1. Directory and File Initialization

DATA_PATH = './Data/Datasets'
CSV_BASE_PATH = './Data/sources/Index/'

These paths define the locations where CSV files with stock symbols are stored and where processed data will be saved.

2. Handling CSV Input

if not os.path.exists(CSV_BASE_PATH):
    st.error(f"Directory not found: {CSV_BASE_PATH}")
else:
    csv_files = [file for file in os.listdir(CSV_BASE_PATH) if file.endswith('.csv')]
    selected_csv = st.selectbox("Choose a CSV file:", csv_files)

This snippet checks for the existence of the directory and populates a dropdown with available CSV files. Users can select a file containing stock symbols.

3. Data Download and Storage

for symbol in SYMBOLS:
    file_name = os.path.join(folder_path, f'{symbol}.json')
    if os.path.exists(file_name):
        existing_files += 1
        continue

    try:
        data = YFinance3(symbol)
        with open(file_name, 'w') as file:
            json.dump(data.info, file)
        new_downloads += 1
    except Exception as e:
        st.error(f"Error processing {symbol}: {e}")

The code iterates through stock symbols, downloads the data using the YFinance3 module, and saves it in JSON format. It tracks existing files, new downloads, and errors for status reporting.

4. Data Loading and Processing

def load_data(json_data):
    data['Symbol'].append(json_data.get('symbol', np.nan))
    data['Name'].append(json_data.get('longName', np.nan))
    data['Industry'].append(json_data.get('industry', np.nan))
    ...

The load_data function extracts key financial metrics from the JSON files, such as:

EPS (forward)
Price-to-Book Ratio (PB)
Free Cash Flow Yield (FCFY)
52-week Range

The extracted data is appended to a structured dictionary.

5. DataFrame Creation and NaN Handling

df = pd.DataFrame(data)
df_exceptions = df[df.isna().any(axis=1)]
df = df.dropna().reset_index(drop=True)

The code creates a pandas DataFrame from the extracted data, dropping rows with missing values while logging exceptions for user review.

6. Computing the 52-Week Range

df['52w Range'] = ((df['Price'] - df['52w Low']) / (df['52w High'] - df['52w Low'])) * 100

This computation helps assess a stock's position within its 52-week trading range.

7. Displaying and Saving Processed Data

st.write("### Processed Data", df)
output_file = os.path.join(folder_path, 'processed_data.csv')
df.to_csv(output_file, index=False)
st.success(f"Processed data saved to {output_file}")

The cleaned and processed data is displayed in Streamlit and saved as a CSV file for future reference.

Sample Output

Symbol	Name	Industry	Price	EPS (fwd)	PB	FCFY	ROE
AAPL	Apple	Tech	170.3	6.55	9.8	2.3%	42%
MSFT	Microsoft	Tech	310.0	8.20	7.2	3.1%	40%

Potential Enhancements

Stock Screening: Implement filters based on key metrics like ROE and PEG ratios.
Data Visualization: Plot historical trends for key metrics.
Backtesting Strategies: Use fundamental data as input for backtesting stock strategies.
Real-Time Updates: Automate daily updates for new financial data.

Conclusion

This project provides a powerful yet straightforward approach to fundamental stock analysis using Python and Streamlit. It automates data collection, processing, and presentation, empowering users to make better-informed investment decisions.

Let me know if you'd like to enhance any part of the blog or add new features to the code!

ArcticDB Financial and Historical analysis just way faster.

Tikam Singh Alma — Mon, 10 Feb 2025 09:46:58 +0000

Storing and Analyzing Financial Data with ArcticDB

ArcticDB is a high-performance database designed for financial and time-series data, making it an excellent choice for stock data storage and analytics. In this guide, we will explore how to use ArcticDB to efficiently store financial data from CSV files and perform financial analysis using Python.

This solution integrates Streamlit for visualization, ArcticDB for data storage, and yFinance for stock data retrieval.

Key Features

Efficient Data Storage: Store multi-year historical stock data with a powerful ArcticDB backend.
Financial Data Analysis: Perform stock screening, backtesting strategies, and historical data analysis.
User-Friendly Dashboard: Streamlit interface for intuitive interactions.

Step-by-Step Guide

Step 1: Directory and Database Configuration

The project assumes a specific directory structure for storing CSV files and the database.

# Configuration
DATA_DIR = "./Data/sources/Index"
DB_PATH = "lmdb://./stock_db"
CSV_MAPPING = {
    "US_DATA": ["SP500list.csv", "NASDAQ.csv", "NYSE01.csv", "NYSE02.csv"],
    "IND_DATA": ["NIFTY750.csv", "NSE_ALL.csv", "AllCash.csv"]
}

Ensure that the directory contains the required CSV files, each listing stock symbols.

Step 2: ArcticDB Connection Setup

Create a function to establish a connection to ArcticDB.

from arcticdb import Arctic

# Initialize ArcticDB connection
def get_arctic_connection():
    ac = Arctic(DB_PATH)
    return ac

Step 3: CSV File Processing

Fetch available CSV files and map them to appropriate database libraries.

import os

def get_library_name(csv_file):
    csv_lower = csv_file.lower()
    for lib_name, files in CSV_MAPPING.items():
        if any(f.lower() == csv_lower for f in files):
            return lib_name
    if "nse" in csv_lower or "nifty" in csv_lower:
        return "IND_DATA"
    return "US_DATA"

# Get list of available CSV files
def get_available_csvs():
    mapped_files = [f for files in CSV_MAPPING.values() for f in files]
    return [f for f in mapped_files if os.path.exists(os.path.join(DATA_DIR, f))]

Step 4: Updating the Database

Download stock data for the symbols listed in the selected CSV file and store it in ArcticDB.

import yfinance as yf
import pandas as pd
from datetime import datetime
import streamlit as st

def update_database(csv_file, lib_name):
    ac = get_arctic_connection()
    if lib_name not in ac.list_libraries():
        ac.create_library(lib_name)

    lib = ac[lib_name]
    symbols = pd.read_csv(os.path.join(DATA_DIR, csv_file))['Symbol'].tolist()
    arctic_symbol = os.path.splitext(csv_file)[0]

    existing_data = pd.DataFrame()
    if lib.has_symbol(arctic_symbol):
        existing_data = lib.read(arctic_symbol).data

    new_data_list = []
    progress_bar = st.progress(0)
    status_text = st.empty()

    for i, symbol in enumerate(symbols):
        try:
            # Download 2 years of data
            data = yf.download(
                symbol,
                start=(datetime.today() - pd.DateOffset(years=2)).strftime('%Y-%m-%d'),
                end=datetime.today().strftime('%Y-%m-%d'),
                auto_adjust=False
            )

            if not data.empty:
                data = data[['Open', 'High', 'Low', 'Close', 'Volume']]
                data.columns = [c.lower() for c in data.columns]
                data.index = data.index.tz_localize(None)
                data = data.reset_index().assign(symbol=symbol)
                new_data_list.append(data)

            progress_bar.progress((i + 1) / len(symbols))
            status_text.text(f"Processing {i + 1}/{len(symbols)}: {symbol}")
        except Exception as e:
            st.error(f"Error processing {symbol}: {str(e)}")

    if new_data_list:
        new_data = pd.concat(new_data_list)
        new_data.set_index(['symbol', 'Date'], inplace=True)
        new_data.sort_index(inplace=True)

        if not existing_data.empty:
            combined_data = pd.concat([existing_data, new_data])
            combined_data = combined_data[~combined_data.index.duplicated(keep='last')]
        else:
            combined_data = new_data

        lib.write(arctic_symbol, combined_data.sort_index())
        st.success(f"Database {lib_name} updated with {csv_file} data in symbol {arctic_symbol}!")
    else:
        st.warning("No new data was downloaded.")

Step 5: Database Status Overview

Generate a status report for all available databases.

from datetime import datetime

# Check database status
def get_database_status():
    status_data = []
    ac = get_arctic_connection()

    for csv_file in get_available_csvs():
        lib_name = get_library_name(csv_file)
        arctic_symbol = os.path.splitext(csv_file)[0]
        record_count = 0
        latest_date = "N/A"

        if lib_name in ac.list_libraries():
            lib = ac[lib_name]
            if lib.has_symbol(arctic_symbol):
                df = lib.read(arctic_symbol).data
                record_count = len(df)
                latest_date = df.index.get_level_values('Date').max().strftime("%Y-%m-%d") if not df.empty else "N/A"

        status_data.append({
            "CSV File": csv_file,
            "Database": lib_name,
            "Arctic Symbol": arctic_symbol,
            "Records": record_count,
            "Last Updated": latest_date
        })

    return pd.DataFrame(status_data)

Step 6: Running the Streamlit App

Use the main() function to initialize the Streamlit app.

import streamlit as st
import yfinance as yf
import pandas as pd
from arcticdb import Arctic
import os
from datetime import datetime
from glob import glob

# Configuration
DATA_DIR = "./Data/sources/Index"
DB_PATH = "lmdb://./stock_data"
CSV_MAPPING = {
    "US_DATA": [
        "SP500list.csv", "NASDAQ.csv", "NYSE01.csv", "NYSE02.csv", 
    ],
    "IND_DATA": ["NIFTY750.csv", "NSE_ALL.csv", "AllCash.csv"]
}

def get_arctic_connection():
    """Initialize and return ArcticDB connection"""
    ac = Arctic(DB_PATH)
    return ac

def get_library_name(csv_file):
    """Determine library name based on CSV filename"""
    csv_lower = csv_file.lower()
    for lib_name, files in CSV_MAPPING.items():
        if any(f.lower() == csv_lower for f in files):
            return lib_name
    if "nse" in csv_lower or "nifty" in csv_lower:
        return "IND_DATA"
    return "US_DATA"

def get_available_csvs():
    """Get list of available CSV files from mapping that exist in data directory"""
    mapped_files = [f for files in CSV_MAPPING.values() for f in files]
    return [f for f in mapped_files if os.path.exists(os.path.join(DATA_DIR, f))]

def update_database(csv_file, lib_name):
    """Update database with symbols from CSV file using MultiIndex"""
    ac = get_arctic_connection()
    if lib_name not in ac.list_libraries():
        ac.create_library(lib_name)

    lib = ac[lib_name]
    symbols = pd.read_csv(os.path.join(DATA_DIR, csv_file))['Symbol'].tolist()
    arctic_symbol = os.path.splitext(csv_file)[0]  # Use CSV filename as symbol name

    # Load existing data if available
    existing_data = pd.DataFrame()
    if lib.has_symbol(arctic_symbol):
        existing_data = lib.read(arctic_symbol).data

    new_data_list = []
    progress_bar = st.progress(0)
    status_text = st.empty()

    for i, symbol in enumerate(symbols):
        try:
            # Download 2 years of data
            data = yf.download(
                symbol,
                start=(datetime.today() - pd.DateOffset(years=2)).strftime('%Y-%m-%d'),
                end=datetime.today().strftime('%Y-%m-%d'),
                auto_adjust=False
            )

            if not data.empty:
                # Prepare data with MultiIndex
                data = data[['Open', 'High', 'Low', 'Close', 'Volume']]
                data.columns = [c.lower() for c in data.columns]
                data.index = data.index.tz_localize(None)
                data = data.reset_index().assign(symbol=symbol)
                new_data_list.append(data)

            progress_bar.progress((i+1)/len(symbols))
            status_text.text(f"Processing {i+1}/{len(symbols)}: {symbol}")

        except Exception as e:
            st.error(f"Error processing {symbol}: {str(e)}")

    if new_data_list:
        # Combine all new data
        new_data = pd.concat(new_data_list)
        new_data.set_index(['symbol', 'Date'], inplace=True)
        new_data.sort_index(inplace=True)

        # Merge with existing data
        if not existing_data.empty:
            combined_data = pd.concat([existing_data, new_data])
            # Remove duplicates keeping latest data
            combined_data = combined_data[~combined_data.index.duplicated(keep='last')]
        else:
            combined_data = new_data

        # Write to ArcticDB
        lib.write(arctic_symbol, combined_data.sort_index())
        st.success(f"Database {lib_name} updated with {csv_file} data in symbol {arctic_symbol}!")
    else:
        st.warning("No new data was downloaded.")

def get_database_status():
    """Generate status report for all databases"""
    status_data = []
    ac = get_arctic_connection()

    for csv_file in get_available_csvs():
        lib_name = get_library_name(csv_file)
        arctic_symbol = os.path.splitext(csv_file)[0]
        record_count = 0
        latest_date = "N/A"

        if lib_name in ac.list_libraries():
            lib = ac[lib_name]
            if lib.has_symbol(arctic_symbol):
                df = lib.read(arctic_symbol).data
                record_count = len(df)
                latest_date = df.index.get_level_values('Date').max().strftime("%Y-%m-%d") if not df.empty else "N/A"

        status_data.append({
            "CSV File": csv_file,
            "Database": lib_name,
            "Arctic Symbol": arctic_symbol,
            "Records": record_count,
            "Last Updated": latest_date
        })

    return pd.DataFrame(status_data)

def main():
    st.set_page_config(page_title="Stock Database Manager", layout="wide")
    st.title("📈 Stock Data Management System")

    # Sidebar controls
    st.sidebar.header("Database Operations")
    selected_csv = st.sidebar.selectbox("Select CSV File", get_available_csvs())
    lib_name = get_library_name(selected_csv)

    if st.sidebar.button("📥 Download to DB"):
        update_database(selected_csv, lib_name)

    # Main display area
    st.header("Database Status Overview")
    status_df = get_database_status()
    st.dataframe(
        status_df.style.applymap(
            lambda x: "background-color: #e6ffe6" if x != "N/A" else "",
            subset=["Last Updated"]
        ),
        use_container_width=True
    )

    # Show raw data preview
    st.subheader("Selected CSV Preview")
    try:
        csv_path = os.path.join(DATA_DIR, selected_csv)
        df_preview = pd.read_csv(csv_path).head(10)
        st.dataframe(df_preview, use_container_width=True)
    except Exception as e:
        st.error(f"Error loading CSV: {str(e)}")

if __name__ == "__main__":
    main()

Conclusion

By following this guide, you can efficiently manage and analyze stock data using ArcticDB. The provided code snippets help in setting up a robust system for data storage and financial analysis. With features like stock screening, backtesting strategies, and historical analysis, this solution provides a comprehensive platform for financial analytics.

Advanced ArcticDB Usage for Technical Analysis with Python

Tikam Singh Alma — Wed, 05 Feb 2025 17:12:33 +0000

ArcticDB is a powerful tool for efficiently managing and analyzing time-series financial data. When combined with technical analysis techniques, it can streamline backtesting and algorithmic trading strategies. Below, I demonstrate advanced features like deduplication, metadata management, and multi-symbol data analysis for technical indicators.

Why Use ArcticDB for Technical Analysis?

High Performance: Faster read/write operations for large datasets.
Versioning: Keep track of data versions for backtesting.
Metadata Support: Tag datasets with useful information (e.g., source, retrieval time).
Batch Processing: Efficiently handle multiple stock symbols.

Step 1: Fetch and Store Multiple Stocks with Metadata

import yfinance as yf
from arcticdb import Arctic
import pandas as pd

DB_PATH = "lmdb://./advanced_stock_db"
LIBRARY_NAME = "technical_analysis_data"

# Connect and create library
ac = Arctic(DB_PATH)
if LIBRARY_NAME not in ac.list_libraries():
    ac.create_library(LIBRARY_NAME)

lib = ac[LIBRARY_NAME]

symbols = ["AAPL", "MSFT", "GOOG", "TSLA"]

for symbol in symbols:
    ticker = yf.Ticker(symbol)
    hist_data = ticker.history(period="5y", auto_adjust=False)

    # Add metadata with data source and timestamp
    metadata = {"source": "Yahoo Finance", "retrieval_date": pd.Timestamp.now()}

    if not hist_data.empty:
        lib.write(symbol, hist_data, metadata=metadata)
        print(f"{symbol} data stored with metadata.")

Step 2: Batch Read and Compute Technical Indicators

import numpy as np

# Batch read multiple stock data
symbols_data = lib.batch_read(symbols)

for symbol_data in symbols_data:
    df = symbol_data.data
    symbol = symbol_data.symbol

    # Compute Technical Indicators: Bollinger Bands and MACD
    df['SMA_20'] = df['Close'].rolling(window=20).mean()
    df['Upper_Band'] = df['SMA_20'] + 2 * df['Close'].rolling(window=20).std()
    df['Lower_Band'] = df['SMA_20'] - 2 * df['Close'].rolling(window=20).std()

    # MACD calculation
    df['EMA_12'] = df['Close'].ewm(span=12, adjust=False).mean()
    df['EMA_26'] = df['Close'].ewm(span=26, adjust=False).mean()
    df['MACD'] = df['EMA_12'] - df['EMA_26']
    df['Signal_Line'] = df['MACD'].ewm(span=9, adjust=False).mean()

    # Write back updated data
    lib.write(symbol, df)
    print(f"{symbol} updated with technical indicators.")

Step 3: Deduplication and Versioning

# Fetch and update AAPL data to simulate updates
result = lib.read("AAPL")
existing_data = result.data

# Simulate new overlapping data
new_data = existing_data.tail(10)  # Fake overlapping update

# Combine and deduplicate
combined = pd.concat([existing_data, new_data])
deduplicated_data = combined[~combined.index.duplicated(keep='last')]

# Write as a new version
lib.write("AAPL", deduplicated_data, prune_previous_version=False)
print("AAPL data updated without deleting previous version.")

Step 4: Metadata and Symbol Statistics

# Fetch metadata
result = lib.read("AAPL")
print("Metadata for AAPL:", result.metadata)

# Get symbol statistics
info = lib.get_symbol_info("AAPL")
print("Symbol Info for AAPL:", info)

Step 5: Visualization and Analysis

import matplotlib.pyplot as plt

# Plot Bollinger Bands and Closing Prices
df = lib.read("AAPL").data
plt.figure(figsize=(12, 6))
plt.plot(df.index, df['Close'], label='Close Price')
plt.plot(df.index, df['SMA_20'], label='20-Day SMA')
plt.fill_between(df.index, df['Upper_Band'], df['Lower_Band'], color='lightgray', label='Bollinger Bands')
plt.title("AAPL Bollinger Bands")
plt.legend()
plt.show()

Summary of Advanced Features

Feature	Description
Metadata Handling	Store additional metadata per dataset
Batch Processing	Handle multiple symbols simultaneously
Deduplication	Combine and clean overlapping data
Versioning	Maintain multiple versions for backtesting
Technical Indicators	Compute SMA, Bollinger Bands, and MACD

This workflow demonstrates how ArcticDB can handle complex financial analysis tasks efficiently. Want more on strategy backtesting or real-time data analysis?

Using ArcticDB for Financial Analysis with Python and yFinance

Tikam Singh Alma — Wed, 05 Feb 2025 17:08:46 +0000

Using ArcticDB for Financial Analysis with Python and yFinance

Efficient financial data storage and analysis are essential for traders, analysts, and data scientists. ArcticDB, a high-performance database optimized for time-series data, combined with yFinance, makes it easy to fetch, store, and analyze financial data at scale.

Why Use ArcticDB for Financial Data?

Fast read/write operations for large datasets
Supports DataFrame operations like deduplication
Versioning for historical data comparisons
Optimized for time-series data

Step 1: Install Required Libraries

pip install arcticdb yfinance pandas

Step 2: Fetch Stock Data with yFinance

import yfinance as yf
import pandas as pd

# Fetch historical data for Apple (AAPL)
ticker = yf.Ticker("AAPL")
hist_data = ticker.history(period="5y", auto_adjust=False)
print(hist_data.head())

Step 3: Store Data in ArcticDB

from arcticdb import Arctic

DB_PATH = 'lmdb://./stock_db'
LIBRARY_NAME = 'financial_data'

# Connect to ArcticDB and create a library
ac = Arctic(DB_PATH)
if LIBRARY_NAME not in ac.list_libraries():
    ac.create_library(LIBRARY_NAME)

lib = ac[LIBRARY_NAME]

# Write data to ArcticDB
lib.write("AAPL", hist_data)
print("Data written to ArcticDB.")

Step 4: Read and Analyze Data from ArcticDB

# Read data from ArcticDB
result = lib.read("AAPL")
df = result.data
print(df.tail())

# Simple moving average (SMA) example
df['SMA_50'] = df['Close'].rolling(window=50).mean()

# Plot the stock price with SMA
df[['Close', 'SMA_50']].plot(title="AAPL Stock Price with 50-Day SMA")

Step 5: Update and Deduplicate Data

# New data update
new_data = ticker.history(period="1mo", auto_adjust=False)
combined = pd.concat([result.data, new_data])
filtered = combined[~combined.index.duplicated(keep='last')]

lib.write("AAPL", filtered.sort_index())
print("Data updated.")

Conclusion

By integrating ArcticDB and yFinance, you can efficiently fetch, store, and analyze financial data for better decision-making. ArcticDB’s versioning and high-performance capabilities make it a game-changer for large-scale financial analysis.

Want to take your financial analysis even further? Explore more with technical indicators and machine learning techniques!

Ways to resolve CORS issues

Tikam Singh Alma — Mon, 30 Sep 2024 18:22:10 +0000

To resolve CORS issues, you need to add the appropriate headers either in the web server (like Apache or Nginx), in the backend (like Django, Go, or Node.js), or in the frontend frameworks (like React or Next.js). Below are the steps for each platform:

1. Web Servers

Apache

You can configure CORS headers in Apache's configuration files (such as .htaccess, httpd.conf, or apache2.conf), or within a specific virtual host configuration.

Add the following lines to enable CORS:

<IfModule mod_headers.c>
    Header set Access-Control-Allow-Origin "*"
    Header set Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
    Header set Access-Control-Allow-Headers "Content-Type, Authorization"
</IfModule>

To apply CORS for specific domains:

  Header set Access-Control-Allow-Origin "https://example.com"

If credentials are required:

  Header set Access-Control-Allow-Credentials "true"

Ensure the mod_headers module is enabled. If not, enable it using:

sudo a2enmod headers
sudo systemctl restart apache2

Nginx

In Nginx, you can configure CORS headers in the nginx.conf or within a specific server block.

Add the following lines:

server {
    location / {
        add_header Access-Control-Allow-Origin "*";
        add_header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS";
        add_header Access-Control-Allow-Headers "Content-Type, Authorization";
    }

    # Optional: Add for handling preflight OPTIONS requests
    if ($request_method = OPTIONS) {
        add_header Access-Control-Allow-Origin "*";
        add_header Access-Control-Allow-Methods "GET, POST, OPTIONS, PUT, DELETE";
        add_header Access-Control-Allow-Headers "Authorization, Content-Type";
        return 204;
    }
}

If credentials are required:

  add_header Access-Control-Allow-Credentials "true";

Then restart Nginx:

sudo systemctl restart nginx

2. Backend Frameworks

Django

In Django, you can add CORS headers using the django-cors-headers package.

Install the package:

   pip install django-cors-headers

Add 'corsheaders' to INSTALLED_APPS in your settings.py:

   INSTALLED_APPS = [
       ...
       'corsheaders',
   ]

Add the CORS middleware to your MIDDLEWARE:

   MIDDLEWARE = [
       'corsheaders.middleware.CorsMiddleware',
       'django.middleware.common.CommonMiddleware',
       ...
   ]

Set the allowed origins in settings.py:

   CORS_ALLOWED_ORIGINS = [
       "https://example.com",
   ]

To allow all origins:

  CORS_ALLOW_ALL_ORIGINS = True

If credentials are required:

  CORS_ALLOW_CREDENTIALS = True

To allow specific headers or methods:

  CORS_ALLOW_HEADERS = ['Authorization', 'Content-Type']
  CORS_ALLOW_METHODS = ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS']

Go (Golang)

In Go, you can handle CORS manually in the HTTP handler or use a middleware like rs/cors.

Using the rs/cors middleware:

Install the package:

   go get github.com/rs/cors

Use it in your application:

   package main

   import (
       "net/http"
       "github.com/rs/cors"
   )

   func main() {
       mux := http.NewServeMux()

       // Example handler
       mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
           w.Write([]byte("Hello, World!"))
       })

       // CORS middleware
       handler := cors.New(cors.Options{
           AllowedOrigins:   []string{"https://example.com"}, // Or use * for all
           AllowedMethods:   []string{"GET", "POST", "PUT", "DELETE", "OPTIONS"},
           AllowedHeaders:   []string{"Authorization", "Content-Type"},
           AllowCredentials: true,
       }).Handler(mux)

       http.ListenAndServe(":8080", handler)
   }

Node.js (Express)

In Express (Node.js), you can use the cors middleware.

Install the cors package:

   npm install cors

Add the middleware in your Express app:

   const express = require('express');
   const cors = require('cors');
   const app = express();

   // Enable CORS for all routes
   app.use(cors());

   // To allow specific origins
   app.use(cors({
       origin: 'https://example.com',
       methods: ['GET', 'POST', 'PUT', 'DELETE'],
       allowedHeaders: ['Authorization', 'Content-Type'],
       credentials: true
   }));

   // Example route
   app.get('/', (req, res) => {
       res.send('Hello World');
   });

   app.listen(3000, () => {
       console.log('Server running on port 3000');
   });

3. Frontend Frameworks

React

In React, CORS is handled by the backend, but during development, you can proxy API requests to avoid CORS issues.

Add a proxy to the package.json:

   {
     "proxy": "http://localhost:5000"
   }

This will proxy requests during development to your backend server running on port 5000.

For production, the backend should handle CORS. If needed, use a tool like http-proxy-middleware for more control.

Next.js

In Next.js, you can configure CORS in the API routes.

Create a custom middleware for API routes:

   export default function handler(req, res) {
       res.setHeader('Access-Control-Allow-Origin', '*'); // Allow all origins
       res.setHeader('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, OPTIONS');
       res.setHeader('Access-Control-Allow-Headers', 'Authorization, Content-Type');

       if (req.method === 'OPTIONS') {
           // Handle preflight request
           res.status(200).end();
           return;
       }

       // Handle the actual request
       res.status(200).json({ message: 'Hello from Next.js' });
   }

In next.config.js, you can also modify response headers:

   module.exports = {
       async headers() {
           return [
               {
                   source: '/(.*)', // Apply to all routes
                   headers: [
                       {
                           key: 'Access-Control-Allow-Origin',
                           value: '*', // Allow all origins
                       },
                       {
                           key: 'Access-Control-Allow-Methods',
                           value: 'GET, POST, PUT, DELETE, OPTIONS',
                       },
                       {
                           key: 'Access-Control-Allow-Headers',
                           value: 'Authorization, Content-Type',
                       },
                   ],
               },
           ];
       },
   };

Summary of Where to Add Headers:

Web Servers (Apache, Nginx): Configure in server configuration files (e.g., .htaccess, nginx.conf).
Backend Frameworks:
- Django: Use django-cors-headers.
- Go: Manually add headers or use a middleware like rs/cors.
- Node.js (Express): Use the cors middleware.
Frontend: In development, use proxy setups (like React's proxy or Next.js custom headers) to avoid CORS issues, but always handle CORS in the backend in production.

How fixing memory leaks in React can improve overall performance.

Tikam Singh Alma — Fri, 14 Oct 2022 10:45:18 +0000

What is memory leaks in react app and what is the root problem?

Memory leaks are very common and can be very problematic if not handled properly.
A memory leak is a commonly faced issue when developing React applications. It causes many problems, including:

affecting the project's performance by reducing the amount of available memory
slowing down the application
crashing the system
Reduces the overall app latency and performance

According to Wikipedia, a memory leak is a type of resource leak that occurs when a computer program incorrectly manages memory allocations in a way that memory that is no longer needed is not released. A memory leak may also happen when an object is stored in memory but cannot be accessed by the running code.

Simply put, a memory leak is said to occur whenever inaccessible or un-referenced data exists in memory. Nowadays, many modern programming languages have techniques for clearing out data that is no longer needed, garbage collection, but it turns out there are other not-so-popular errors which can expose your React app to memory leaks and, to a great extent, reduce the performance of your app.

A memory leak is a commonly faced issue when developing React applications. A memory leak, in React, is a type of resource leak that occurs when an application incorrectly manages memory allocations. That memory, which is not needed anymore, is not released for other processes to use. A memory leak may also happen when an object is stored in a memory, but cannot be accessed by the running code.

Once you have a memory leak, you will get a similar error in console log

Can't perform a React state update on an unmounted component. This is a no-op, but it indicates a memory leak in your application. To fix, cancel all subscriptions and asynchronous tasks in a useEffect cleanup function.

What causes a memory leak?

A memory leak appears when React applications created subscriptions that were made when a component was mounted and did not unsubscribe them when those components were unmounted.

These subscriptions could be:

DOM Event listeners
WebSocket subscriptions
Requests to an API

Here is a normal scenario that causes this memory leak issue:
Basically, when doing multiple callbacks to api or functions using mounted functions and not unmounting it causes memory leaks.

The user performs an action that triggers an event handler to fetch data from an API.
After that, a user clicks on a link, which navigates to another page before completing step #1.
Now, the first action completes and passes the data retrieved from the API and calls function, which updates the state.

Since the component was unmounted and function is being called in a component that is no longer mounted, it causes memory leak issue -- and in the console, you'll get a warning.

Why should you clean-up or fix the memory leaks?

Memory leaks are commonly faced issues when developing React applications. It causes many problems, including:

Affecting the project’s performance by reducing the amount of available memory
Slowing down the application
Refreshing the page randomly
Crashing the system
Overloading database with huge amounts of queries

What is solution

Basically, we need to remove subscriptions when the component unmounts.
Having understood the problem,what we have to do is, to unmount the compoment or you can cancel the function as soon as API or function calls fetches data or done it's works.
Situation is cancel the request the moment our component unmounts.

Cancel the function call or API call using Abortcontroller
using properly coding

Using Abortcontroller?

What is abortcontroller?
The AbortController interface represents a controller object that allows you to abort one or more Web requests as and when desired.

AbortControllers are created with the new AbortController() syntax, initializing an instance of the AbortController class. Every AbortController object has a read-only signal property which is passed into requests, and an abort() method which is whenever you want to cancel a request.

const Cars = () =>  {
 const [carList, setCarList] = useState([]);
 useEffect( () => {
   let abortController;
   (async () {
     abortController = new AbortController();
     let signal = abortController.signal;
     const { data } = await axios.get('API_ENDPOINT', { signal: signal });
     setCarList(data)
   })();
   return () => abortController.abort();
  }, []);

  return (
   <>...</>
  )
}

Calling abortController.abort() after the request has been completed doesn't throw any errors. The abortController simply does not take any action on an already complete request.

Most of the time this solution don't work.Yes,I have tried this, so the next solution worked 100% everytime.

Using proper return function?

As react developers, we use useEffect() 80% of the time to call API and callback functions.
useEffect() is two edge sword it is used to used to cut propery but it also hurts the holder if not used proplery.

export default function Level02() {
  console.log('renderLevel02');
  const [count, setCount] = useState(0);
  useEffect(() => {
    setInterval(() => {
      setCount(count + 1);
    }, 500);
  });
  return <div>Level 2: count => {count}</div>;
}

Most side-effects happen inside useEffect. This code also has a huge resource leak and is implemented incorrectly. Default behavior of useEffect is to run after every render, so new interval will be created every time count changes.

cleanup the above function

useEffect(() => {
    const interval = setInterval(() => {
      setCount(count + 1);
    }, 300);
    return () => clearInterval(interval);
  }, []);

To prevent resource leaks, everything must be disposed when lifecycle of a hook ends. In this case returned function will be called after component unmounts.

This code does not have resource leaks, but is implemented incorrectly, just like previous one.

use "count" dependency

useEffect(() => {
  const interval = setInterval(() => {
    setCount(count + 1);
  }, 500);
  return () => clearInterval(interval);
}, [count]);

Giving array of dependencies to useEffect will change its lifecycle. In this example useEffect will be called once after mount and every time count changes. Cleanup function will be called every time count changes to dispose previous resource.

functional updates for useState

useEffect(() => {
  const interval = setInterval(() => {
    setCount(c => c + 1);
  }, 500);
  return () => clearInterval(interval);
}, []);

In previous example we ran useEffect on each count change. The was necessary because we needed to have always up-to-date current value.

useState provides API to update previous state without capturing the current value. To do that, all we need to do is provide lambda to setState .

This code works correctly and more efficiently. We are using a single setInterval during the lifecycle of a component. clearInterval will only be called once after component is unmounted.

Now this is the perfect cleanup function.

Thanks for reading!

Best way to learn CSS : CSS Games Online

Tikam Singh Alma — Mon, 15 Aug 2022 18:47:08 +0000

Best way to learn anything is through some engagements and visualization, and gamification it is. With online games you can learn more deeply anything, here are some website where I have learned CSS very deep and in detailed theory and hands-on coding.

FLEXBOX FROGGY

Website - http://flexboxfroggy.com/

Knights Of The Flexbox Table

Website - https://knightsoftheflexboxtable.com/

Flexbox Defense

Website - http://www.flexboxdefense.com/

Flexbox Zombies

Website - https://mastery.games/flexboxzombies/

META Tags to boost SEO and Page Rankings

Tikam Singh Alma — Sat, 23 Jul 2022 18:53:00 +0000

Meta tags are snippets of text that describe a page's content. With Open Graph Protocol meta tags are on steroids, whenever someone share links on Twitter or Whatsapp, we see website thumbnail, gif with description, that's what open graph protocol does to meta tags.

Meta tags are tags within the HTML code of your website that describe the different components of your site to search engines. These tags allow Bing or Google search algorithms to read descriptions of the makeup of your site.

The Open Graph protocol enables any web page to become a rich object in a social graph. For instance, this is used on Facebook to allow any web page to have the same functionality as any other object on Facebook.

Plain META Tags

<meta charset='utf-8'>

<!-- title -->
<title>···</title>
<meta property='og:title'  content='···'>
<meta name='twitter:title' content='···'>

<!-- url -->
<link rel='canonical'       href='http://···'>
<meta property='og:url'  content='http://···'>
<meta name='twitter:url' content='http://···'>

<!-- description -->
<meta name='description'         content='···'>
<meta property='og:description'  content='···'>
<meta name='twitter:description' content='···'>

<!-- image -->
<meta property="og:image"  content="http://···">
<meta name="twitter:image" content="http://···">

<!-- ua -->
<meta http-equiv='X-UA-Compatible' content='IE=edge,chrome=1'>

<!-- viewport -->
<meta name='viewport' content='width=device-width'>
<meta name='viewport' content='width=1024'>

Progressive web apps

Add to homescreen

<meta name='mobile-web-app-capable' content='yes'>
<meta name='apple-mobile-web-app-capable' content='yes'>

<meta name='apple-mobile-web-app-status-bar-style' content='black'>
<!-- black | black-translucent | default -->

Theme color

<meta name='theme-color' content='#ff00ff'>

Android-only. See: Theme color

Manifest

<link rel='manifest' href='/manifest.json'>

Android-only. See: Manifest

Icons

<!-- Minimal -->
<link rel='icon' type='image/png' href='favicon@32.png'>
<link rel='icon' sizes='192x192' href='icon@192.png'>
<link rel='apple-touch-icon' href='icon@152.png'>
<meta name='msapplication-square310x310logo' content='icon@310.png'>

<!-- Apple -->
<link rel='apple-touch-icon' href='touch-icon-iphone.png'>
<link rel='apple-touch-icon' sizes='76x76' href='touch-icon-ipad.png'>
<link rel='apple-touch-icon' sizes='120x120' href='touch-icon-iphone-retina.png'>
<link rel='apple-touch-icon' sizes='152x152' href='touch-icon-ipad-retina.png'>

<!-- Microsoft -->
<meta name='msapplication-square70x70logo' content='icon_smalltile.png'>
<meta name='msapplication-square150x150logo' content='icon_mediumtile.png'>
<meta name='msapplication-wide310x150logo' content='icon_widetile.png'>

The Open Graph Protocol Meta Tags

Basic Metadata

og:title - The title of your object as it should appear within the graph, e.g., "The Rock".
og:type - The type of your object, e.g., "video.movie". Depending on the type you specify, other properties may also be required.
og:image - An image URL which should represent your object within the graph.
og:url - The canonical URL of your object that will be used as its permanent ID in the graph, e.g., "https://www.imdb.com/title/tt0117500/"

Example

<html prefix="og: https://ogp.me/ns#">
<head>
<title>The Rock (1996)</title>
<meta property="og:title" content="The Rock" />
<meta property="og:type" content="video.movie" />
<meta property="og:url" content="https://www.imdb.com/title/tt0117500/" />
<meta property="og:image" content="https://ia.media-imdb.com/images/rock.jpg" />
...
</head>
...
</html>

Optional Metadata
The following properties are optional for any object and are generally recommended:

og:audio - A URL to an audio file to accompany this object.
og:description - A one to two sentence description of your object.
og:determiner - The word that appears before this object's title in a sentence. An enum of (a, an, the, "", auto). If auto is chosen, the consumer of your data should chose between "a" or "an". Default is "" (blank).
og:locale - The locale these tags are marked up in. Of the format language_TERRITORY. Default is en_US.
og:locale:alternate - An array of other locales this page is available in.
og:site_name - If your object is part of a larger web site, the name which should be displayed for the overall site. e.g., "IMDb".
og:video - A URL to a video file that complements this object.

Example

<meta property="og:audio" content="https://example.com/bond/theme.mp3" />
<meta property="og:description" 
  content="Sean Connery found fame and fortune as the
           suave, sophisticated British agent, James Bond." />
<meta property="og:determiner" content="the" />
<meta property="og:locale" content="en_GB" />
<meta property="og:locale:alternate" content="fr_FR" />
<meta property="og:locale:alternate" content="es_ES" />
<meta property="og:site_name" content="IMDb" />
<meta property="og:video" content="https://example.com/bond/trailer.swf" />

Example of Image thumbnail meta for social sharing

<!-- Primary Meta Tags -->  
<title>Meta Tags — Preview, Edit and Generate</title>  
<meta name="title" content="Meta Tags — Preview, Edit and Generate">  
<meta name="description" content="With Meta Tags you can edit and experiment with your content then preview how your webpage will look on Google, Facebook, Twitter and more!">  

<!-- Open Graph / Facebook -->  
<meta property="og:type" content="website">  
<meta property="og:url" content="https://metatags.io/">  
<meta property="og:title" content="Meta Tags — Preview, Edit and Generate">  
<meta property="og:description" content="With Meta Tags you can edit and experiment with your content then preview how your webpage will look on Google, Facebook, Twitter and more!">  
<meta property="og:image" content="https://aws-huddleup.s3.ap-south-1.amazonaws.com/meta-image.png">  

<!-- Twitter -->  
<meta property="twitter:card" content="summary_large_image">  
<meta property="twitter:url" content="https://www.yourwebsite.com/">  
<meta property="twitter:title" content="Website Title">  
<meta property="twitter:description" content="Website Description">  
<meta property="twitter:image" content="https://aws-s3.ap-south-1.amazonaws.com/meta-image.png">

Use these website to create and preview meta tags live and test it.

References:

Promise.all() - JavaScript Function

Tikam Singh Alma — Sat, 23 Jul 2022 18:37:37 +0000

In simple words, a promise is a placeholder for a value that's going to be available sometime later.

The Promise.all() method takes an iterable of promises as an input, and returns a single Promise that resolves to an array of the results of the input promises. This returned promise will fulfill when all of the input's promises have fulfilled, or if the input iterable contains no promises. It rejects immediately upon any of the input promises rejecting or non-promises throwing an error, and will reject with this first rejection message / error.

Try it at - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all

Promises are useful when handling asynchoronous operations.

Promise.all() is a built-in helper that accepts an array of promises (or generally an iterable). The function returns a promise:

const allPromise = Promise.all([promise1, promise2, ...]);

Then you can extract promises resolved values using a then-able syntax:

allPromise.then(values => {

values; // [valueOfPromise1, valueOfPromise2, ...]

}).catch(error => {

error; // rejectReason of any first rejected promise

});

or async/await syntax:

try {    const values = await allPromise;    
     values; // [valueOfPromise1, valueOfPromise2, ...]  
} catch (error) {    
     error;  // rejectReason of any first rejected promise  
     }

If all promises are resolved successfully, then allPromise fulfills with an array containing fulfilled values of individual promises. The order of promises in the array does matter — you'll get the fulfilled values in that order.
But if at least one promise rejects, then allPromise rejects right away (without waiting for other promises to resolve) with the same reason.

Graphql requests with CURL

Tikam Singh Alma — Sat, 23 Jul 2022 18:29:21 +0000

Make a GraphQL query request

curl 'https://countries.trevorblades.com/' \

-X POST \

-H 'content-type: application/json' \

--data '{

"query": "{ continents { code name } }"

}'

Make a mutation request

curl 'https://graphql-api-url' \

-X POST \

-H 'content-type: application/json' \

--data '{

"query":"mutation { createUser(name: \"John Doe\") }"

}'

Pass additional headers

If you need to pass authorization in a header, add another -H argument:

curl 'https://countries.trevorblades.com/' \
  -X POST \
  -H 'Authorization: Token xxxxx' \
  -H 'content-type: application/json' \
  --data '{
    "query": "{ continents { code name } }"
  }'

Printing response headers

If you want to see response headers, add -D - (dump headers to stdout):

curl 'https://countries.trevorblades.com/' \
  -X POST \
  -D - \
  -H 'content-type: application/json' \
  --data '{
    "query": "{ continents { code name } }"
  }'

Pretty printing response JSON

If you want to pretty print the output JSON, add | python -m json.tool in the end of the curl call.

curl 'https://countries.trevorblades.com/' \
  -X POST \
  -H 'content-type: application/json' \
  --data '{
    "query": "{ continents { code name } }"
  }' | python -m json.tool

Configuring Domains and Sub-Domains in Apache webserver.

Tikam Singh Alma — Tue, 15 Mar 2022 08:40:40 +0000

What are the Sub-domains?Why are Domains and Sub-Domains required?

Domains are well website name or the identity or a website, that one buys from a domain registrar website like google or GoDaddy and that is the address of your personal or professional website.
And subdomains are the subset or part of a personal or professional website, where users can add and extend the main domain's functions and features.
For example, a user wants to start his Food tech business and also wants to write blogs about food, and also wants to tracks the business transactions of his business.

Each feature has different functionality on the application side, the calculation, tracking, and analysis on the dashboard, the text editor and content management for the blog, etc.

Here, comes the subdomains part, why we need it.
The user will create his home page on a different directory, blog features have different code and stores on the different directory and same for dashboard also, and now the apache webserver will be able to manage all these directories and able to handle the requests on domains and sub-domains using configuration on the virtual host.

Configuring VirtualHost to Enable Domains and Sub-Domains

Change directory to the /etc/apache2/ and select the configuration file which you want to edit for adding subdomains.
There must be two files one is my-domain.conf and the other is default configuration 000-default.conf

$ sudo vi /etc/apache2/sites-available/my-domain.conf
OR 
$ sudo vi /etc/apache2/sites-available/000-default.conf

Add Wildcard Subdomain

Home Page - mysite.com (Main Domain)

 <VirtualHost *:80>
   DocumentRoot /var/www/home
   ServerName mysite.com
 </VirtualHost>

Blog Page - blog.mysite.com (Sub-Domain)

<VirtualHost *:80>
   DocumentRoot /var/www/blog
   ServerName blog.mysite.com
 </VirtualHost>

Dashboard - dashboard.mysite.com (Sub-domain)

<VirtualHost *:80>
   DocumentRoot /var/www/dash
   ServerName dashboard.mysite.com
 </VirtualHost>

Wildcard Domain

<VirtualHost *:80>
     ServerAlias *.mysite.com
     DocumentRoot /var/www/html/wildcard/
 </VirtualHost>

Final Configuration

## HomePage
 <VirtualHost *:80>
   DocumentRoot /var/www/home
   ServerName mysite.com
 </VirtualHost>

## Blog Page
<VirtualHost *:80>
   DocumentRoot /var/www/blog
   ServerName blog.mysite.com
 </VirtualHost>

## Dashboard Page
<VirtualHost *:80>
   DocumentRoot /var/www/dash
   ServerName dashboard.mysite.com
 </VirtualHost>

## Wild Card
<VirtualHost *:80>
     ServerAlias *.mysite.com
     DocumentRoot /var/www/html/wildcard/
 </VirtualHost>

Testing and Dry Run the Configuration

Check the configuration by running the command:

sudo apachectl configtest

Reload and Restart the server

# Reload
$ sudo service apache2 reload

# Restart
$ sudo service apache2 restart