DEV Community: instanceofGod

PIN Encryption & Decryption with RSA in Node.js

instanceofGod — Tue, 17 Mar 2026 16:41:57 +0000

Why PIN encryption matters
Core terms defined
How RSA works
Anatomy of a key pair
The OpenSSLUtil class, line by line
The full encrypt → transmit → decrypt flow
Working with keys in the terminal
Common pitfalls & security notes
Conclusion

1. Why PIN encryption matters

Every time a user taps their 4-digit PIN into a mobile banking app or payment terminal, those four digits travel from device to server. If they travel as plain text — say, {"pin": "1234"} — anyone who can intercept the HTTP request, peek at server logs, or read a database dump will see that PIN immediately. That is a catastrophic failure mode.

The solution is asymmetric encryption. The server holds a private key that it never shares. The client has a copy of the corresponding public key. The client encrypts the PIN before it leaves the device, and only the server — with its private key — can decrypt it. No intermediary, no log, no network sniffer can read the PIN in transit.

The code snippet in this article is a production-ready implementation of this pattern, built on Node.js's built-in crypto module using RSA-OAEP encryption with SHA-256 — the modern, recommended approach. This article walks through every part of it, explaining the concepts before the code.

2. Core terms defined

Before touching a single line of code, you need a working vocabulary. These terms will come up again and again.

Term	Definition
Plaintext	The original readable data before any encryption. In our case, the PIN — e.g. the string `"1234"`.
Ciphertext	The scrambled output after encryption. Looks like random bytes. Example: a 256-byte binary blob that encodes to a long base64 string.
Encryption	The process of transforming plaintext into ciphertext using a key and an algorithm. Without the correct decryption key, ciphertext is meaningless.
Decryption	The reverse: transforming ciphertext back into plaintext using the correct key.
RSA	Rivest–Shamir–Adleman. A widely used asymmetric encryption algorithm based on the mathematical difficulty of factoring very large numbers.
Asymmetric encryption	A scheme that uses two different keys: a public key to encrypt, and a private key to decrypt. You can freely share the public key — knowing it does not help you decrypt.
Symmetric encryption	A scheme that uses the same key for both encryption and decryption (e.g. AES). Fast, but the key must be shared securely — which is the hard part.
Public key	One half of the RSA key pair. Safe to share with anyone. Used to encrypt data or verify signatures.
Private key	The other half. Must be kept secret. Used to decrypt data or create signatures. If an attacker gets this, all security is lost.
OAEP padding	Optimal Asymmetric Encryption Padding. The modern, secure padding scheme for RSA. Uses a hash function (SHA-256 in this codebase) to produce randomised, tamper-resistant ciphertext. Resistant to padding oracle attacks.
PKCS#8	A standard format for encoding private keys. Node.js v17+ requires private keys to be in PKCS#8 format (begins with `-----BEGIN PRIVATE KEY-----`).
PEM format	Privacy Enhanced Mail. A base64-encoded wrapper for certificates and keys with a human-readable header and footer. The `-----BEGIN ...-----` / `-----END ...-----` lines you see in key files.
Base64	An encoding (not encryption!) that converts binary data into printable ASCII characters. Used to safely include binary ciphertext in JSON or URL parameters.
Digital signature	A cryptographic proof that a message was created by the holder of a private key and has not been tampered with. Created with the private key; verified with the public key.
Buffer	Node.js's representation of raw binary data. Crypto operations work on Buffers, not JavaScript strings.
SHA-256	A cryptographic hash function that produces a fixed 256-bit digest of any input. Used both in OAEP padding and in the signing algorithm.

3. How RSA works

You do not need to understand the mathematics in depth, but a conceptual model will prevent a lot of confusion later.

RSA relies on a simple fact: multiplying two large prime numbers together is easy, but factoring the result back into those two primes is computationally infeasible. The public and private keys are derived from this relationship — one cannot be practically computed from the other.

Encryption with the public key:

Anyone can encrypt a message
Freely distribute your public key
OAEP padding makes output non-deterministic
Only the private key holder can read it

Decryption with the private key:

Only the key holder can decrypt
Private key never leaves the server
Decryption always yields the same original plaintext
Losing the private key = losing access

A common analogy: the public key is like an open padlock you hand to everyone. Anyone can snap it shut (encrypt). Only you have the physical key that opens it (decrypt).

Note — RSA is not for bulk data

RSA can only encrypt data smaller than its key size minus padding overhead. A 2048-bit key with OAEP-SHA256 can encrypt at most ~190 bytes of data. This is perfect for short secrets like PINs, passwords, or symmetric keys — but you would never RSA-encrypt a file. For large data, RSA is used to encrypt a random AES key, then AES encrypts the actual data. This hybrid approach is what TLS does.

4. Anatomy of a key pair

The example gives you a 2048-bit RSA key pair. Let's understand what you are looking at.

The public key

Safe to share, embed in your mobile app, or publish in your documentation. It begins with:

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAnFBPODvVUdzXxGkXG0OP
HQ0zDS3xf1uD7tY8mJKWrjSLMf2CeZtWYNlT2xq40R7x5VTn1rnKbgnBZg2ynyLh...
-----END PUBLIC KEY-----

This is PEM format — the binary key data encoded in base64, wrapped with header and footer lines. The actual key material is the base64 between the markers.

The private key (PKCS#8)

Never share this. It begins with -----BEGIN PRIVATE KEY----- (not BEGIN RSA PRIVATE KEY, which is the older PKCS#1 format). Node.js v17+ requires PKCS#8. This is why the setup guide includes a conversion command:

# Convert PKCS#1 to PKCS#8 (Node.js v17+ requires this)
openssl pkcs8 -topk8 -nocrypt -in old_private.pem -out new_private_pkcs8.pem

Key format summary

Header line	Format	Used for
`BEGIN RSA PRIVATE KEY`	PKCS#1	Older OpenSSL tools. Not accepted by Node.js v17+.
`BEGIN PRIVATE KEY`	PKCS#8	Modern standard. Required by Node.js `createPrivateKey`.
`BEGIN ENCRYPTED PRIVATE KEY`	PKCS#8 encrypted	Private key protected with a passphrase.
`BEGIN PUBLIC KEY`	X.509 SubjectPublicKeyInfo	Standard public key format.

5. The OpenSSLUtil class, line by line

Now let's walk through the implementation. The class has three main responsibilities: loading the key pair, encrypting with the public key, and decrypting with the private key. There is also a bonus signing method.

Constructor and key loading

class OpenSSLUtil {
  constructor() {
    this.keyPair = this._readKeyPair();
    // Key parsing happens once at startup — not on every request.
  }

  _readKeyPair() {
    const { privateKey: privatePem, passphrase } = config.rsaConfig;

    const privateKey = crypto.createPrivateKey({
      key: privatePem,
      format: 'pem',
      ...(passphrase && { passphrase })
    });

    // Derive the public key from the private key — no need to store both
    const publicKey = crypto.createPublicKey(privateKey);

    return { privateKey, publicKey };
  }
}

Why derive the public key from the private key?

A private key mathematically contains all the information needed to reconstruct the public key. Calling crypto.createPublicKey(privateKey) is safe, convenient, and avoids having to store or sync two separate key files. You only need to protect the private key.

Encrypt method — OAEP-SHA256

encrypt(text) {
  try {
    const encrypted = crypto.publicEncrypt(
      {
        key: this.keyPair.publicKey,
        padding: crypto.constants.RSA_PKCS1_OAEP_PADDING,
        oaepHash: 'sha256'
        // OAEP with SHA-256: secure, modern, tamper-resistant
      },
      Buffer.from(text)  // convert string → binary Buffer
    );
    return encrypted.toString('base64');  // binary → base64 string
  } catch (e) {
    logger.error('Could not encrypt data: ', e);
    return null;
  }
}

Breaking this down step by step:

Buffer.from(text) — Converts the string "1234" into raw bytes. RSA encryption operates on binary data, not JavaScript strings.
RSA_PKCS1_OAEP_PADDING — Applies OAEP padding using SHA-256 as the hash function. OAEP uses a random seed internally, so encrypting the same PIN twice produces completely different ciphertexts — making brute-force and rainbow-table attacks impractical.
oaepHash: 'sha256' — Specifies SHA-256 as the hash used inside the OAEP padding scheme. This must match on both encrypt and decrypt.
.toString('base64') — Encodes the binary output as a base64 string, making it safe to include in JSON request bodies or HTTP headers.

Decrypt method — OAEP-SHA256

decrypt(encrypted) {
  try {
    // Accept either a base64 string or a raw Buffer
    const buffer = typeof encrypted === 'string'
      ? Buffer.from(encrypted, 'base64')  // base64 string → binary Buffer
      : encrypted;                         // already a Buffer, use as-is

    const decrypted = crypto.privateDecrypt(
      {
        key: this.keyPair.privateKey,
        padding: crypto.constants.RSA_PKCS1_OAEP_PADDING,
        oaepHash: 'sha256'  // must match what was used in encrypt()
      },
      buffer
    );
    return decrypted.toString('utf8');  // binary → original string
  } catch (e) {
    logger.error('Could not decrypt data: ', e);
    return null;
  }
}

Warning — Padding scheme and hash must match on both ends

Both encrypt() and decrypt() use RSA_PKCS1_OAEP_PADDING with oaepHash: 'sha256'. If you encrypt with one hash and attempt to decrypt with another, decryption will silently fail and return null. Always keep these parameters consistent across your entire stack — including any client-side or terminal-based encryption.

Sign method

Signing is separate from encryption. A signature does not hide the content — it proves authorship and integrity. You sign with your private key; anyone with your public key can verify the signature.

sign(text) {
  try {
    const sign = crypto.createSign('SHA256');
    // 1. Feed the data into the sign object
    sign.update(text, 'utf8');
    sign.end();
    // 2. Finalize: hash the data with SHA-256, then RSA-sign the hash
    return sign.sign(this.keyPair.privateKey, 'base64');
  } catch (e) {
    logger.error('Could not sign data: ', e);
    return null;
  }
}

The signing algorithm does two things internally: it first runs SHA256(text) to produce a 32-byte digest, then RSA-encrypts that digest with the private key. The result is a 256-byte (2048-bit) signature, returned as base64.

Module export as singleton

// Export a single shared instance (not the class itself)
module.exports = new OpenSSLUtil();

// Usage elsewhere in the codebase:
const openssl = require('./utils/openssl-util');
const encryptedPin = openssl.encrypt('1234');
const recoveredPin = openssl.decrypt(encryptedPin);

Exporting new OpenSSLUtil() rather than the class itself means the key parsing happens once at startup. Every module that requires this file gets the same instance with the already-loaded keys — no repeated I/O or parsing overhead per request.

6. The full encrypt → transmit → decrypt flow

Here is the complete lifecycle of a PIN from a client device to a server response, as the codebase implements it.

Client has public key
        ↓
User enters PIN
        ↓
Encrypt with public key (OAEP-SHA256)
        ↓
Base64 encode
        ↓
HTTPS POST
        ↓
Server decrypts with private key
        ↓
Plaintext PIN

Step 1 — Client encrypts the PIN (terminal simulation)

# Encrypt "1234" using OAEP with SHA-256 and output as base64
echo -n "1234" | openssl pkeyutl -encrypt -pubin -inkey public.pem \
    -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256 | base64

# Output:
mK9xZq2Tp7fVnBsLuR4dAe8cWyOhXiGj...
# Run again — you'll get a completely different string (OAEP is non-deterministic)

Step 2 — Client sends the encrypted PIN

{
  "pin": "mK9xZq2Tp7fVnBsLuR4dAe8cWyOhXiGj..."
}

Why is this safe even over plain HTTP?

Even if an attacker intercepts this JSON, they see only the base64 ciphertext — not the PIN. Without the private key (which never leaves the server), they cannot decrypt it. That said, you should still use HTTPS. Defence in depth means multiple layers of protection.

Step 3 — Server decrypts and verifies

const openssl = require('../../utils/openssl-util');

router.post('/', async (req, res) => {
  const { username, pin } = req.body;

  // Decrypt the incoming base64 string back to plaintext
  const decryptedPin = openssl.decrypt(pin);

  if (!decryptedPin) {
    return res.status(400).json({ error: 'Invalid encrypted PIN' });
  }

  // Never store the raw PIN — hash it before comparison
  const pinHash = crypto
    .createHash('sha256')
    .update(decryptedPin)
    .digest('hex');

  // Compare hash against what is stored in the database
  const user = await findUserByPhone(phonenumber);
  if (!user || user.pinHash !== pinHash) {
    return res.status(401).json({ error: 'Invalid credentials' });
  }

  const token = generateSessionToken(user.id);
  res.json({ token });
});

7. Working with keys in the terminal

The provided setup guide includes several OpenSSL terminal commands. Here is what each one does and when to use it.

Generate a fresh key pair

# Generate a 2048-bit RSA private key and immediately convert to PKCS#8
openssl genrsa 2048 | openssl pkcs8 -topk8 -nocrypt -out private_pkcs8.pem

# Derive the public key from the private key
openssl rsa -in private_pkcs8.pem -pubout -out public.pem

Encrypt a PIN for testing (OAEP)

# Encrypt "1234" using OAEP with SHA-256 and output as base64
echo -n "1234" | openssl pkeyutl -encrypt -pubin -inkey public.pem \
    -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256 | base64

# Verify round-trip: decrypt the output back to the original PIN
echo "mK9xZq2..." | base64 -d | openssl pkeyutl -decrypt \
    -inkey private_pkcs8.pem -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha256
# Output: 1234

Convert the key to a JSON-safe escaped string

Config files like config.json cannot store multi-line strings. The private key PEM has newlines, so you need to escape them as \n.

# Node.js method
node -e "console.log(require('fs').readFileSync('private_pkcs8.pem','utf8').replace(/\n/g,'\\n'))"

# Python method (often more reliable across platforms)
python3 -c "print(open('private_pkcs8.pem').read().replace('\n', '\\n'))"

# awk method
awk 'NF {printf "%s\\n", $0} END {print ""}' private_pkcs8.pem

The resulting single-line string can be pasted directly into your config:

{
  "config": {
    "privateKey": "-----BEGIN PRIVATE KEY-----\nMIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEA...\n-----END PRIVATE KEY-----\n"
  }
}

Warning — Never commit private keys to version control

The private key in this article is a test key. Never commit a real private key to a git repository, even a private one. Store keys in environment variables, a secrets manager (AWS Secrets Manager, HashiCorp Vault), or an encrypted config service.

8. Common pitfalls & security notes

Don't log decrypted PINs

The catch blocks in the utility log errors. Never add a console.log(decryptedPin) for debugging. Application logs often end up in log aggregation platforms, and PINs in plaintext in logs are a compliance violation.

Singleton means shared state

The module exports new OpenSSLUtil() — a singleton. The key pair is shared across all requests. This is fine if keys are read-only after initialization. But if you ever implement key rotation, you need to handle the transition carefully to avoid decryption failures on in-flight requests.

OAEP encryption is non-deterministic — that's a feature

Encrypting "1234" twice will produce two completely different ciphertexts. OAEP padding incorporates a random seed on every operation. This means an attacker cannot use a pre-computed rainbow table to map ciphertexts back to PINs, and cannot tell whether two encrypted values are the same PIN. Each encryption is cryptographically unique.

Encryption vs signing — knowing when to use each

Scenario	Use	Why
Sending a PIN from client to server	Encrypt (public key, OAEP)	Keeps PIN confidential in transit
Server proves a response came from it	Sign (private key)	Client can verify with public key
Client proves a request came from it	Sign with client's private key	Requires client-side key management
Storing a PIN for comparison	Hash (bcrypt / SHA-256)	One-way — even server can't recover plaintext

9. Conclusion

PIN encryption with RSA-OAEP in Node.js is a well-trodden pattern that, when implemented correctly, provides strong guarantees: a PIN that leaves a client device is unreadable to anyone who intercepts it, because only the server's private key can unlock it.

The key ideas to carry forward are:

Asymmetric encryption solves the key-distribution problem. Share the public key freely; guard the private key absolutely.
OAEP with SHA-256 is the modern, secure padding choice. It is non-deterministic, tamper-resistant, and immune to padding oracle attacks — use it for all new RSA encryption.
Buffer ↔ string conversions are explicit in Node.js. Always know whether you are working with binary data or a string, and which encoding (base64, utf8, hex) bridges them.
PKCS#8 is the modern format. If you see BEGIN RSA PRIVATE KEY, convert it before passing it to Node.js v17+.
Encryption is not storage. After decrypting a PIN on the server, hash it before comparing to the stored value. RSA encryption is for transit security, not at-rest security.

From here, your natural next steps are implementing key rotation, considering hybrid encryption (RSA wrapping an AES key) for any payloads larger than a short PIN, and exploring the browser's WebCrypto API for client-side encryption using the same OAEP scheme.

Building a Production-Ready RAG Application with FastAPI, LangChain, and Google Gemini: A Deep Dive

instanceofGod — Fri, 20 Feb 2026 20:02:38 +0000

Picture this: You’ve just deployed a shiny new AI assistant for your company. Your stakeholders are thrilled. Then, a highly-valued client asks it a crucial question about your brand-new, unreleased product documentation. The AI, possessing all the confidence in the world, proceeds to completely hallucinate an answer based on what it read on the internet two years ago.

This is the fatal flaw of standard Large Language Models (LLMs). Out of the box, they are incredibly articulate but fundamentally ignorant about your proprietary, real-time, or deeply domain-specific data. They confidently guess when they don't know the answer.

Enter Retrieval-Augmented Generation (RAG)—the silver bullet for LLM hallucination.

Think of RAG as giving your AI an open-book test. Instead of relying on its static memory, a RAG system securely searches your private databases for the exact, relevant context and injects it directly into the prompt before the LLM generates a response. You instantly transform a generic AI into a precise domain expert on your custom data.

While there are plenty of simple "hello world" RAG tutorials out there, building something that scales requires careful software design. In this comprehensive deep-dive tutorial, we aren't just writing toy Jupyter cells. I will walk you through building a modular, multi-storage, production-ready RAG web service from scratch using FastAPI, LangChain, and Google Gemini—an architecture that you can actually deploy to the real world.

1. Our Technology Stack

Instead of bloated orchestrators, we'll build on a lean, highly performant stack:

FastAPI: A modern, wildly fast web framework for building our backend API. It gives us async capabilities, automatic data validation, and built-in Swagger UI.
LangChain & LCEL: The leading framework to orchestrate LLM workflows. We'll heavily utilize LangChain Expression Language (LCEL) for declarative, readable pipelines.
Google Generative AI (Gemini): We'll power our system using two models:
- gemini-embedding-001 for mapping text to vectors.
- gemini-2.5-flash for blindingly fast and intelligent text generation.
Durable Vector Stores: To store and query embeddings, our application dynamically supports:
- Pinecone: A fully managed, cloud-native vector database.
- FAISS: Facebook AI Similarity Search for blistering local performance.
- Cloud Backups: Built-in support for synchronizing our FAISS indexes with AWS S3 or Google Cloud Storage (GCS).

2. Project Architecture

Before jumping into the implementation, let's map out our project structure. A good RAG application separates the HTTP layer, the orchestration layer, and the data layer.

rag-app/
├── main.py                          # FastAPI application entry point
├── endpoints.py                     # API route definitions
├── rag.py                           # Core RAG logic & LLM orchestration
├── vector_stores/                   # Vector Database integrations
│   ├── __init__.py
│   ├── pinecone_store.py            # Managed Cloud Vector DB
│   ├── faiss_store.py               # Local Disk FAISS
│   ├── s3_store.py                  # AWS S3 backed FAISS
│   └── gcs_store.py                 # Google Cloud Storage backed FAISS
├── data/                            # Your raw PDFs, TXTs, MDs, etc.
├── .env                             # Environment variables
└── requirements.txt                 # Dependencies

To follow along, make sure your core dependencies are installed:

pip install fastapi uvicorn langchain langchain-core langchain-google-genai langchain-pinecone pinecone pydantic python-dotenv faiss-cpu google-cloud-storage boto3

3. The Data Layer: Multi-Backend Vector Stores

Vector stores hold our document "embeddings" (mathematical representations of text in n-dimensional space). This allows us to perform rapid "similarity searches" using cosine similarity or euclidean distance to find text chunks semantically related to a user's question.

Because production environments vary, we designed our data layer to be dynamic based on environment variables.

Option A: Cloud-Native with Pinecone

For a fully managed experience, we use Pinecone (vector_stores/pinecone_store.py):

import os
from langchain_pinecone import PineconeVectorStore
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import DirectoryLoader
from pinecone import Pinecone

def setup_pinecone(embeddings, data_dir):
    """Setup Pinecone vector store"""
    index_name = os.getenv("PINECONE_INDEX_NAME", "rag-index")
    pc = Pinecone(api_key=os.getenv("PINECONE_API_KEY"))
    index = pc.Index(index_name)

    # Idempotency: Avoid re-processing if the index is already populated
    if index.describe_index_stats()['total_vector_count'] > 0:
        print("Loading existing Pinecone index...")
        return PineconeVectorStore(index=index, embedding=embeddings)

    print("Creating new Pinecone index from documents...")
    loader = DirectoryLoader(data_dir, glob="**/*.*")
    documents = loader.load()

    # Chunk the documents to prevent hitting LLM context windows
    splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
    document_chunks = splitter.split_documents(documents)

    return PineconeVectorStore.from_documents(document_chunks, embeddings, index_name=index_name)

Key Takeaways here:

Idempotency: We check index.describe_index_stats() to ensure we don't ingest the same data multiple times on server restarts.
Chunking: RecursiveCharacterTextSplitter breaks huge documents into smaller 500-character chunks with a 50-character overlap (to prevent cutting ideas in half).

Option B: Local & Cloud-Backed FAISS

If you don't want to use a managed service, FAISS is incredibly fast. However, FAISS runs in-memory. To make it durable across deployments, we've implemented loaders that sync the index directory to S3 or GCS.

Here is a look at our GCS implementation (vector_stores/gcs_store.py):

from google.cloud import storage
from langchain_community.vectorstores import FAISS
# ... imports omitted for brevity

def save_faiss_to_gcs(vector_store, bucket_name, gcs_prefix="faiss_index"):
    local_path = "/faiss_index"
    vector_store.save_local(local_path)
    client = storage.Client()
    bucket = client.bucket(bucket_name)

    # Upload local FAISS index files to GCS
    for file in os.listdir(local_path):
        blob = bucket.blob(f"{gcs_prefix}/{file}")
        blob.upload_from_filename(os.path.join(local_path, file))

def load_faiss_from_gcs(embeddings, bucket_name, gcs_prefix="faiss_index"):
    # Downloads files back from GCS to local disk before instantiating FAISS
    # ...
    return FAISS.load_local(local_path, embeddings, allow_dangerous_deserialization=True)

Our setup_faiss_gcs function tries to load from GCS first; if the index isn't there, it creates it locally from the data/ folder and then pushes it to the bucket. We have an identical setup for AWS S3 using boto3.

4. The Brains: Orchestrating with LangChain (LCEL)

Now we need to glue together our embedding model, our vector store, our prompt, and our Gemini LLM. LangChain makes this elegant using the LangChain Expression Language (LCEL).

Let's look at rag.py:

import os
from dotenv import load_dotenv
from langchain_google_genai import GoogleGenerativeAIEmbeddings, ChatGoogleGenerativeAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser
from vector_stores import setup_faiss, setup_pinecone, setup_faiss_gcs, setup_faiss_s3

load_dotenv()
google_api_key = os.getenv("GOOGLE_API_KEY")
data_dir = os.getenv("DATA_DIR", "data/")
vector_store_type = os.getenv("VECTOR_STORE", "faiss").lower()
use_gcs = os.getenv("USE_GCS_STORAGE", "false").lower() == "true"
use_s3 = os.getenv("USE_S3_STORAGE", "false").lower() == "true"

# 1. Initialize the Generator LLM (Gemini 2.5 Flash)
llm = ChatGoogleGenerativeAI(model="gemini-2.5-flash", google_api_key=google_api_key)

def setup_rag_system():
    # 2. Setup Embeddings
    embeddings = GoogleGenerativeAIEmbeddings(model="gemini-embedding-001", google_api_key=google_api_key)

    # 3. Dynamically route to the right vector database
    if vector_store_type == "pinecone":
        vector_store = setup_pinecone(embeddings, data_dir)
    elif use_gcs:
        vector_store = setup_faiss_gcs(embeddings, data_dir)
    elif use_s3:
        vector_store = setup_faiss_s3(embeddings, data_dir)
    else:
        vector_store = setup_faiss(embeddings, data_dir)

    # 4. Create the Retriever (Fetch Top 5 most relevant chunks)
    return vector_store.as_retriever(search_type="similarity", search_kwargs={"k": 5})

def get_qa_chain():
    retriever = setup_rag_system()

    # 5. Construct the System Prompt
    prompt = ChatPromptTemplate.from_template(
        "Answer the question based strictly on the following context:\n\n{context}\n\nQuestion: {question}"
    )

    def format_docs(docs):
        return "\n\n".join(doc.page_content for doc in docs)

    # 6. Build the LCEL Pipeline
    chain = (
        {"context": retriever | format_docs, "question": RunnablePassthrough()}
        | prompt
        | llm
        | StrOutputParser()
    )
    return chain

# Create a singleton instance to use across API calls
qa_chain_instance = get_qa_chain()

async def get_rag_response(query: str):
    return qa_chain_instance.invoke(query)

Deep Dive into LCEL: How data relates

The most beautiful part of this script is Langchain's piping syntax:

{"context": retriever | format_docs, ...} | prompt | llm | StrOutputParser()

Here is exactly what happens when qa_chain_instance.invoke("What is our refund policy?") is called:

Routing: The query "What is our refund policy?" is passed to the dictionary.
Retrieval: For the context key, the query goes into the retriever. The retriever converts the text to a vector using gemini-embedding-001, searches Pinecone/FAISS, and returns the top 5 Document objects.
Formatting: Those 5 documents are piped (|) into format_docs, which extracts the raw text and concatenates it with double newlines.
Prompting: The filled dictionary (context + question) is piped into the ChatPromptTemplate, injecting the values into the string.
Generation: The massive prompt is sent to gemini-2.5-flash.
Parsing: Because ChatModels return metadata-heavy AI Message objects, the StrOutputParser extracts just the raw markdown string.

All of this happens synchronously or asynchronously, abstracted cleanly into one chain.

5. Exposing the Pipeline via FastAPI

To facilitate the deployment of this pipeline for operational use, we expose it over HTTP using FastAPI. By leveraging FastAPI's Pydantic integrations, we effortlessly add strict input validation and automatic OpenAPI (Swagger) documentation.

In endpoints.py, we define the /chat route:

from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from rag import get_rag_response

router = APIRouter()

# Define our expected JSON Request Body
class Payload(BaseModel):
    chat_input: str

@router.post("/chat")
async def query_rag_system(request: Payload):
    try:
        # Pass the input securely to our RAG service
        response = await get_rag_response(request.chat_input)
        return {"input": request.chat_input, "response": response}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

And in main.py, we strap it all together:

from fastapi import FastAPI
from endpoints import router

app = FastAPI(title="RAG Application with Langchain and Gemini")

# Register our route controllers
app.include_router(router)

6. Running, Testing, and Deployment Options

To fire up your robust new RAG server, spin up Uvicorn in your terminal:

uvicorn main:app --reload

By default, the server spins up on http://127.0.0.1:8000. Thanks to FastAPI, you instantly have an interactive UI available.

Visit http://127.0.0.1:8000/docs.
Open the POST /chat endpoint.
Pass a payload into the chat_input.

{
  "chat_input": "What data formatting does our application support?"
}

Behind the scenes, the system will embed your query, search your data sources, generate a context-aware prompt, consult Google Gemini, and seamlessly stream back a laser-accurate answer over HTTP.

Deploying to Production

Because the entire app is stateless (state is managed by Pinecone/S3/GCS), you can quickly containerize this application with Docker.

Example Docker command flow:

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

Deploy the image to Google Cloud Run, AWS App Runner, or Railway, provide your .env variables, and you have an instantaneously scalable AI backend.

Conclusion & Next Steps

Building an intelligent, context-aware chatbot doesn't require a messy monolith of code. By cleanly separating our concerns—FastAPI for networking, Langchain for orchestration, Pinecone/FAISS for storage, and Google Gemini for raw intelligence—we've crafted a highly scalable codebase.

Where to go from here?

Conversation Memory: Try wrapping our chain in RunnableWithMessageHistory to sustain complex, multi-turn chat interactions.
Metadata Filtering: If you serve multiple users, update the Pinecone/FAISS retriever to apply metadata filtering based on user_id, ensuring strict data isolation per user.
Agentic Workflows: Replace our linear chain with langgraph to allow the LLM to decide whether it needs to search the vector database, search the web, or use an API tool.

Happy Coding! Let me know in the comments how you plan to use RAG in your own projects, and if you have any questions setting up your vector indexes!

Secure PIN Processing: A Deep Dive into ISO 9564-1 PIN Blocks, RSA, and AES Encryption

instanceofGod — Fri, 13 Feb 2026 21:55:36 +0000

In the modern payments landscape, security isn’t just a feature — it’s the absolute foundation. Every time you type your PIN into an ATM, a point-of-sale (POS) terminal, or a payment app, you are trusting a complex, global system with your financial identity. That simple 4-to-12 digit number initiates a sophisticated cryptographic handshake, orchestrated in milliseconds, to ensure your PIN remains a secret (never travels in “clear text”) from the moment it leaves your fingertips to the moment it’s verified by your bank.

In this article, we’ll break down the mechanics of the ISO 9564–1 Format 0 (ISO-0) PIN block — the industry-standard recipe for protecting a PIN.

More importantly, we’ll build a robust, full-stack architecture that uses a powerful combination of RSA-OAEP and AES (Advanced Encryption Standard) to keep this sensitive data secure all the way from the browser to the backend.

Note on Security Standards: While many legacy systems still use Triple DES (3DES), it’s important to note that NIST deprecated 3DES in 2023 due to vulnerabilities (e.g., CVE-2016–2183) and its small 64-bit block size. Modern implementations should use AES-256 for symmetric encryption, which is what we’ll implement here.

Part 1: The Anatomy of an ISO-0 PIN Block

The ISO-0 format (also known as ANSI X9.8) is the classic, widely-adopted industry standard for protecting PINs. It works by combining the user’s PIN with their Card Number (PAN) using a bitwise XOR operation. Its genius lies in its simplicity: it combines the secret (PIN) with the known (PAN) to ensure the final encrypted block is unique to that specific card and transaction.

1. Constructing the PIN Field.

We start by creating a fixed-length, 8-byte (16 hex character) block dedicated to the PIN.

First Nibble (4 bits): The control digit. For Format 0, this is always 0. Second Nibble: The length of the PIN. A PIN 1234 has a length of 4, so we use 4.
The PIN: The actual PIN digits, stored as hexadecimal characters. 1234 becomes 12 34.
Padding: The remaining bytes are filled with the hexadecimal value F (1111 binary) to reach the full 8-byte length.
Example: PIN 1234 becomes the 16-character hex string: 0 4 12 34 FF FF FF FF -> 041234FFFFFFFFFF.

2. Constructing the PAN Field.

If we encrypted the PIN field directly, the same PIN for two different cards would produce the same encrypted block — a massive security vulnerability. To prevent this, we “salt” the PIN with the cardholder’s PAN. The standard method uses the last 12 digits of the card number, excluding the final check digit (the Luhn digit). This 12-digit number is then left-padded with zeros to create another 8-byte (16 hex character) block.

Example: PAN 4012345678901234 (last digit 4 is the check digit). We take 012345678901 and pad it: 0000123456789012.

3. The XOR Operation: The Digital Blender.

The final Clear PIN Block is produced by XORing the PIN Field and the PAN Field together. XOR is a bitwise operation that compares two bits. If the bits are the same, the result is 0; if they are different, the result is 1.

PIN Field:     04 12 34 FF FF FF FF FF
PAN Field:     00 00 12 34 56 78 90 12
XOR Operation: ---------------------------------
Clear PIN Block: 04 12 26 CB A9 87 6F ED

This resulting 8-byte block is now ready for its final layer of protection: encryption. The beauty of XOR is that it’s mathematically reversible. When the bank receives the encrypted block, they decrypt it and XOR the result with the same PAN digits to retrieve the original PIN.

Part 2: The Architecture of Trust — Never Expose the Master Key

The formatting is only half the battle. The true test of a secure system is how it manages its cryptographic keys.

A common mistake in payment app designs is placing the symmetric encryption key — the Zone PIN Key (ZPK) — directly on the frontend. This is a catastrophic security risk. If a malicious actor can inspect your JavaScript code, decompile your mobile app, or intercept network traffic, they can steal the key and decrypt any user’s PIN.

The solution to this is a Hybrid Encryption Architecture. This approach leverages the strengths of two different types of cryptography to create a robust, defense-in-depth security model.

The Cryptographic Handshake Workflow
Here is how a secure system handles a transaction:

1. Key Exchange (The Setup): When the client application (e.g., a web browser) starts, its first action is to request an RSA Public Key from the server. This key is designed for encryption only.
2. Frontend Encryption (The Seal): The user enters their PIN and PAN. The browser, using the Web Crypto API, performs a critical operation. It encrypts the raw JSON data containing the {pin, pan} using the server's RSA Public Key and the secure RSA-OAEP padding scheme. This is the seal. Because RSA is asymmetric, the frontend can encrypt data with the public key, but it is mathematically infeasible for it—or any attacker—to decrypt it. The private key is the only thing that can break the seal.
3. Secure Transmission (The Courier): The resulting encrypted binary blob is sent over HTTPS to the Node.js backend. Even if an attacker were to intercept this message, they would only see ciphertext that is useless without the private key.
4. Backend Processing (The Vault): Inside the secure environment of the server, the private key is used to decrypt the payload. Only now, with the raw pin and pan in memory, does the server perform the sensitive operations:
It constructs the ISO-0 clear PIN block using the XOR logic.
It then encrypts this block using the Zone PIN Key (ZPK) with the AES-256 algorithm.
This layered approach ensures that the most critical secret — the ZPK — remains safely inside the server’s “trusted zone” and is never exposed to the user’s device — a potentially hostile environment.

Part 3: Under the Hood — Implementation Highlights (AES Edition)

Let’s look at the code that brings this architecture to life, highlighting the critical security considerations at each layer — now updated to use AES.

Frontend: Sealing the Secret with Web Crypto (JavaScript)

Modern browsers provide the Web Crypto API, a powerful, native layer for performing secure cryptographic operations without relying on potentially vulnerable external libraries. This part remains unchanged, as it handles the RSA encryption layer.

// The data to protect
const dataToEncrypt = new TextEncoder().encode(
    JSON.stringify({ pin: userPIN, pan: userPAN })
);
const encryptedData = await window.crypto.subtle.encrypt(
    {
        name: "RSA-OAEP"
    },
    importedPublicKey, 
    dataToEncrypt
);

Backend: The Vault Logic with AES (Node.js)

On the server, we reconstruct the PIN block. A crucial detail here is the use of setAutoPadding(false). PIN block encryption is a very specific operation. The input must be exactly 8 bytes, and the output must be exactly 8 bytes. Standard block cipher padding schemes like PKCS#7 would alter the length, breaking the strict format required by the ISO standard.

With AES, we must also specify the correct mode. For PIN block encryption, AES in ECB mode is typically used to maintain compatibility with legacy systems, though other modes can be used with careful IV management.

const crypto = require('crypto');

/**
 * Encrypts an 8-byte clear PIN block using AES-256
 * @param {Buffer} clearBlockBuffer - The 8-byte clear PIN block
 * @param {string} zpkHex - The Zone PIN Key in hex (32 bytes for AES-256)
 * @returns {Buffer} - The 8-byte encrypted PIN block
 */
function encryptPinBlockWithAES(clearBlockBuffer, zpkHex) {
    // Ensure we're working with a 32-byte key for AES-256
    const key = Buffer.from(zpkHex, 'hex');
    if (key.length !== 32) {
        throw new Error('AES-256 requires a 32-byte (64 hex character) key');
    }

    // Create AES-256 cipher in ECB mode
    // Note: ECB mode is used here for PIN block compatibility
    const cipher = crypto.createCipheriv('aes-256-ecb', key, null);

    // CRITICAL: Disable automatic padding. PIN blocks are exactly 8 bytes.
    cipher.setAutoPadding(false); 

    // Encrypt the 8-byte clear block to produce an 8-byte encrypted block
    return Buffer.concat([cipher.update(clearBlockBuffer), cipher.final()]);
}

// Example usage:
// const encryptedPinBlock = encryptPinBlockWithAES(clearBlockBuffer, zpkHex);

Part 4: Beyond the PIN Block — The Wider World of Payment Security

While the ISO-0 PIN block is fundamental for “Card Present” transactions (like at an ATM or a chip-enabled terminal), it’s just one piece of a much larger security puzzle.

- CVV/CVC (Card Verification Value): This is the static 3 or 4-digit code printed on your card. It’s not a random number. It’s cryptographically calculated by the issuing bank using a secret Card Verification Key (CVK) and specific data from your card (like PAN and expiry date). It serves as proof that you are in physical possession of the card during a “Card Not Present” (online) transaction.
- 3D Secure (3DS): This is the protocol behind challenges like “Verified by Visa” or “Mastercard SecureCode.” It’s an active, real-time authentication layer that shifts liability for fraud from the merchant to the card issuer. By redirecting the user to their bank’s authentication page (or sending an OTP), it provides an extra layer of proof that the legitimate cardholder is initiating the transaction.
- Tokenization: Modern payment systems often replace sensitive PAN data with tokens. This further reduces risk by ensuring that even if data is intercepted, the actual card number isn’t exposed.

Conclusion: The Future of PIN Security
Building secure payment systems is not about using a single, unbreakable lock. It’s about understanding how to combine different cryptographic primitives — like XOR for formatting, asymmetric RSA for secure key exchange, and symmetric AES for robust data encryption — into a cohesive, layered defense.

By strictly separating the “encryption” layer on the client from the “translation and final sealing” layer on the server, we create a system where a user’s most sensitive data — their PIN — is protected from the moment it is typed until it reaches the secure confines of the banking switch. It’s a fascinating and critical field where mathematical principles are the bedrock of global financial trust.

For a complete, runnable implementation of this architecture with AES encryption, check out thi repository. The code provides a practical, end-to-end example of the concepts discussed in this article.

Cloud Resume API Challenge with GCP

instanceofGod — Sun, 21 Jul 2024 17:26:52 +0000

The job search can be a competitive landscape, and having a standout resume is crucial. But what if your resume could be even more dynamic and accessible? Enter the Cloud Resume API Challenge, a fantastic opportunity to showcase your technical skills and create a modern, API-driven resume.

The primary goal of the challenge is to construct a serverless API that can serve resume data in JSON format. In order to achieve the goal of the project i utilised google cloud resources, particularly Google Cloud Functions and Cloud Firestore, along with GitHub Actions for continuous delivery and continuous deployment (CI/CD).

Not only did I gain valuable insights into the world of resume APIs, but I also honed my skills in several key areas which I decided to share with you here:

1. Cloud Functions

I delved into the world of cloud functions, a serverless execution environment that allows you to run code without managing servers. This was a great way to create a lightweight and scalable backend for my API.

2. Cloud Firestore

Cloud Firestore, a NoSQL database, served as the perfect storage solution for my resume data. Its flexible schema and ease of use made it a breeze to manage my information.

3. Google APIs
Working with Google APIs provided a powerful way to interact with my Cloud Firestore data and expose it through the API.
Building the Cloud Function with Node.js

For the challenge, I opted to use Node.js with JavaScript as my development environment. Here's a sample code snippet showcasing a basic Cloud Function that retrieves resume data from Firestore:


const { Firestore } = require('@google-cloud/firestore')
const functions = require('@google-cloud/functions-framework')
const dotenv = require('dotenv')
dotenv.config()

functions.http('resume', (req, res) => {
  const firestore = new Firestore({
    projectId: process.env.PROJECT_ID
  })
  const document = firestore.collection('resume').doc('kZXU70VsRQ2OuDBAjE')
  document
    .get()
    .then((data) => res.send(data.data()))
    .catch((err) => {
      console.log(err)
    })
})

This function retrieves data from a collection named "resume" in Firestore.

4. CI/CD with GitHub Actions
I implemented a CI/CD pipeline using GitHub Actions to ensured automated deployment to cloud function whenever I pushed changes to my code.

Beyond the Basics

The Cloud Resume API Challenge offers a springboard for further exploration. Here are some ways to take your project to the next level:

Authentication and Authorization: Implement mechanisms to control access to your resume data using authentication and authorization techniques.
Versioning: Integrate versioning to track changes to your resume data over time.
Additional Data Points: Include additional data points like portfolio links or project details to make your resume more interactive.
A Rewarding Journey

Participating in the Cloud Resume API Challenge was a rewarding journey. It allowed me to showcase my technical skills, build a modern resume format, and gain valuable experience with cloud technologies. If you're looking for a way to stand out from the crowd and level up your technical expertise, I highly recommend giving this challenge a try!

4 Ways to backup mySql database to a csv file

instanceofGod — Thu, 13 Jun 2024 10:44:01 +0000

There are several methods to create a CSV backup of your MySQL database. Some third-party database management tools offer additional features for backing up to CSV.
If you prefer command-line control, the mysqldump utility is powerful and flexible. If you are familiar with the python programing language, there are packages to help you write python scripts to backup your database. The method you choose depends on your comfort level and technical expertise

In this article, i share 4 different ways to backup your MySQL database to a CSV file.

Using `mysqldump` and `mysql` Command-Line Tools

Export the Database to SQL File

Use mysqldump to create a dump of your database. This step is optional but useful for backing up the entire database structure and data.

   mysqldump -u username -p database_name > database_backup.sql

Replace username with your MySQL username, database_name with the name of the database, and database_backup.sql with the name you want for your backup file.

Export Table to CSV File

You can export a specific table to a CSV file using the SELECT statement with INTO OUTFILE in MySQL. Here’s how you can do it:

   SELECT * FROM table_name
   INTO OUTFILE '/path/to/your/file.csv'
   FIELDS TERMINATED BY ','
   ENCLOSED BY '"'
   LINES TERMINATED BY '\n';

Replace table_name with the name of the table you want to export, and /path/to/your/file.csv with the full path where you want to save the CSV file.

Ensure the MySQL server has the appropriate permissions to write to the specified path.
The FIELDS TERMINATED BY ',' specifies that fields are separated by commas.
The ENCLOSED BY '"' ensures that fields are enclosed in double quotes.
The LINES TERMINATED BY '\n' specifies the end of a row.

Run the SQL Command

You can execute the SQL command directly through the MySQL shell or using a script. Here’s how to do it from the MySQL shell:

   mysql -u username -p database_name -e "SELECT * FROM table_name INTO OUTFILE '/path/to/your/file.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\n';"

Replace the placeholders with your actual database details.

Example in MySQL Shell

   mysql -u username -p

Select the database:

   USE database_name;

Export the table:

   SELECT * FROM table_name
   INTO OUTFILE '/path/to/your/file.csv'
   FIELDS TERMINATED BY ','
   ENCLOSED BY '"'
   LINES TERMINATED BY '\n';

Permissions Note

Make sure the MySQL user has the FILE privilege to write files to the server.
The directory specified in the INTO OUTFILE path must be writable by the MySQL server process.

Using a Bash Script

For automation, you can create a script (e.g., db_backup.sh) to export the table:

#!/bin/bash

DB_USER="username"
DB_PASS="password"
DB_NAME="database_name"
TABLE_NAME="table_name"
OUTPUT_FILE="/path/to/your/file.csv"

mysql -u $DB_USER -p$DB_PASS -e "SELECT * FROM $TABLE_NAME INTO OUTFILE '$OUTPUT_FILE' FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\n';"

Make the script executable:

chmod +x db_backup.sh

The run the script:

./db_backup.sh

Using Python and the pandas library

Install the necessary packages: You need to install pandas and mysql-connector-python (or PyMySQL) to connect to the MySQL database and manipulate the data.

   pip install pandas mysql-connector-python

Create a Python script: sample script to export a MySQL table to a CSV file using pandas.

   import pandas as pd
   import mysql.connector

   # Database configuration
   db_config = {
    'user': 'username',
    'password': 'password',
    'host': 'localhost',
    'database': 'database_name'
   }

   # SQL query to select data from the table
   query = "SELECT * FROM table_name"

   # Connect to the MySQL database
   connection = mysql.connector.connect(**db_config)

   # Read data from the database into a pandas DataFrame
   df = pd.read_sql(query, connection)

   # Close the connection
   connection.close()

   # Export the DataFrame to a CSV file
   output_file = '/path/to/your/file.csv'
   df.to_csv(output_file, index=False)

   print(f"Data has been exported to {output_file}")

Code Explanation

Database Configuration:
- Replace 'username', 'password', 'localhost', and 'database_name' with your actual database credentials.
SQL Query:
- The query variable holds the SQL query to select all data from the specified table. Replace 'table_name' with the name of the table you want to export.
Connecting to the Database:
- Establish a connection to the MySQL database using mysql.connector.connect() with the provided configuration.
Reading Data into a DataFrame:
- Use pd.read_sql(query, connection) to execute the query and load the data into a pandas DataFrame.
Closing the Connection:
- Close the database connection using connection.close().
Exporting to CSV:
- Use df.to_csv(output_file, index=False) to export the DataFrame to a CSV file. Replace '/path/to/your/file.csv' with the desired file path for the CSV file.
Confirmation:
- Print a confirmation message indicating the location of the exported file.

Running the Script

Save the script to a file, for example export_to_csv.py, and run it:

python export_to_csv.py

This script will connect to the specified MySQL database, execute the query to retrieve data from the specified table, and export the data to a CSV file.

Using csvkit to Export a MySQL Table to a CSV File

csvkit is a suite of command-line tools for converting to and working with CSV files.
To export a MySQL table to a CSV file using csvkit, you can use the csvsql command.
Certainly! csvkit is a suite of command-line tools for converting to and working with CSV files. To export a MySQL table to a CSV file using csvkit, you can use the csvsql command.

Install csvkit and mysql-connector-python: Make sure you have csvkit and mysql-connector-python installed. You can install it using pip:

   pip install csvkit mysql-connector-python

import subprocess
import mysql.connector

# Database configuration
db_config = {
    'user': 'username',
    'password': 'password',
    'host': 'localhost',
    'database': 'database_name'
}

# Table name and output file
table_name = 'table_name'
output_file = '/path/to/your/file.csv'

# Connect to the MySQL database to get the connection details
connection = mysql.connector.connect(**db_config)
cursor = connection.cursor()

# Construct the csvsql command
csvsql_command = [
    'csvsql',
    '--db',
    f'mysql+mysqlconnector://{db_config["user"]}:{db_config["password"]}@{db_config["host"]}/{db_config["database"]}',
    '--query',
    f'SELECT * FROM {table_name}',
    '--output',
    output_file
]

# Execute the csvsql command
subprocess.run(csvsql_command, check=True)

# Close the connection
cursor.close()
connection.close()

print(f"Data has been exported to {output_file}")

Database Configuration:
- Replace 'username', 'password', 'localhost', and 'database_name' with your actual database credentials.
Table Name and Output File:
- Replace 'table_name' with the name of the table you want to export.
- Replace '/path/to/your/file.csv' with the desired file path for the CSV file.
Connecting to the Database:
- Establish a connection to the MySQL database using mysql.connector.connect() with the provided configuration.
Constructing the csvsql Command:
- Use csvsql with the --db option to specify the database connection string.
- The --query option specifies the SQL query to run.
- The --output option specifies the output file for the CSV data.
Executing the Command:
- Use subprocess.run() to execute the constructed csvsql command.
Closing the Connection:
- Close the database connection and cursor.
Confirmation:
- Print a confirmation message indicating the location of the exported file.

Save the script to a file, for example export_to_csv_with_csvkit.py, and run it:

python export_to_csv_with_csvkit.py

This script will connect to the specified MySQL database, execute the query to retrieve data from the specified table, and export the data to a CSV file using csvkit.

Note: The subprocess module in Python is used to spawn new processes, connect to their input/output/error pipes, and obtain their return codes

It allows you to run external commands and interact with them programmatically. This is especially useful for automating command-line tasks and integrating external tools into your Python scripts.

Key Functions in subprocess
subprocess.run(): This function runs a command, waits for it to complete, and then returns a CompletedProcess instance.
subprocess.Popen(): This is a more powerful and flexible function for spawning new processes. It allows more complex interactions with the process.

Example Usage
Here’s a simple example demonstrating the use of subprocess.run():

import subprocess

# Define the command to run
command = ['echo', 'Hello, World!']

# Run the command
result = subprocess.run(command, capture_output=True, text=True)

# Print the command's output
print(result.stdout)

Explanation of Parameters:

command: A list where the first element is the command to run, and the subsequent elements are the arguments to the command.

capture_output: If set to True, it captures the standard output and standard error.

text: If set to True, the output is returned as a string instead of bytes.

NOTE: the csvkit command an also be run as a one line command using if you have csvkitn installed. You can install csvkit with pip install csvkit:

csvsql --db mysql://[username]:[password]@localhost/[database_name] --query "SELECT * FROM [table_name]" > [output_file.csv]

JavaScript Array methods

instanceofGod — Sat, 01 Jun 2024 14:10:54 +0000

JavaScript provides a rich set of array methods to manipulate and work with arrays. Here is a comprehensive list of these methods along with sample usage:

Mutator Methods
These methods modify the array they are called on.

push()

Adds one or more elements to the end of an array and returns the new length of the array.

sample:

let arr = [1, 2, 3];
arr.push(4);  // arr is now [1, 2, 3, 4]

pop()

Removes the last element from an array and returns that element.

sample:

let arr = [1, 2, 3];
let lastElement = arr.pop();  // arr is now [1, 2]; lastElement is 3

shift()

Removes the first element from an array and returns that element.

sample:

let arr = [1, 2, 3];
let firstElement = arr.shift();  // arr is now [2, 3]; firstElement is 1

unshift()

Adds one or more elements to the beginning of an array and returns the new length of the array.

sample:

let arr = [1, 2, 3];
arr.unshift(0);  // arr is now [0, 1, 2, 3]

splice()

Adds and/or removes elements from an array.

sample:


let arr = [1, 2, 3, 4];
arr.splice(1, 2, 'a', 'b');  // arr is now [1, 'a', 'b', 4]

sort()

Sorts the elements of an array in place and returns the sorted array.

sample:


let arr = [3, 1, 4, 1, 5, 9];
arr.sort();  // arr is now [1, 1, 3, 4, 5, 9]

reverse()

Reverses the order of the elements in an array in place.

sample:

let arr = [1, 2, 3];
arr.reverse();  // arr is now [3, 2, 1]

Accessor Methods

These methods do not modify the array and typically return a new array or a single value.

concat()

Merges two or more arrays.

sample:

let arr1 = [1, 2];
let arr2 = [3, 4];
let newArr = arr1.concat(arr2);  // newArr is [1, 2, 3, 4]

join()

Joins all elements of an array into a string.

sample:


let arr = [1, 2, 3];
let str = arr.join('-');  // str is '1-2-3'

slice()

Returns a shallow copy of a portion of an array into a new array.

sample:

let arr = [1, 2, 3, 4, 5];
let slicedArr = arr.slice(1, 3);  // slicedArr is [2, 3]

indexOf()

Returns the first index at which a given element can be found in the array.

sample:

let arr = [1, 2, 3];
let index = arr.indexOf(2);  // index is 1

lastIndexOf()

Returns the last index at which a given element can be found in the array.

sample:

let arr = [1, 2, 3, 2];
let index = arr.lastIndexOf(2);  // index is 3

includes()

Determines whether an array includes a certain value.

sample:

let arr = [1, 2, 3];
let hasTwo = arr.includes(2);  // hasTwo is true

toString()

Returns a string representing the array.

sample:

let arr = [1, 2, 3];
let str = arr.toString();  // str is '1,2,3'

toLocaleString()

Returns a localized string representing the array.

sample:


let arr = [1, 2, 3];
let str = arr.toLocaleString();  // str might vary based on locale

Iteration Methods

These methods iterate over the array, typically applying a function to each element.

forEach()

Executes a provided function once for each array element.

sample:

let arr = [1, 2, 3];
arr.forEach(element => console.log(element));  // logs 1, 2, 3

map()

Creates a new array with the results of calling a provided function on every element.

sample:


let arr = [1, 2, 3];
let doubled = arr.map(x => x * 2);  // doubled is [2, 4, 6]

filter()

Creates a new array with all elements that pass the test implemented by the provided function.

sample:

let arr = [1, 2, 3, 4];
let evens = arr.filter(x => x % 2 === 0);  // evens is [2, 4]

reduce()

Applies a function against an accumulator and each element to reduce it to a single value.

sample:

let arr = [1, 2, 3, 4];
let sum = arr.reduce((acc, curr) => acc + curr, 0);  // sum is 10

reduceRight()

Applies a function against an accumulator and each element (from right to left) to reduce it to a single value.

sample:

let arr = [1, 2, 3, 4];
let sum = arr.reduceRight((acc, curr) => acc + curr, 0);  // sum is 10

some()

Tests whether at least one element passes the test implemented by the provided function.

sample:

let arr = [1, 2, 3];
let hasEven = arr.some(x => x % 2 === 0);  // hasEven is true

every()

Tests whether all elements pass the test implemented by the provided function.

sample:

let arr = [1, 2, 3];
let allPositive = arr.every(x => x > 0);  // allPositive is true

find()

Returns the first element that satisfies the provided testing function.

sample:


let arr = [1, 2, 3];
let found = arr.find(x => x > 1);  // found is 2

findIndex()

Returns the index of the first element that satisfies the provided testing function.

sample:


let arr = [1, 2, 3];
let index = arr.findIndex(x => x > 1);  // index is 1

flat()

Creates a new array with all sub-array elements concatenated into it recursively up to the specified depth.

sample:

let arr = [1, [2, [3, 4]]];
let flatArr = arr.flat(2);  // flatArr is [1, 2, 3, 4]

flatMap()

First maps each element using a mapping function, then flattens the result into a new array.

sample:


let arr = [1, 2, 3];
let flatMapped = arr.flatMap(x => [x, x * 2]);  // flatMapped is [1, 2, 2, 4, 3, 6]

entries()

Returns a new Array Iterator object that contains the key/value pairs for each index in the array.

sample:

let arr = ['a', 'b', 'c'];
let iterator = arr.entries();
for (let [index, value] of iterator) {
  console.log(index, value);  // logs 0 'a', 1 'b', 2 'c'
}

keys()

Returns a new Array Iterator that contains the keys for each index in the array.

sample:

let arr = ['a', 'b', 'c'];
let iterator = arr.keys();
for (let key of iterator) {
  console.log(key);  // logs 0, 1, 2
}

values()

Returns a new Array Iterator object that contains the values for each index in

Securely Connecting to PostgreSQL on a Virtual Machine from Anywhere

instanceofGod — Mon, 22 Apr 2024 11:21:47 +0000

Prerequisites:

A virtual machine set up on a cloud provider (AWS, Google Cloud, Azure) or a local virtualization platform (VirtualBox).
Basic understanding of command line and Linux concepts.

Deploying PostgreSQL on a Virtual Machine (VM) and connecting to it from anywhere involves several steps. This guide walks you through deploying PostgreSQL on a Virtual Machine (VM) and connecting to it remotely with enhanced security practices.

Once your VM is running, you'll need to install PostgreSQL. This process varies depending on the operating system of your VM, but it generally involves downloading and installing the PostgreSQL package.

1. Install and Configure PostgreSQL

1.1. Update and Install:

sudo apt update
sudo apt install postgresql postgresql-contrib -y

1.2. Verify PostgreSQL user:

PostgreSQL creates a user named postgres. Verify its existence:

sudo cat /etc/passwd | grep -i postgres
postgres:x:113:120:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash

To connect to Postgres, switch to the postgres user and run psql:

sudo -i -u postgres

psql
postgres=#

1.3. Create a Secure User (Optional):

For better security, create a dedicated user for remote access instead of using the default postgres user. We'll create a user named ubuntu and grant them necessary privileges based on your specific needs. Avoid granting superuser access (equivalent to root in Linux) unless absolutely essential.

To create a Postgres user run the following command which will give an interactive prompt for configuring the new user. For the sake of simplicity, the ubuntu user will be a superuser, which is the equivalent of being a root user on linux. The super user will have the ability to create/delete/modify databases and users.

createuser --interactive
Enter name of role to add: ubuntu
Shall the new role be a superuser? (y/n) y

# Set password for myuser (replace 'password' with a strong password)
ALTER USER ubuntu PASSWORD 'password';

# Grant appropriate privileges (adjust as needed)
GRANT CONNECT ON DATABASE postgres TO ubuntu;
GRANT CREATE DATABASE TO ubuntu;

\q
exit

psql
Type "help" for help.

postgres=# \du
                                   List of roles
 Role name |                         Attributes                         | Member of
-----------+------------------------------------------------------------+-----------
 postgres  | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
 ubuntu    | Superuser, Create role, Create DB                          | {}

postgres=#

Exit out of the psql by running \q and also exit out of the postgres user by running exit on the command line

Let's try to run psql as the ubuntu user now. An error similar to the one below should be observed

$ psql
psql: error: could not connect to server: FATAL:  database "ubuntu" does not exist

The reason for this is that Postgres by default tries to connect to a database that is the same name as the user. Since the user is ubuntu it tries to connect to a database called ubuntu as well which does not exist. We can go in and create a database called ubuntu so that it will automatically connect, however I find this unnecessary. Instead we can pass in the -d flag and connect to a database that we know exists like the postgres

psql -d postgres

With the above steps we have successfully installed and configured Postgres on the server.

Configure PostgreSQL to accept connections from any location.

By default PostgreSQL is configured to be bound to “localhost”, on port 5432 . It means any attempt to connect to the postgresql server from outside the machine will be refused.

netstat -nlt
Proto Recv-Q Send-Q Local Address           Foreign Address         State

tcp        0      0 127.0.0.1:5432          0.0.0.0:*               LISTEN

To allow your postgres server to accept connection from any location or a specific location (IP address), you will need to edit two files:

i.pg_hba.conf

ii.postgresql.conf

You can use the following command to easily locate the files on your server

sudo -u postgres psql -c "SHOW hba_file" -c "SHOW config_file"

`pg_hba.conf`

From 20.1. The pg_hba.conf File: "_Client authentication is controlled by a configuration file, which traditionally is named pg_hba.conf and is stored in the database cluster's data directory. (HBA stands for host-based authentication.) _"

host    all   all   0.0.0.0/0   trust
host    all   all   ::/0        trust

`postgresql.conf`

The configuration file comes with helpful hints to get this working

To allow your Postgres server to accept connection from any location or a specific location (IP address), change the listen_addresses from ‘localhost’ to the IP address of the server you want to connect from. To allow for connection from anywhere( not advised), change ‘localhost” to “*”.

Remember, exposing your database to the internet comes with security risks. Always ensure you have strong, unique passwords and consider setting up additional security measures.

listen_addresses = 'localhost'          # what IP address(es) to listen on;
                                        # comma-separated list of addresses;
                                        # defaults to 'localhost'; use '*' for all
                                        # (change requires restart)

For a quick and dirty solution just change it to

listen_addresses = '*'

Restart postgres

After the configuration, you will need to restart the postgres server. Once PostgreSQL is restarted, it will start listening on all IP addresses.

To restart PostgreSQL:

$ sudo systemctl restart postgresql@11-main # change 11 to the version of your postgres

# or

$ pg_ctl restart

Open Postgres port on server firewall rule

Remember to open the port that PostgreSQL uses (default is 5432) on your VM's firewall. This step will depend on your VM's operating system and firewall settings.

Finally, you can connect to your PostgreSQL database from anywhere using a PostgreSQL client. You'll need your VM's public IP address, the database name, and the login credentials.

Secure Remote Connection with SSH Tunneling

Another option to securely connect to postgres server is by using SSH Tunneling. To enable secure remote access, we'll leverage SSH Tunneling. This creates an encrypted tunnel between your local machine and the VM, allowing you to connect to PostgreSQL through this secure channel.

4.1. Establish an SSH Tunnel:

Use your preferred SSH client to create a tunnel. Here's an example command:

Bash

ssh -L 5433:localhost:5432 <username>@<VM_IP_Address>

Replace the placeholders:

<username> with your VM username.
<VM_IP_Address> with the public IP address of your VM.

This command creates a local port (5433) on your machine that forwards traffic to the PostgreSQL port (5432) on the VM.

4.2 Connect to PostgreSQL Remotely:

Use a PostgreSQL client on your local machine and connect to:

localhost:5433  # Port you forwarded in the tunnel (5433)

Use the credentials of the user you created (e.g., ubuntu).

The steps above establish a secure connection using SSH Tunneling. To further enhance security, consider these additional measures:

Never modify listen_addresses in postgresql.conf to '*'. This exposes your database server to the entire internet, making it vulnerable to attacks.
Configure firewall rules on your VM to restrict access to the PostgreSQL port (default 5432) from only your specific IP address. This adds an extra layer of protection.
Implement strong password policies and enforce secure user authentication methods. Utilize the pg_hba.conf file to control which users can connect from where and how they can authenticate. Refer to PostgreSQL documentation for detailed configuration options.

5. Conclusion:

By following these steps, you can securely connect to your PostgreSQL database on a VM from anywhere or by using SSH tunneling. Remember, security should be a top priority. Implement the additional security measures mentioned above to create a more robust defense for your database server.

For further details on PostgreSQL configuration and security best practices, refer to the official PostgreSQL documentation https://www.postgresql.org/docs/.

Helpful link and references:

https://stackoverflow.com/questions/24504680/connect-to-postgres-server-on-google-compute-engine

Enable Password Authentication in AWS EC2 Instance

instanceofGod — Mon, 16 May 2022 13:56:26 +0000

Password authentication is disabled by default in aws ec2 instance. The only way to access your server is by using ssh with -i flag and followed by private key attached to the server.

However, there are scenarios when you may need to access the server with just username and password. Take note that this method of accessing your server is not encouraged, as it could open your server to attacks.

How to Enable Password Authentication in EC2

First, you have to ssh into your server with your private key to set password for the user and also make some changes to the sshd_config file.

The steps to enable password authentication are highlighted below:

ssh -i privatekey username@host_ip

Setup a password for the user using the passwd command

sudo passwd username

Open and modify the sshd_config file.

sudo vim /etc/ssh/sshd_config

sudo nano /etc/ssh/sshd_config

If you prefer to use nano as editor

Changes the PasswordAuthentication line from ‘no’ to ‘yes’

PasswordAuthentication yes

Enable root login, Change value from “prohibit-password” to “yes” (Optional)

PermitRootLogin yes

Restart the “sshd” service using the following command.

sudo service sshd restart

or 
sudo systemctl restart ssh

That’s all you have to do. Now you can login to the ec2 server using the password you set for the user, without the private key.

When you type the command below, you will be prompted to enter your password!

A beginners' Journey into Web Development.

instanceofGod — Mon, 16 Aug 2021 22:03:27 +0000

Hey, thinking of starting a career into web development but don't know how and where to begin?

One of the most difficult decision to make by beginners when venturing into web development is how to begin, and what resources are available to aid their learning.

If this is where you find yourself now, like me, when I first started learning web development ( I wish I had seen and read an article like this then), this article will help to provide a roadmap to help you to learn faster and better.

Start from the Basic.

As foundation is important in construction, so is it in software development. Starting from the basic means starting from ground up, learning the foundational concepts of HTML such as tags, semantics, structure, layout etc. Want to Start Learning HTML now, **https://www.tutorialrepublic.com/html-tutorial/** is one of the good site to start your journey into web development.

You need to understand how to arrange elements on the web page, that is, putting elements where they ought to be on the page. One good tool that than help you achieve this is FIGMA. Learn more about figma **HERE https://trydesignlab.com/figma-101-course/introduction-to-figma/**

Advance to Aesthetics

After you are comfortable with html and have a good knowledge of laying out your page, giving your code good visual appearance with CSS is the next step. CSS is used for styling your HTML pages. While HTML is like an unpainted building, CSS is the 'paint' that makes the we page appealing to the eyes.

Other Things to learn includes Version Control. Want to know more about Version Control using GIT, go to : https://www.freecodecamp.org/news/learn-the-basics-of-git-in-under-10-minutes-da548267cc91/

Choose a Programming Language

There are several programming languages that does different or same thing in different ways.

You will need to decide your goal and objective for the future before deciding which one to learn first. Of course, you can learn many languages over time, but it is better to pick one and be a master of it, than not being able to do anything productive with the knowledge of several programming languages.

Examples of Programming language are: PHP, Python, Java, JavaScript, Go, Rust, C#, C++, MySql, Kotlin, Ruby, Perl and many more. Make proper research about the language, what you can use it to achieve, whether it fulfils your future goal as a software developer.

Fortunately, the internet abounds with resources- Video, Articles, Guides or Courses to make your learning journey smooth and sound. You can checkout the following resource:

PHP - https://www.w3schools.com/php/default.asp

JavaScript - https://www.w3schools.com/js/default.asp

Python - https://www.w3schools.com/python/default.asp

Go - https://www.w3schools.com/go/index.php

ESLint Prettier & Husky Configuration for Project

instanceofGod — Thu, 18 Mar 2021 11:39:57 +0000

Setting up Prettier with ESLint, and also checking commit with husky.

You will need to install Eslint, Prettier and Pre-Commit.
The 3 packages does different things, their usefulness are highlighted below:

ESLint - For Linting, and Enforcing Style Guide.
Prettier - For Code Formatting.
Pre-Commit - For Making sure code that's being committed is followed guidelines and is formatted properly.

Installation

The packages can be installed locally in your project or globally:

Setting up ESLint

npm install --save-dev eslint

After installing eslint, you then need to set it up, that is, configure it.

npx eslint --init

Sample of the text that will be inside the eslintrc.json file

{
"extends": ["airbnb", "prettier" "eslint:recommended"],
"plugins": ["prettier"],
"rules": {
"semi": ["error", "always"],
"quotes": ["error", "double"]

"prettier/prettier": ["error"] // if you choose to use prettier rules.
},

}

Setting up Prettier

npm install --save-dev prettier eslint-config-prettier eslint-plugin-prettier

Then create .prettierrc file in your project’s root directory.

You can type echo {} .prettierrc in your terminal to create the file

touch {} .prettierrc

This will be where you configure your prettier formatting settings.

Sample of the settings you can configure inside the prettierrc.json file


{ 
"useTabs": true,
"tabSize": 2,
"semi": true,
"singleQuote": true,
"trailingComma": "es5",
"bracketSpacing": true,
"parser":"typescript"

}

Setting up Pre-commit and Pre-push

This is a pretty important step. This will ensure that before any developer in your team commits any code, the changes made by him are validated with
EsLint, and the code is properly formatted.

The fastest way to start using lint-staged is to run following command in your terminal:

npx mrm lint-staged

It will install and configure husky and lint-staged depending on code quality tools from package.json dependencies, so please make sure you install (npm install --save-dev) and configure all code quality tools like Prettier, ESlint prior that.

Another alternative to implement this; we will need to configure some packages

The First package we need is husky which will make adding these hooks very easy.
We also need a package called lint-staged that will let us check only the pages which are changed. So, only the staged files are checked and the rest of the code remains untouched.
pretty quick will check for any unformatted files and format them using Prettier.

Installation of the packages

npm install --save-dev husky lint-staged pretty-quick

After installing these packages we need to add configuration for these in our package.json file

"lint-staged": {
"*.js": "eslint --cache --fix"
},
"husky": {
"hooks": {
"pre-commit": "lint-staged && pretty-quick --staged"
}
}

As the name suggests, when you're trying to commit the changes, this will run both commands "lint-staged and pretty-quick".

The lint-staged will run eslint command on javascript files that are staged,
The pretty-quick will format the JavaScript files if they aren't using Prettier.

If any of the staged files are not properly linted or formatted this will give you a warning and will stop you from committing the
changes in the code.

DEV Community: instanceofGod

PIN Encryption & Decryption with RSA in Node.js

Table of Contents

1. Why PIN encryption matters

2. Core terms defined

3. How RSA works

4. Anatomy of a key pair

The public key

The private key (PKCS#8)

Key format summary

5. The OpenSSLUtil class, line by line

Constructor and key loading

Encrypt method — OAEP-SHA256

Decrypt method — OAEP-SHA256

Sign method

Module export as singleton

6. The full encrypt → transmit → decrypt flow

Step 1 — Client encrypts the PIN (terminal simulation)

Step 2 — Client sends the encrypted PIN

Step 3 — Server decrypts and verifies

7. Working with keys in the terminal

Generate a fresh key pair

Encrypt a PIN for testing (OAEP)

Convert the key to a JSON-safe escaped string

8. Common pitfalls & security notes

Don't log decrypted PINs

Singleton means shared state

OAEP encryption is non-deterministic — that's a feature

Encryption vs signing — knowing when to use each

9. Conclusion

Building a Production-Ready RAG Application with FastAPI, LangChain, and Google Gemini: A Deep Dive

1. Our Technology Stack

2. Project Architecture

3. The Data Layer: Multi-Backend Vector Stores

Option A: Cloud-Native with Pinecone

Option B: Local & Cloud-Backed FAISS

4. The Brains: Orchestrating with LangChain (LCEL)

Deep Dive into LCEL: How data relates

5. Exposing the Pipeline via FastAPI

6. Running, Testing, and Deployment Options

Deploying to Production

Conclusion & Next Steps

Secure PIN Processing: A Deep Dive into ISO 9564-1 PIN Blocks, RSA, and AES Encryption

Part 1: The Anatomy of an ISO-0 PIN Block

Part 2: The Architecture of Trust — Never Expose the Master Key

Part 3: Under the Hood — Implementation Highlights (AES Edition)

Part 4: Beyond the PIN Block — The Wider World of Payment Security

Cloud Resume API Challenge with GCP

4 Ways to backup mySql database to a csv file

Using mysqldump and mysql Command-Line Tools

Example in MySQL Shell

Using a Bash Script

Using Python and the pandas library

Code Explanation

Running the Script

Using csvkit to Export a MySQL Table to a CSV File

JavaScript Array methods

push()

pop()

shift()

unshift()

splice()

sort()

reverse()

Accessor Methods

concat()

join()

slice()

indexOf()

lastIndexOf()

includes()

toString()

toLocaleString()

Iteration Methods

forEach()

map()

filter()

reduce()

reduceRight()

some()

Using `mysqldump` and `mysql` Command-Line Tools

`pg_hba.conf`

`postgresql.conf`