DEV Community: Shreshth Goyal

I Stopped Planning and Just Shipped — Here’s What Happened

Shreshth Goyal — Tue, 22 Apr 2025 18:59:41 +0000

I Built FundLap in a Weekend to Fix Mutual Fund Overlap — Now It's Helping Thousands Optimize Their Portfolios

The Problem No One Was Solving

Most mutual fund tools either hide critical insights, lock them behind paywalls, or overwhelm users with complexity.

As a data-obsessed investor, I found this deeply frustrating. I wanted something simple, powerful, and free — something that showed me the real picture of my portfolio, including the overlap that kills diversification and the fees that quietly drain returns.

FundLap started as a weekend experiment. Today, it's a free public platform built for serious investors, finance-curious developers, and anyone who wants to take control of their mutual fund strategy.

Try it at 👉 fundlap.com

From Frustration to First Build

A few months ago, I was trying to compare a few large-cap funds. Despite having different names and narratives, I realized they were holding the same top 10 stocks, in nearly identical proportions.

Even worse, I couldn’t find a free tool that highlighted this overlap clearly — especially not in a way that helped me decide what to do next. I didn’t just want analytics; I wanted actionable insights.

So I sat down on a Friday night and started building something lightweight. Two days later, FundLap was live.

It’s not just a product — it’s a belief: Investors deserve clarity.

Building as a Developer, Thinking Like an Investor

This wasn’t a YC-funded sprint or a 6-month roadmap. I’m a solo dev who writes TypeScript on weekdays and fiddles with data visualizations at night.

I didn’t plan to build a “startup” — I just wanted to solve a problem for myself. That made everything sharper. Fewer features, faster load, more insight per pixel.

Here’s what I optimized for:

🔍 Immediate answers: You shouldn’t need an MBA to interpret your portfolio
⚡ Speed: Zero login, zero friction — just paste fund names and compare
🧰 APIs for builders: I love tools with APIs, so I made sure FundLap had one too

It’s easy to forget this: Every time we hit “invest,” we’re trusting a system we don’t fully see. As a dev, I had the tools to look deeper — so I did.

You Think You’re Diversified? This Link Might Prove You Wrong.

Most investors assume picking three different funds = instant diversification.

But here’s a live link that might surprise you:

👉 Check this portfolio on FundLap

It’s a combo of:

Parag Parikh Flexi Cap Fund
SBI Nifty 50 ETF
Nippon India Large Cap Fund

On the surface, they sound unrelated. Different fund houses. Different strategies.

But the overlap? It’s significant. 45%, 27%, even 93% duplication in holdings.

We’re not talking minor crossover — we’re talking redundancy that eats into your returns and inflates your fees.

That page gives you:

Real overlap data (down to the exact holdings)
Suggested replacements that maintain performance
Visuals showing how sectors stack up

No login. No cost. Just raw portfolio insight.

Sometimes, the truth about your portfolio is just one chart away.

For Indie Hackers: Why I Opened the API

I didn’t just want to build a tool — I wanted to make it buildable.

So I launched the FundLap API here:

👉 FundLap API on RapidAPI

I’ve always loved open APIs — they let people remix tools into things I’d never think of. This one’s no different.

You can use it to:

Build a fund comparison Telegram bot
Auto-update a Notion dashboard with overlap scores
Power your own finance tracker with metadata, returns, and diversification visuals

It’s clean, fast, and ready to plug into whatever you’re building.

Final Thoughts

FundLap wasn’t born in a pitch deck. It came from hitting walls as an investor — and having the skills to code a way around them.

If you’ve ever looked at your portfolio and thought, “Is this really working together?” — this tool’s for you.

And if you’re a builder? There’s room to remix it into something even better.

Start here 👉 fundlap.com

Or fork the vision with the API 👉 RapidAPI

Built on a weekend. Still shipping. 🚀

Create Your Own AI RAG Chatbot: A Python Guide with LangChain

Shreshth Goyal — Sun, 20 Oct 2024 06:04:39 +0000

Who wouldn’t want instant answers from their documents? That’s exactly what RAG chatbots do—combining retrieval with AI generation for quick, accurate responses!

In this guide, I’ll show you how to create a chatbot using Retrieval-Augmented Generation (RAG) with LangChain and Streamlit. This chatbot will pull relevant information from a knowledge base and use a language model to generate responses.

I’ll walk you through each step, providing multiple options for response generation, whether you use OpenAI, Gemini, or Fireworks—ensuring a flexible and cost-effective solution.

What is Retrieval-Augmented Generation (RAG)?

RAG is a method that combines retrieval and generation to deliver more accurate and context-aware chatbot responses. The retrieval process pulls relevant documents from a knowledge base, while the generation process uses a language model to create a coherent response based on the retrieved content. This ensures your chatbot can answer questions using the most recent data, even if the language model itself hasn’t been specifically trained on that information.

Imagine you have a personal assistant who doesn’t always know the answer to your questions. So, when you ask a question, they look through books and find relevant information (retrieval), then they summarize that information and tell it back to you in their own words (generation). This is essentially how RAG works, combining the best of both worlds.

In a Flowchart RAG process will somewhat look like this:

Now, let’s get started, and get our own chatbot!

Setting Up the Project Environment

We'll be using Python mostly in this TUTO, if you are JS head you can follow the explanations and go through the documentation of langchain js.

First, we need to set up our project environment. This includes creating a project directory, installing dependencies, and setting up API keys for different language models.

1. Create a Project Folder and Virtual Environment

Start by creating a project folder and a virtual environment:

mkdir rag-chatbot
cd rag-chatbot
python -m venv venv
source venv/bin/activate

2. Install Dependencies

Next, create a requirements.txt file to list all necessary dependencies:

langchain==0.0.329
streamlit==1.27.2
faiss-cpu==1.7.4
python-dotenv==1.0.0
tiktoken==0.5.1
openai==0.27.10
gemini==0.3.1
fireworks==0.4.0
sentence_transformers==2.2.2

Now, install these dependencies:

pip install -r requirements.txt

3. Setting Up API Keys

We’ll be using OpenAI, Gemini, or Fireworks for the chatbot’s response generation. You can choose any of these based on your preferences.

Don't worry if you are experimenting, Fireworks provide $1 worth of API keys for free, and gemini-1.5-flash model is also free to an extent!

Set up a .env file to store the API keys for your preferred model:

# Uncomment your API key
# OPENAI_API_KEY=your_openai_api_key_here
# GEMINI_API_KEY=your_gemini_api_key_here
# FIREWORKS_API_KEY=your_fireworks_api_key_here

Make sure to sign up for these services and get your API keys. Both Gemini and Fireworks offer free tiers, while OpenAI charges based on usage.

Document Processing and Chunking

To give the chatbot context, we’ll need to process documents and split them into manageable chunks. This is important because large texts need to be broken down for embedding and indexing.

1. Create `document_processor.py`

Create a new Python script called document_processor.py to handle document processing:

from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter

class DocumentProcessor:
    def __init__(self, file_path):
        self.file_path = file_path

    def load_and_split(self):
        loader = TextLoader(self.file_path)
        documents = loader.load()
        splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
        texts = splitter.split_documents(documents)
        return texts

if __name__ == "__main__":
    processor = DocumentProcessor("data/sample_text.txt")
    texts = processor.load_and_split()
    print(f"Processed {len(texts)} text chunks")

This script loads a text file and splits it into smaller chunks of about 1000 characters with a small overlap to ensure that no context is lost between chunks. Once processed, the documents are ready to be embedded and indexed.

Creating Embeddings and Indexing

Now that we have our documents chunked, the next step is to convert them into embeddings (numerical representations of text) and index them for fast retrieval. (as machines understand numbers easier than words)

1. Create `embedding_indexer.py`

Create another script called embedding_indexer.py:

from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS

class EmbeddingIndexer:
    def __init__(self):
        self.embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")

    def create_vectorstore(self, texts):
        vectorstore = FAISS.from_documents(texts, self.embeddings)
        return vectorstore

if __name__ == "__main__":
    from document_processor import DocumentProcessor

    processor = DocumentProcessor("data/sample_text.txt")
    texts = processor.load_and_split()

    indexer = EmbeddingIndexer()
    vectorstore = indexer.create_vectorstore(texts)
    print("Vector store created successfully")

In this script, the embeddings are created using a Hugging Face model (all-MiniLM-L6-v2). We then store these embeddings in a FAISS vectorstore, which allows us to quickly retrieve similar text chunks based on a query.

Implementing Retrieval and Response Generation

Here comes the exciting part: combining retrieval with language generation! You’ll now create a RAG chain that fetches relevant chunks from the vectorstore and generates a response using a language model. (vectorstore is a database where we stored our data converted to numbers as vectors)

1. Create `rag_chain.py`

Let’s create the file rag_chain.py:

from langchain.chains import RetrievalQA
from langchain.llms import OpenAI, Gemini, Fireworks
from dotenv import load_dotenv
import os

load_dotenv()

class RAGChain:
    def __init__(self, vectorstore):
        self.vectorstore = vectorstore
        self.llm = self.get_llm()

    def get_llm(self):
        if os.getenv("OPENAI_API_KEY"):
            return OpenAI(api_key=os.getenv("OPENAI_API_KEY"), temperature=0)
        elif os.getenv("GEMINI_API_KEY"):
            return Gemini(api_key=os.getenv("GEMINI_API_KEY"), temperature=0)
        elif os.getenv("FIREWORKS_API_KEY"):
            return Fireworks(api_key=os.getenv("FIREWORKS_API_KEY"), temperature=0)
        else:
            raise ValueError("No valid API key found! Please set one in .env file.")

    def create_chain(self):
        retriever = self.vectorstore.as_retriever(search_kwargs={"k": 3})
        qa_chain = RetrievalQA.from_chain_type(
            llm=self.llm,
            chain_type="stuff",
            retriever=retriever,
            return_source_documents=True
        )
        return qa_chain

if __name__ == "__main__":
    from document_processor import DocumentProcessor
    from embedding_indexer import EmbeddingIndexer

    processor = DocumentProcessor("data/sample_text.txt")
    texts = processor.load_and_split()

    indexer = EmbeddingIndexer()
    vectorstore = indexer.create_vectorstore(texts)

    rag_chain = RAGChain(vectorstore)
    qa_chain = rag_chain.create_chain()

    query = "What is the capital of France?"
    result = qa_chain({"query": query})
    print(f"Answer: {result['result']}")

Here, we give you the choice between OpenAI, Gemini, or Fireworks based on the API key you provide. The RAG chain will retrieve the top 3 most relevant documents and use the language model to generate a response.

You can switch between models depending on your budget or usage preferences—Gemini and Fireworks are free, while OpenAI charges based on usage.

Building the Chatbot Interface

Now, we’ll build a simple chatbot interface to take user input and generate responses using our RAG chain.

1. Create `chatbot.py`

Create a new file called chatbot.py:

class Chatbot:
    def __init__(self, qa_chain):
        self.qa_chain = qa_chain

    def get_response(self, user_input):
        try:
            response = self.qa_chain({"query": user_input})
            return response['result']
        except Exception as e:
            return f"An error occurred: {str(e)}"

if __name__ == "__main__":
    from rag_chain import RAGChain
    from document_processor import DocumentProcessor
    from embedding_indexer import EmbeddingIndexer

    processor = DocumentProcessor("data/sample_text.txt")
    texts = processor.load_and_split()

    indexer = EmbeddingIndexer()
    vectorstore = indexer.create_vectorstore(texts)

    rag_chain = RAGChain(vectorstore)
    qa_chain = rag_chain.create_chain()

    chatbot = Chatbot(qa_chain)

    while True:
        user_input = input("You: ")
        if user_input.lower() in ['exit', 'quit', 'bye']:
            print("Chatbot: Goodbye!")
            break
        response = chatbot.get_response(user_input)
        print(f"Chatbot: {response}")

This script creates a command-line chatbot interface that continuously listens for user input, processes it through the RAG chain, and returns the generated response.

Creating the Streamlit UI

It’s time to make your chatbot even more user-friendly by building a web interface using Streamlit. This will allow users to interact with your chatbot through a browser.

1. Create `app.py`

Create app.py:

import streamlit as st
from document_processor import DocumentProcessor
from embedding_indexer import EmbeddingIndexer
from rag_chain import RAGChain
from chatbot import Chatbot

@st.cache_resource
def initialize_chatbot(file_path):
    processor = DocumentProcessor(file_path)
    texts = processor.load_and_split()
    indexer = EmbeddingIndexer()
    vectorstore = indexer.create_vectorstore(texts)
    rag_chain = RAGChain(vectorstore)
    return Chatbot(rag_chain.create_chain())

st.title("RAG Chatbot")

uploaded_file = st.file_uploader("Upload a text file for the knowledge base", type="txt")

if uploaded_file:
    with open("temp_knowledge_base.txt", "wb") as f:
        f.write(uploaded_file.getbuffer())

    chatbot = initialize_chatbot("temp_knowledge_base.txt")

    if "messages" not in st.session_state:
        st.session_state.messages = []

    for message in st.session_state.messages:
        st.chat_message(message["role"]).markdown(message["content"])

    if prompt := st.chat_input("Ask a question"):
        st.chat_message("user").markdown(prompt)
        st.session_state.messages.append({"role": "user", "content": prompt})

        response = chatbot.get_response(prompt)
        st.chat_message("assistant").markdown(response)
        st.session_state.messages.append({"role": "assistant", "content": response})
else:
    st.write("Please upload a text file to start chatting!")

2. Run the Streamlit App

To run your Streamlit app, simply use:

streamlit run app.py

This will launch a web interface where you can upload a text file, ask questions, and receive answers from the chatbot.

Optimizing Performance

For better performance, you can experiment with chunk size and overlap when splitting the text. Larger chunks provide more context, but smaller chunks may make retrieval faster. You can also use Streamlit caching to avoid repeating expensive operations like generating embeddings.

If you want to optimize costs, you can switch between OpenAI, Gemini, or Fireworks depending on the query complexity—use OpenAI for complex questions and Gemini or Fireworks for simpler ones to reduce costs.

Wrapping Up

Congratulations! You've successfully created your own RAG-based chatbot. Now, the possibilities are endless:

Create your own personalized study buddy.
No more going through long documentations—just "RAG it out" for quick, accurate answers!

The journey starts here, and the potential is limitless!

You can follow my work on GitHub. Feel free to reach out—my DMs are always open on X and LinkedIn.

A Beginner's Guide to Tree-sitter

Shreshth Goyal — Sat, 06 Apr 2024 11:46:29 +0000

Navigating through extensive or complex code can be challenging, but Tree-sitter is here to assist.

Key Background Knowledge

Tree-sitter

Tree-sitter generates parsers based on a language and provides insights about the code as seen by the engine. It takes that messy code and weaves it into a clear, structured map, revealing the relationships between different parts. This map, called an Abstract Syntax Tree (AST), acts like a visual blueprint, making complex code easier to understand.

Abstract Syntax Tree (AST)

AST (Abstract Syntax Tree) breaks down code into its fundamental elements – variables, functions, and expressions – and shows how they interact, similar to a family tree where each person is connected to other via a bloodline. The root of the tree represents the entire program, with branches stemming out for different sections. Just like a family tree clarifies relationships, the AST makes code connections crystal clear.
More formally it is a graph representation of source code primarily used by compilers to read code and generate the target binaries.

You can play with your code here, and visualise ASTs for the same.

Coding Time!

Example code

Lets's see AST visualisation and Tree-sitter working for the following Python code!



def greet(name):
  print("Hello, " + name + "!")

In constructing the Abstract Syntax Tree (AST), the function definition is designated as the root node, with "greet" and "name" forming its direct descendants or children, illustrating the hierarchical relationship between the function and its components. The print statement is then integrated into the AST as another child node of the function definition, with a further breakdown of its internal syntactic structure to represent the operation and its operands as shown in the following image:

AST formed above can be visualised as a tree as follows:

Isn't this cool?

Using Tree-sitter in Your JavaScript Project

While Tree-sitter itself is majorly writtern in Rust, there are JavaScript bindings that allow you to interact with it from your JavaScript code. Here's a glimpse into what you can do:

Installation: First, you'll need to install the Tree-sitter library and the Python parser using npm:



npm install tree-sitter tree-sitter-python --legacy-peer-deps

legacy-peer-deps flag is added due to peerDependencies conflict

Write Some Python Code: Create a simple Python file with some code you'd like to analyze (like the greet function from before). Save this file as your_code.py.
Import the Libraries: In your JavaScript file, import the necessary libraries:



const TreeSitter = require('tree-sitter');
const Python = require('tree-sitter-python');
const  fs  =  require('fs').promises;

Parse the Code: Use the libraries to parse your Python file and generate the AST. Here's how:



async function parseCode(codePath) {
  // Load the Python parser
  const  parser  =  new  TreeSitter();
  // Set the language to the parser
  parser.setLanguage(Python);
  // Read the code file content
  const codeContent = await readFile(codePath, 'utf8');
  // Parse the code using the chosen parser
  const tree = await parser.parse(codeContent);
  return tree.rootNode;
}

Explore the AST (Doing it on our own!)

This can be overwhelming in the beginning, stick with me through this.

The AST is a complex data structure, but we can explore its basic elements. Here's a breakdown of the rootNode object:

rootNode.type: This property tells you the type of node (e.g., "function_definition" for our example).
rootNode.children: This is an array containing child nodes of the current node.
rootNode.text: This property might contain the actual text content of the node (e.g., the function name or the string to be printed).

Here's some example code to explore the root node of the AST:



async function printASTNodeInfo(rootNode) {
  console.log(`Node type: ${rootNode.type}`);
  // Loop through child nodes
  for (const child of rootNode.children) {
    console.log(`  - Child node type: ${child.type}`);
    // Explore child nodes recursively
    if (child.children.length > 0) {
      await printASTNodeInfo(child);
    } else {
      // For leaf nodes (no children), print the text content
      if (child.text) {
        console.log(`    - Text content: ${child.text}`);
      }
    }
  }
}

Adding the main function: Integrating all the functions together



async  function  main(codePath) {
try {
const  rootNode  =  await  parseCode(codePath);
await  printASTNodeInfo(rootNode);
} catch (error) {
console.error(`Failed to parse code: ${error.message}`);
}}
// Calling the main function
main(YOUR_CODE_PATH);

Explanation:

Recursive Exploration: The code now includes a nested function call to printASTNodeInfo for each child node. This creates a recursive loop, where it explores the child node's type, then further explores its children if it has any. This allows you to traverse the entire AST structure.
Leaf Nodes: The code checks for the presence of child nodes. If a child node doesn't have any children itself (meaning it's a leaf node), it might contain the actual text content of the code element it represents. The code checks for the text property and prints it if available.

Running the Code:

Make sure you have Node.js and npm installed.
Create two files: your_code.py (containing your Python code) and analyze_code.js (containing your JavaScript code with the functions above).
In analyze_code.js, modify the parseCode function to return the entire AST (tree) instead of just the root node.
Call the parseCode function in your script, passing the path to your Python file as node analyze_code.js in your terminal.
Once you have the AST (tree), call printASTNodeInfo with the tree.rootNode as the argument.

Example Output:

Assuming your Python code is the greet function example, the output might look something like:



Node type: function_definition
  - Child node type: identifier
    - Text content: greet
  - Child node type: parameter
    - Text content: name
  - Child node type: block_statement
    - Child node type: expression_statement
      - Child node type: call_expression
        - Text content: print
        - Child node type: string_literal
          - Text content: Hello,  + name + !Node type: module
  - Child node type: function_definition
Node type: function_definition
  - Child node type: def
    - Text content: def
  - Child node type: identifier
    - Text content: greet
  - Child node type: parameters
Node type: parameters
  - Child node type: (
    - Text content: (
  - Child node type: identifier
    - Text content: name
  - Child node type: )
    - Text content: )
....

This gives you a glimpse into the AST structure, showing the different node types and potentially some text content for leaf nodes. Remember, this is a basic exploration, and the AST can get more complex depending on the code you analyze.

Seeing Tree-sitter in Action (Playground Version)

While Tree-sitter itself requires some coding knowledge, there are online tools that allow you to see it work! Here's an example using https://tree-sitter.github.io/tree-sitter/playground:

Visit the Tree-sitter Playground (https://tree-sitter.github.io/tree-sitter/playground).
Select "Python" from the language dropdown menu.
Paste your Python code snippet (like the greet function) into the code editor.
Click the "Parse" button.

Yayy! If your code is valid, you'll see the AST generated below the code editor. You can click on different nodes in the tree to see more information about that specific part of the code.

Beyond the Basics:

While exploring the raw AST structure can be informative, some libraries offer functionalities to visualize the AST as a tree structure. These libraries are separate installations and might require additional configuration. However, the core concept remains the same: you use Tree-sitter to generate the AST and then leverage other tools for visualization or further analysis.

GitGraph is one of my latest project majorly built with the help of tree-sitter. This transforms your codebase (currently supports python only codebases) into an interactive graph, revealing relationships between functions, classes, and files. Drag, zoom, and hover for deep dives, with sidebars displaying neighbor nodes and imported functions. Hyperlinks take you straight to the source code.
Master your Python project's structure with GitGraph's clear and concise dependency view. You can view the backend source code here.
Do drop a ⭐ if you like it!

This concludes our exploration of Tree-sitter in JavaScript. Remember, this is just the beginning! As you delve deeper into Python development, Tree-sitter can be a valuable tool for understanding complex code structures and making your coding journey more efficient.

Project Ideas

Some project ideas where you can try your hands on:

Add more language supports to GitGraph.
A browser extension that summarizes code files on GitHub using Tree-sitter, providing quick insights into the code's structure and complexity.
A Code Complexity Analyzer that evaluates the complexity of functions and methods in a codebase, providing recommendations for simplification or refactoring based on the AST.

Dive deeper and follow me on Twitter, GitHub, and LinkedIn!

If you found this guide helpful and created your own awesome CLI tool, I'd love to see it! Please share your GitHub repositories in the comments section below. Let's build together!

A Guide to Building CLI Tools in JavaScript

Shreshth Goyal — Thu, 28 Mar 2024 22:03:04 +0000

Welcome to an exciting journey into the world of command-line interface (CLI) tools using JavaScript.

In this guide, I'll walk you through creating a CLI tool that sets up a basic project structure, explaining every step and piece of code to ensure you can follow along.

Setting Up Your Development Environment

Before we dive into the world of CLI tools, let's set up our environment:

Install Node.js and npm: Head to Node.js's official website and download the recommended version. Installing Node.js also installs npm, the package manager you'll use to handle JavaScript packages.
Verify Your Installation: Ensure everything is installed correctly by opening your terminal and running:

   node --version
   npm --version

Seeing version numbers confirms you're all set!

Building Your CLI Tool: Project Setup Automation

Our goal is to create a CLI tool that automatically generates a basic project structure, saving you the hassle of manually creating folders and files every time you start a new project.

Step 1: Kickstarting Your Project

Create a Project Directory: This is where your CLI tool's code will reside.
```
mkdir my-project-setup
cd my-project-setup
```
Initialize Your npm Package: This step generates a package.json file, which is crucial for managing your project's dependencies and configurations.
```
npm init -y
```

Step 2: Crafting Your CLI Application

Create the Main File: Name this file index.js. It will contain the logic of your CLI tool.
Incorporate a Shebang Line: At the beginning of index.js, add:
```
#!/usr/bin/env node
```
This line tells your system to use Node.js to execute this script.

Implementing the Logic: The code below creates a predefined project structure with directories and files:

const fs = require('fs'); // File System module to handle file operations

// Define the project structure: directories and their respective files

const projectStructure = {
  'src': ['index.js'],
  'public': ['index.html', 'styles.css'],
};

// Iterate over the structure, creating directories and files

Object.entries(projectStructure).forEach(([dir, files]) => {
  fs.mkdirSync(dir, { recursive: true }); // Create directories
  files.forEach(file => fs.writeFileSync(`${dir}/${file}`, '')); // Create files
});

console.log("Project structure created successfully!");

Here's what each part of the code does:

Require fs module: This is Node.js's file system module, which you'll use to create directories and files.
Define project structure: We specify which directories to create and what files they should contain.
Create directories and files: Using fs.mkdirSync and fs.writeFileSync, the script creates each directory and file according to the structure defined.

4. Make Your Script Executable: Modify package.json to include a "bin" section. This tells npm which command should execute your script:

"bin": {

  "setup-project": "./index.js"

}

Step 3: Testing and Linking Your Tool Locally

Before sharing your tool, test it:

Link Your Tool: Run npm link in your project directory. This command creates a symlink that allows you to run your CLI tool from anywhere in your terminal.
Run Your Tool: Simply type setup-project in your terminal in project's directory. If everything is set up correctly, you'll see the "Project structure created successfully!" message, with the project structure as mentioned.

YES, THAT'S ALL IT TOOK!

Step 4: Enhancing Functionality

Your tool now automates project setup, but there's room for improvement. Consider adding more features or handling user input to customize the project structure. Explore packages like yargs for parsing command-line arguments. You can learn more about yargs through their official documentation here.

Step 5: Sharing Your Tool on npm

Ready to share your CLI tool with the world? Here's how:

Sign Up on npm: If you don't have an account, create one at https://www.npmjs.com/signup.
Log In via Terminal: Run npm login and enter your npm credentials.
Publish Your Package: In your project directory, execute npm publish.

Congratulations! Your CLI tool is now available on npm for everyone to use.

If you have followed me till here, do checkout my first CLI tool I made in Javascript, and published on npm: Naturalshell. Naturalshell provides AI in your terminal, no need of memorising shell commands now!
You can checkout the npm package here and the github repository here.
Do drop a ⭐, and feel free to add new features and build on naturalshell with me!

This tool leverages AI to parse natural language and provide shell commands to users based on their needs. Beyond just providing commands, it also offers concise explanations, ensuring users not only know what to do but understand how it works. With the added convenience of editing commands directly within the tool and the ability to execute them immediately, NaturalShell delivers a user-friendly, intuitive experience

Wrapping Up

You've taken a significant first step into the world of CLI tool development with JavaScript. By building a simple yet functional tool, you've learned the essentials of creating, testing, and publishing CLI applications. Keep experimenting, learning, and building. The command line is a powerful ally, and now, it's yours to command. Happy coding!

If you found this guide helpful and created your own awesome CLI tool, I'd love to see it! Please share your GitHub repositories in the comments section below. Let's build together!

User Authorization in NodeJS using PostgreSQL

Shreshth Goyal — Mon, 30 Aug 2021 17:57:00 +0000

User authentication is one of the integral parts while developing a web application. Although we can use third party apps like Firebase or Passport to authenticate our user, but in this blog we'll use a simple and self-developed approach by creating REST APIs using NodeJS and PostgreSQL which will help us understand fundamentals of authentication.

Prerequisites and Technologies Used

nodeJS
nodemon
express
cors
pg
bcrypt
jwt
dotenv

Also it is preferred to have an account on ElephantSQL.

Basic Structure

Once you have all the dependencies set up you can start by first making a folder structure. Here we will be using MVC folder structure, which stands for Model --> View --> Controller type of structure. Here we will be making different folders for each task.

In your root directory run the following command

touch server.js
mkdir routes controller routes configs

This will create all the folders that we will be required to authorize our users. server.js will be our root file where we will be initiating our server with the help of ExpressJS.

It might be daunting for few, but stick with me till the end and you'll surely get an idea how to create your server and how to work through it authorizing and authenticating new users to your application.

Starting Our Server

In our main server.js file we will be structuring our server.
Our server.js should look like

const  express  =  require("express");

const app = express(); //Initialized express

app.use(express.json());
app.use(cors());

const port = process.env.PORT || 5000;

app.get("/", (req, res) => {

res.status(200).send("Engine Started, Ready to take off!");

})

app.listen(port, () => {

console.log(`Here we go, Engines started at ${port}.`);

})

Here we added cors which enables cross origin sharing of resources. You can learn more about it here.
Here we process.env.port looks for any free port to run our server locally if our port is not free as in our case port 5000. We can check if our server is running properly by running localhost:5000.

'Users' table schema in PostgresSQL

To create our users table now we can run postgres locally too, but in this blog we are using ElephantSQL to ease connecting it remotely with our backend. After creating an account on ElephantSQL, copy the database URL from the details section to your clipboard.
We have to run the following SQL queries to create our users table.

CREATE TABLE "users" (

"id" SERIAL PRIMARY KEY,

"name" text NOT NULL,

"email" text NOT NULL UNIQUE,

"phonenumber" text NOT NULL UNIQUE,

"password" varchar NOT NULL,

);

Users that register themselves should have a unique E-mail and a unique phone-number to prevent multiple registrations of same users, and spamming of user-bots.

Now we will have our users table where we can store our users information.

Working with our Node App

Now as we have constructed our database we need to configure and connect it to our server. In our project directory we will direct ourselves too configs directory and create two files database.js and dotenv.js.

Before adding our Postgres Credentials we will make a new file .env in our root directory to store all the values as Enviornment variables so if any other person gets our code they wont be able to access our credentials.

In our .env, add the following lines

DB_URL = paste your database URL here

Now in our dotenv.js we will add the following lines

if (process.env.NODE_ENV !== 'production') {

require('dotenv').config() //Configuring dotenv during development stage

}

which means that if we are in a development stage server has to configure our dotenv files.

Now to connect our database to our backend we have to add these lines in our database.js to configure our postgres database.

const { Client } = require("pg");

const client = new Client(process.env.DB_URL); //Configuring PostgresSQL Database

module.exports = client;

Now as we have configured our database, to connect this to our database we have to add these lines to our server.js.

require("./configs/dotenv");
const  client  =  require("./configs/database");

client.connect((err) => { //Connected Database

if (err) {

console.log(err);

}

else {

console.log("Data logging initiated!");}

});

By doing so through our console we'll get to know whether we are connected to our database or not on our local host.

Making and Routing our User endpoint

In our server.js we will add the following lines of code, just to tell the server that whenever we fetch /users it has to run our user method.

const  user  =  require("./routes/user");

app.use("/user",  user);  //Route for /user endpoint of API

Here we can make multiple endpoints for different tasks.

We'll know what routes/user is in the following section.

Now in our routes directory we will make users.js and add the following code

const express = require('express');

const router = express.Router();

const {register} = require("../controllers/register");

const {login} = require("../controllers/login");

router.post('/register' , register); //POST request to register the user

router.post('/login' , login); // POST request to login the user

module.exports = router;

Here we are telling the server to go to these endpoints and make requests based on corresponding methods.

For example :- If our user fetches at /user/register our server will make a POST request on our register method which we will write in our next section.

If you have gotten this far in this blog, tap yourself on the back as you have created your own server, have set your own enviornment variables, created, configured and connected your own postgreSQL database to your server and have completed your first step in authorizing users to your web applications.

From here onwards it'll get more interesting and you will be writing functions that will work on Real World Projects and you'll fall in love with it.

Registering Users to our App

In our controllers directory we will create register.js where we will add function to register our user.

Here we will be making SQL queries such that to prevent SQL injection. Interested people can know more about SQL injection from here and will be doing error handling at each step, also will be using try-catch methods to write the functions.

Before moving forward with our function we need to update our .env file.
We will add follwoing variables to our .env file

SECRET_KEY = any random string here

In register.js we will add the following code

const  bcrypt  =  require("bcrypt");

const  client  =  require("../configs/database");

const  jwt  =  require("jsonwebtoken");

//Registration Function

exports.register  =  async (req, res) => {
const { name, email, phonenumber, password } =  req.body;
try {
const  data  =  await client.query(`SELECT * FROM users WHERE email= $1;`, [email]); //Checking if user already exists
const  arr  =  data.rows;
if (arr.length  !=  0) {
return  res.status(400).json({
error: "Email already there, No need to register again.",
});
}
else {
bcrypt.hash(password, 10, (err, hash) => {
if (err)
res.status(err).json({
error: "Server error",
});
const  user  = {
name,
email,
phonenumber,
password: hash,
};
var  flag  =  1; //Declaring a flag

//Inserting data into the database

client
.query(`INSERT INTO users (name, email, phonenumber, password) VALUES ($1,$2,$3,$4);`, [user.name, user.email, user.phonenumber, user.password], (err) => {

if (err) {
flag  =  0; //If user is not inserted is not inserted to database assigning flag as 0/false.
console.error(err);
return  res.status(500).json({
error: "Database error"
})
}
else {
flag  =  1;
res.status(200).send({ message: 'User added to database, not verified' });
}
})
if (flag) {
const  token  = jwt.sign( //Signing a jwt token
{
email: user.email
},
process.env.SECRET_KEY
);
};
});
}
}
catch (err) {
console.log(err);
res.status(500).json({
error: "Database error while registring user!", //Database connection error
});
};
}

This code will register your user in your database, we can test this endpoint on Postman or ThunderClient extension on VS code.

We'll understand this code in next section.

Understanding Registration function

To understand our registration function we'll break it into different blocks.

Block 1 : Checking if user is already present in our database

const  data  =  await client.query(`SELECT * FROM users WHERE email= $1;`, [email]); //Checking if user already exists
const  arr  =  data.rows;  
if (arr.length  !=  0) {
return  res.status(400).json({
error: "Email already there, No need to register again.",
});
}
...

Here we are querying our client i.e. is our database to check where user's email is already present in our database or not. Then we check whether the response of this query's rows has some length (If email is present) or not.

If length is 0, we give an error response to the user stating that he need not register again.

Block 2 : Hashing user's Password

bcrypt.hash(password, 10, (err, hash) => {
if (err)
res.status(err).json({
error: "Server error",
});
const  user  = {
name,
email,
phonenumber,
password: hash,
};

...

Here we uses bcrypt to hash user's password so if any third-party gets our database our user's password are safe and secure and cannot be cracked by third-party. 10 parameter is in the function is the number of salt rounds this function does to store the password.
You can know more about bcrypt from here.

Then we are making a user object to store all the inputted values and hashed password of the user.

Block 3 : Inserting User's Information in our Database

var  flag  =  1; //Declaring a flag

//Inserting data into the database

client
.query(`INSERT INTO users (name, email, phonenumber, password) VALUES ($1,$2,$3,$4);`, [user.name, user.email, user.phonenumber, user.password], (err) => {

if (err) {
flag  =  0; //If user is not inserted is not inserted to database assigning flag as 0/false.
console.error(err);
return  res.status(500).json({
error: "Database error"
})
}
else {
flag  =  1;
res.status(200).send({ message: 'User added to database' });
}
})

...

Here we are querying through our database and inserting our user's data with the hashed password. Also here we are declaring a variable named flag which will act as boolean for the following section.

Block 4 : Signing JSON Web Token for each User

if (flag) {
const  token  = jwt.sign( //Signing a jwt token
{
email: user.email
},
process.env.SECRET_KEY
);
};

...

Here if our User is registered to our database (Boolean flag checks this) we sign a Json Web Token for the user. Json Web Token offers signature and encrypts data given to it. In our case we are encrypting user's email to identify it later on whenever user signs into our application.
process.env.SECRET_KEY is the enviornment variable in our .env file which gives a random string with jwt functions encrypts our data.

If you are intrested you can know more about jwt from here or refer to its package documentation from here

Now as we are through with registering our user we can check these endpoint, by making POST request on /users/register and inputting required data.

Hurrayy!! 🥳🥳🥳
As of now you have developed your own backend to register your users and that too by following best practices out there.
You have prevented SQL-Injections, You have done good error handling and securely stored user's data in your database.

Now we will see how to Sign In our user into our application.

Signing In Users to our App

In our controllers directory we will create login.js where we will add function to sign in our user.

Here we will be making SQL queries such that to prevent SQL injection. Intrested people can know more about SQL injection from here and will be doing error handling at each step, also wil be using try-catch methods to write the functions.

We will add the following code to ur login.js file

const bcrypt = require("bcrypt");

const client = require("../configs/database");

const jwt = require("jsonwebtoken");

//Login Function
exports.login = async (req, res) => {
const { email, password } = req.body;
try {
const data = await client.query(`SELECT * FROM users WHERE email= $1;`, [email]) //Verifying if the user exists in the database
const user = data.rows;
if (user.length === 0) {
res.status(400).json({
error: "User is not registered, Sign Up first",
});
}
else {
bcrypt.compare(password, user[0].password, (err, result) => { //Comparing the hashed password
if (err) {
res.status(500).json({
error: "Server error",
});
} else if (result === true) { //Checking if credentials match
const token = jwt.sign(
{
email: email,
},
process.env.SECRET_KEY
);
res.status(200).json({
message: "User signed in!",
token: token,
});
}
else {
//Declaring the errors
if (result != true)
res.status(400).json({
error: "Enter correct password!",
});
}
})
}
} catch (err) {
console.log(err);
res.status(500).json({
error: "Database error occurred while signing in!", //Database connection error
});
};
};

This code will sign in your user in your database, we can test this endpoint on Postman or ThunderClient extension on VS code.

We'll understand this code in next section.

Understanding Sign In function

To understand our sign in function we'll break it into different blocks.

Block 1 : Checking if User registered to our application or not

const { email, password } = req.body;
try {
const data = await client.query(`SELECT * FROM users WHERE email= $1;`, [email]) //Verifying if the user exists in the database
const user = data.rows;
if (user.length === 0) {
res.status(400).json({
error: "User is not registered, Sign Up first",
});
}

...

Here we are querying through our database and checking if the user inputted values are present in our database or not. If our query's response has length 0, which means there is no user with these credentials it throws an error.

Block 2 : Comparing Hashed Password with User's Password

 else {
bcrypt.compare(password, user[0].password, (err, result) => { //Comparing the hashed password
if (err) {
res.status(500).json({
error: "Server error",
});
} else if (result === true) { //Checking if credentials match
const token = jwt.sign(
{
email: email,
},
process.env.SECRET_KEY
);
res.status(200).json({
message: "User signed in!",
token: token,
});
}

...

Here if user is present in our database we are using compare method of bcrypt to check where user inputted password and user's password in database are equal or not.

And if these both passwords are equal we sign a JWT token for the user, where we encrypt user's email.

Block 3 : Handling Errors While Signing in the User

else {
//Declaring the errors
if (result != true)
res.status(400).json({
error: "Enter correct password!",
});
}
})
}
} catch (err) {
console.log(err);
res.status(500).json({
error: "Database error occurred while signing in!", //Database connection error
});
};

In this part of the code we are telling the user if there is any error while logging him in the application be it related to his credentials or if it is related to database.

Now as we are through with signing in our user we can check these endpoint, by making POST request on /users/loigin and inputting required data.

If everything is done well Response 200 OK will be there and you have successfully authorized and authenticated user without using third party apps on your own.

It's time to celebrate as you have done pretty well and gained a lot of knowledge about how to work with node and express and other technologies and work your way out.
There is much more to User Authentication and this is just your beginning. You have learnt so much now explore on your own

What more you can do now?

Validate User's Email, Password and Phone Number using RegEx.
Verify User's Email by sending mails to the user using NodeMailer.
Prevent different attacks like XSS attacks on your server.
Add more endpoints and learn about middlewares.

This is my first blog and a small contribution for the community, some love would be appreciated ♥

You can contact me here regarding any feedback and queries.

DEV Community: Shreshth Goyal

I Stopped Planning and Just Shipped — Here’s What Happened

I Built FundLap in a Weekend to Fix Mutual Fund Overlap — Now It's Helping Thousands Optimize Their Portfolios

The Problem No One Was Solving

From Frustration to First Build

Building as a Developer, Thinking Like an Investor

You Think You’re Diversified? This Link Might Prove You Wrong.

For Indie Hackers: Why I Opened the API

Final Thoughts

Create Your Own AI RAG Chatbot: A Python Guide with LangChain

What is Retrieval-Augmented Generation (RAG)?

Setting Up the Project Environment

1. Create a Project Folder and Virtual Environment

2. Install Dependencies

3. Setting Up API Keys

Document Processing and Chunking

1. Create document_processor.py

Creating Embeddings and Indexing

1. Create embedding_indexer.py

Implementing Retrieval and Response Generation

1. Create rag_chain.py

Building the Chatbot Interface

1. Create chatbot.py

Creating the Streamlit UI

1. Create app.py

2. Run the Streamlit App

Optimizing Performance

Wrapping Up

A Beginner's Guide to Tree-sitter

Key Background Knowledge

Tree-sitter

Abstract Syntax Tree (AST)

Coding Time!

Example code

Using Tree-sitter in Your JavaScript Project

Explanation:

Running the Code:

Seeing Tree-sitter in Action (Playground Version)

Beyond the Basics:

Project Ideas

A Guide to Building CLI Tools in JavaScript

Setting Up Your Development Environment

Building Your CLI Tool: Project Setup Automation

Step 1: Kickstarting Your Project

Step 2: Crafting Your CLI Application

Step 3: Testing and Linking Your Tool Locally

Step 4: Enhancing Functionality

Step 5: Sharing Your Tool on npm

Wrapping Up

User Authorization in NodeJS using PostgreSQL

Prerequisites and Technologies Used

Also it is preferred to have an account on ElephantSQL.

Basic Structure

Starting Our Server

'Users' table schema in PostgresSQL

Working with our Node App

Making and Routing our User endpoint

Registering Users to our App

Understanding Registration function

Signing In Users to our App

Understanding Sign In function

What more you can do now?

1. Create `document_processor.py`

1. Create `embedding_indexer.py`

1. Create `rag_chain.py`

1. Create `chatbot.py`

1. Create `app.py`