DEV Community: Paras 🧙‍♂️

Create small stuff.

Paras 🧙‍♂️ — Fri, 23 Jan 2026 08:12:14 +0000

If you’ve ever stopped yourself from building something because it felt too small, this is for you.

I think we all get stuck at some point on the idea that whatever we build has to be big and meaningful. Often, that pressure comes from our own expectations.

But this one is to remind you about the fun of building small stuff.
No pressure, no expectations, just experimentation.

You make it because something intrigues you. Maybe because something out there is not on par with your requirements. Maybe you just want to learn or try replicating it. Or to have fun in general.

I remember when I started learning web dev, I had a lot of fun following html css tutorials and learning to build things with html css. Small animations, effects, and even css art. One can say I wasted my time and it's not used in the day to day life. But I still did it because it was fun for me. It didn't drive me away from development, instead brought me closer.

And honestly, you're allowed to build things that don't matter to anyone else. Give yourself permission and the freedom. Try not to get stuck only on “big” ideas.

Sometimes the best projects are the ones with no expectations attached.

Here are a few small things that I made, and how they helped me in unexpected ways:

React starter template: I made it for myself to save time on new project. I keep improving it as I use it more and more. I quite like this.
Echoday: A small project I made during Kendo react challenge on dev.to. And interestingly enough, it helped me in one my interviews :p
Minesweeper: An old project of mine. I actually learned how to play Minesweeper properly because of it, and now I love the game.

I hope it made you think a bit about your ideas and projects.

Thank you :)

Github | Linkedin

Web Accessibility Reference Guide

Paras 🧙‍♂️ — Mon, 29 Dec 2025 09:56:41 +0000

Hey everyone,

Recently I put together this web accessibility reference guide repo.

It highlights key things to watch out for when building web UIs, like keyboard support, semantics, and focus behavior.

It’s a collection of accessibility checks I personally find important and easy to miss or forget.

I also added some HTML, CSS, and javascript examples that focus on keyboard accessibility.

Github repo: https://github.com/paras594/web-accessibility-guide

Feel free to bookmark it or star the repo so it’s easy to come back to when needed.

Also if you find anything that can enhance it, you can share it in the comments or raise a PR on the repo.

Thank you. Hope it helps.

Checklist for Engineers Joining a New Project

Paras 🧙‍♂️ — Fri, 04 Jul 2025 08:33:50 +0000

Goal: A quick-reference checklist of dos and don'ts when you're new to a team or project - technical, cultural, and strategic.

Bookmark this for future reference

Mindset

Listen and absorb as much as you can from discussions, meetings and conversations.
Ask a lot of questions from different people in your team such as engineers, PMs, QA, designers. Do not limit yourself to just your domain.
Clear assumptions: It's good to clarify your assumptions even if they are obvious.
Focus on "why X is like that" instead of "why we are not doing Y instead of X". Understanding the context matters more than proposing alternatives early on.
Start small: Ship a tiny fix or feature to learn the end to end flow. Take notes where required to have a reference for next time.

Ramp up smartly

Learn
✅ How to build, run, test and debug the project locally.
✅ Where docs live (internal wikis, Notion, READMEs, Slack pins)
✅ What is the API setup, data flow, and connection between frontend and backend
✅ Project structure, conventions, auth, permissions, and environment variables

Release process
Understand the release process — what happens after you push code? How can you verify your changes? Is there a post-deployment checklist or test plan you need to follow?

This helps you build confidence in the full development lifecycle.

Get familiar with

GitHub workflow (branches, PRs, reviews)
Jira / PM tool workflow (tickets, estimations, priorities)

Know the Business + Team Culture

Understanding the “why” behind decisions and the way your team works helps you navigate smarter and contribute meaningfully. This helps you get the bigger picture.

What business value does the tech serve?
Which business goals are driving the current direction?
Why not switching tech is often a conscious, valid choice
Understand legacy: it exists for a reason, even if it looks messy
Learn the team’s communication norms, review style, meeting rituals

Expectations Over Time

Have realistic expectations for yourself based on the project’s size and complexity. A project is a marathon, not a sprint.

Ramp-up: 3–6 months to be confident, especially in large/legacy projects
Efficiency: 1–2 years to become fast and intuitive
Proficiency: 3–5 years to develop deep technical and domain depth

Final Tips

Take your time learning — there’s no rush to “prove yourself.”
Ship something small, build confidence.
Have fun and bring your unique value to the team.

A React Starter Template for the Impatient Developer

Paras 🧙‍♂️ — Fri, 04 Jul 2025 06:17:21 +0000

I recently created a starter template and wanted to share the process behind it.

If you're not interested in the process, here's the link: paras594/react-starter-template

For those who are interested, read ahead.

The Problem

I like building projects for fun and experimentation, but I always had one main problem: project setup. I've become well-versed in it and have no fear of getting started, but it's inefficient and damn slow. It slows me down that I have to spend a chunk of my time on setup instead of actually building the project.

Sometimes I used Next.js for its defaults with routing, data fetching, and optimizations out of the box. Other times I went with custom setups and Vite. In the past I was dabbling with webpack and custom configurations, but with time it all changed and I never finalized one single approach.

Even though I always thought of building templates I could reuse, I either didn't get the time or was too focused on the project that I let the templating part slip—until now.

The Solution

I found degit out of nowhere. I knew about it but never gave enough thought to it. But this time I understood my goal: have a template setup and add degit to the whole thing.

I quickly sat down, used the Vite + React default template, and invested time in a few key decisions on what to include.

My Goals with the Template

Flexible and extensible - Should adapt to different project needs
Reliable folder structure - Something you get familiar with over time
Formatting and linting support - Without requiring tons of configuration
Styling and components - Easy to get started with

What Made the Cut

Vite + React - Fast builds and development server
TanStack Router - File-based routing with better type safety
TanStack Query - Data fetching & server state management
Tailwind + DaisyUI - Utility-first styling with ready-to-use components
Biome - Linting and formatting
TypeScript - Type safety
PWA support - Progressive web app features

The goal was simple: clone the repo and start building features on day one.

Why These Choices?

Vite is fast,
TanStack Router provides a cleaner approach to handling routing than React Router,
TanStack Query takes care of all the boilerplate that comes with data fetching,
Tailwind + DaisyUI gives great default components without writing tons of classes,
Biome avoids the hassle of keeping ESLint and Prettier updated while eliminating config bloat.

How It Evolves

I actually use this template for my own projects, which means it stays current with my real-world needs. Dependabot keeps the dependencies updated, and I add or remove things as I encounter them in actual development.

I'm also considering branches for different use cases - maybe a minimal version, or branches with specific additions like authentication or database integration. The idea is to have options without bloating the main template.

The template is on GitHub: paras594/react-starter-template

Follow the steps in README and you are good to go.

Final Thoughts

This template isn't revolutionary - it's just a collection of tools that work well together and let you focus on building features instead of configuration. Feel free to use it if it matches what you usually reach for, or ignore it if your stack is different.

EchoDay: Capture and reflect on your day

Paras 🧙‍♂️ — Sat, 22 Mar 2025 05:32:57 +0000

This is a submission for the KendoReact Free Components Challenge.

What We Built

EchoDay is a web app centered around daily reflection, where each day feels like an echo. Whatever you add - notes, memories, or thoughts - comes back to you later, helping you revisit and reflect on moments that matter.

Features
It features a set of handy widgets - one place to store them all.

A Calendar to plan your schedule
Notes for jotting down ideas
An About My Day section to reflect on your day
Todos list to stay on top of tasks
Reminders, so you never miss anything
Memories, where you store cherished images

Oh, it also has a little mood tracker with some fun emojis and moods to choose from :)

Right now, it’s a prototype, but if people love it, we are planning to add a fully functional backend to make it even more dynamic. It’s a simple idea, but one we think can make life a lot more organized and meaningful!

Do share your review in the app. We have included a user review section at the bottom.

Demo

Demo Link: EchoDay
Github Repo: EchoDay Repo
UI/UX Case Study: Behance Case Study

Components Used

Here is a list for reference for the components used in the project

Grid
Icons & Loader
Input & TextArea
StackLayout
Typography
Card
AppBar
Avatar
Rating
Button and more.

Thanks to My Teammate

A heartfelt thanks to Muskan for beautifully designing the web app and making the most of the KendoReact Figma UI Kit. Your creativity and attention to detail truly shine through!

KendoReact Experience

KendoReact is a fun UI library to work with. It provided a wide range of pre-built, customizable components that made the development process smooth and efficient. Features like layout components, grid, icons, typography and popups saved a ton of time and allowed me to focus more on functionality and user experience.

One of the standout features was the ability to install only the necessary packages. This not only kept the project lightweight but also made managing dependencies much easier.

However, I found the documentation could use improvement. While it covers the basics well, certain advanced features lacked detailed explanations, and some examples were too generic to be practically helpful.

In conclusion, building this app was a rewarding experience. Using KendoReact was a delight in this project. Even the free version provided enough components out of the box. Overall, it was a valuable journey, thanks to KendoReact and Dev.to team for hosting this fun challenge.

Project: Echopad

Paras 🧙‍♂️ — Sun, 16 Jun 2024 08:33:32 +0000

Hey Guys,

Excited to share my latest project - EchoPad. A powerful web app that helps you sync texts, links & files seamlessly across devices in real-time and enjoy hassle-free file management.

🔗 EchoPad: https://echopad.vercel.app/

Register & Login on EchoPad.
Type in your block of text, links or upload some files using file manager.
Access your account on another device & there you have your data.

❓Why use EchoPad?

No need to email yourself links, files & texts.
No need to rely on messaging apps to share files across devices.
No need to login your personal accounts & mails on some other device.

And to make it convenient, you can install it and use it like any other app on your device.

⏩ Stay tuned for upcoming features! I am continuously improving and refining the EchoPad.

Technical aspects of project

It's a small Progressive Web App, built using React, Node, Express and MongoDB. Socket.io is also used to help sync data in real time to avoid saving and refreshing data manually in multiple devices. It also supports file upload to be able share some small files across devices. To save some space, files get deleted after 24hrs.

Apart from this, there is not much to it. It's a simple web app. Give it a try and share your thoughts.

What can we learn from Bootstrap CSS

Paras 🧙‍♂️ — Mon, 27 May 2024 09:59:40 +0000

The world of programming has changed a lot. There was a time when writing code was like an art. Principles of programming were the backbone of any software. They are still there, but due to the shift in technology, pace of projects, and change of mindset, we have drifted away from those fundamentals.

I want to show how technologies around us still utilize these fundamentals and doing great in market. One of them is Bootstrap CSS.

That's why I have decided to share my observations on Bootstrap CSS. It's not that great now and there are other frameworks but let's see what Bootstrap does right and what we can learn from it.

So, let's begin.

Understanding the basic design of bootstrap

Bootstrap is based around css classes. It's filled with components that are reusable and composable. Then, it has classes for building layouts, range of form components, some helpers and then some utilities. By combining them all, we can build good looking, functional and user friendly web pages very quickly.

But what can we even learn from Bootstrap? Let's find out!

Principles of programming in Bootstrap?!

It may not seem at first glance, but bootstrap is using many common programming principles in front of our eyes.

KISS: Keep It Simple, Stupid
DRY
Single Responsibility Principle
Separation of Concerns
Law of Demeter

KISS: Keep It Simple, Stupid
Bootstrap is quite simple to pick and learn. Once you understand how they have structured the framework, it's easy to get up and running with. Need a button? Add btn class and it's done. Worried about responsiveness? Got you covered with media query utilities. And that's how Bootstrap utilizes our first principle.

Learning: Keep your code simple and to the point. Overengineering only adds complexity.

DRY: Don't Repeat Yourself
When we first start learning CSS, we tend to write a lot of duplicate stylings, classes etc. But in Bootstrap, we have these component classes that helps us avoid writing duplicate css. If there are two cards in different sections, it's most likely that we are using the same card class in both of them and making small adjustments as per requirement.

Learning: Try to avoid duplicate code and leverage functions, classes, utilities to create reusable components.

Single Responsibility Principle: (S in SOLID)
Every single class in Bootstrap has one responsibility and one function. A btn class is only designed to build button structure. It is not adding extra margins around and doing some crazy stuff with sibling elements. A container class is only responsible for keeping content in center and in a container, nothing more. Same goes for row class, margin utilities etc. It avoids a ton of complexity in building layouts.

Learning: This principle is easy to understand but hard to master. Make sure your components and entities are not fulfilling multiple responsibilities. One entity equals to one responsibility.

(SoC) Separation of Concerns
A component in Bootstrap doesn't interfere with another component directly. Each component, layout class, utility and helper, addresses a specific concern and has a specific job. A container class is independent of row class. row class handles the 12-column grid system, and container class is concerned with only centering and specifying width of content. Imagine a class that is a mix of both (I know you have written such CSS, because we all do)

Learning: Avoid mixing concerns and features. Try to divide it in chunks such that each chunks solves a particular problem.

Law of Demeter: The principle of least knowledge
This one was new for me. The Law of Demeter says that each component should have only limited knowledge of other components. So, if something is closely related to a component, it can have a limited knowledge of it, but not any stranger component. "Only talk to your immediate friends"
In bootstrap, form components and classes only affect form related components, or a btn modifier class only affects a btn and not any other entity.

Learning: Take every parameter as an info. Entity which needs the info, is the only one to have the info and not anyone in between. This also simplifies testing and makes code more readable.

So, these are some of the points that I observed, and I am sure there are many other things that I have not noticed just yet. If a CSS framework can apply these principles and become the most used framework, then maybe we can also take a step back and try to apply these principles that not only solves the problem at hand but also adds value to our projects.

Even today, there are gems from which we can learn and improve our understanding. Open source is filled with such learnings. I encourage everyone to read other's code, understand their approach and try to apply the same and see how it helps and how you can improve.

That's it for this one. Hope you find it helpful. Feel free to share your observations!

Caching in depth

Paras 🧙‍♂️ — Fri, 26 Apr 2024 09:01:18 +0000

In the previous post we learned about what is caching and why it's useful. Now, we are going to understand some more concepts related to caching.

Topics Covered

Comparison with different databases
Cache eviction policies
Cache miss
Cache invalidation strategies
Challenges with caching
Cache metrics

When working with caching database, it may seem a simple task to store the frequently accessed data in cache and retrieve on subsequent calls instead of hitting our main DB. But, it's not that simple.

What if your cache storage is full, and you want to get rid of unnecessary cache data. In this case, how will you determine which data to be discarded and which data to be kept?

What if the data requested is not in cache. Should you save this after fetching from main DB or not?

What amount of your total data should you store in cache? How do you decide the storage size?

These are some of the problems that we face when working with caching databases.

Why cache DB like Redis or Memcached are faster than normal DB like Mongo or MySQL?

Simple answer to this question is, cache databases are in-memory databases and do not store data on disk. On the other hand, databases like MySQL & Mongo store data on disk storage. And because RAM is faster than Disk, it is ideal to make reads & writes faster.

But RAM is also more expensive, therefore, it requires some consideration on what data to store and what data to discard.

In-Memory storage is not the only way, we can also choose to store cache in disk, CDN, etc.

Cache Eviction Policies

Cache eviction policy is just deciding which data to discard and which data to keep when cache reaches its limit.

The main benefit of doing this is, it optimizes our cache performance because we are removing unnecessary data.

There are many policies available that we can choose from:

LRU (Least Recently Used) - remove the data is used least frequently. This policy is based on the time. Example, if a user accesses a web page and is likely to visit it again, then it's a good idea to evict least recently used data.
MRU (Most Recently Used) - remove the most recently used data. Example, if a user watches a movie, it's less likely that he will watch it again, in this case, it's best to evict the most recently used cache.
LFU (Least Frequently Used) - this policy is based on how frequently same data is being accessed. If a data is used less frequently by users, we evict it to make space for new data. Example, databases can cache the results of queries that are more frequently used.
MFU (Most Frequently Used) - this policy is the opposite of LFU and may seem unnecessary. But, there are times when this can be helpful. I don't have an example for this one, so if you have one, do share in comment :)
FIFO (First In, First Out) - remove the data that got added first, to make space for the new data. It's similar to a queue. One simple example can be browser history, in which the oldest item in the history is removed to make room for newly visited page.
LIFO (Last In, First Out) - it may seem similar to MRU, but instead of considering the frequency, we just remove the most recent data from cache.
Size Based Eviction - in applications where data is large in size, we may choose to discard data with the largest size and to make room for new data. This ensures that cache storage is not full just by storing a few amount of data. Example, in a CDN, we can choose to discard the data according to the size and frequency to ensure we have space in cache for future data.
Adaptive Replacement Cache - this strategy of cache eviction is based on algorithm that dynamically decides which data to discard and which data to keep by observing the workload. It maintains a list of frequently used and recently used data. You can explore it in details as this strategy can provide great performance in various scenarios but may take some efforts to setup.

Fun Fact: Caching is also used in problem solving and building efficient algorithms especially in Dynamic Programming.

What should be the size of our cache storage?

Now that we have an idea of what options we can have to discard the data we need, next thing that we need to know is what should be the size of our cache storage.
So, usually it depends on the use case and architecture of your system, but if you don't know what to start with, you can consider 20% of Daily Traffic as the amount of data to cache. Suppose your application has a traffic of around 10gb daily, then you can have the cache storage of 2gb.

What's a cache miss??

When you check the cache storage for data and you don't find one, that is called cache miss. This is when you make call to your main database to get the data and update cache.

Cache Invalidation Strategies

Cache invalidation is deciding when and how cached data should be considered stale or invalid. This makes that data no longer usable.

Key Difference b/w eviction and invalidation
The difference between cache invalidation and cache eviction is cache eviction is deciding what data to discard when cache storage is full, while cache invalidation is marking cache data as invalid.

Benefits of Cache invalidation:

Cache invalidation affects the consistency of data served to the user.
Maintains up to date data in cache, consistent with our main database.

Strategies

There are many strategies for cache invalidation based on different factors that caters to different use cases:

Time Based: In this strategy, we set an expiration on the cached data. Once the expiry is hit, data is marked as stale and we can either discard it or replace it with fresh data. It's a fairly simple and effective strategy. Key thing to keep in mind is to decide the expiration time.
Write Through: we update our cache and main database together when we receive new data. One example of this can be, suppose in a ecommerce website, products count can be updated in both cache and main database. Drawback of this strategy is that it increases the latency as data needs to be saved in cache and main DB before responding to user.
Write Back: This strategy can be applied in real time applications, where multiple users are collaborating. So, we start by storing the updates in cache storage, which results in faster reads and writes, and when a certain threshold is reached, we update the main database using our cache.
Write Behind: While write-back strategy waits for certain condition to trigger the update to main database, write-behind is based on eventually updating the main DB on specified intervals. Example, in online applications, data can be first written to cache, and can be stored in main database on specified intervals.
Key Based: In this strategy, cache data is stored against a key, which can be a unique identifier or a group of relevant attributes. When there are changes to the actual data, such as delete or update, cached data is also marked as invalid to ensure users receive the most current information.

Challenges with caching

Always keep in mind, whenever we introduce something in our system, however small it is, it always brings a level of complexity in our system. Same is true for caching.
Caching may seem simple on surface but due to its versatile nature, it presents several challenges that we need to address to actually derive the value out of it.

Let's see what are some common challenges with caching:

Cache Invalidation: Above, we discussed some of the strategies that are used for cache invalidation. But choosing the right one is not easy. Different architectures require different type of strategies. So, try to look at all the possible scenarios and monitor the cache and data to come up with the right strategy.

Tip: You can use tools like DataDog, perf (Linux), New Relic etc. to monitor cache performance

Cache Eviction: Similar to cache invalidation, choosing the right cache eviction policy is crucial. Don't just go with Least Recently Used eviction policy by default. Analyze data and your system requirements and then decide the best policy.
Cache Security: Make sure your cache database is protected from unauthorized access. Therefore, implement proper access control and authentication system to ensure data privacy and integrity.
Cache Expiration: This can be one of the hardest thing to achieve. If you set a short expiration, your chances of cache miss will increase. And if you set a long expiration, your data will become stale and inconsistent with the DB. So, give this also some thought when setting this up. Discuss with your team and monitor your cache to eventually reach the sweet spot.

Tip: The better you understand your application's data access patterns, the easier it is to setup your caching layer.

Cache Metrics

When trying to setup monitoring for your cache, there are a few metrics that you can focus on to get a better understanding. Our goal with cache metrics is to determine how well our caching strategy is performing under different workloads.

Hit Ratio: Also known as Cache Hit Rate, helps us evaluate how well our cache system is performing.

Cache Hit Ratio(%) = (Number of Cache Hits / Total Number of Cache Accesses) * 100

where,
Number of cache hits is the total number of cache hits where data was found in cache, and
Total Number of cache accesses is the total number of cache hits made

Higher Hit Ratio means cache is used adequately and we are getting good performance by having reduced latency, better performance and less load on our backend. Cache Hit Ratio of 90-97% is ideal for a cache system.

Cache Miss: It is opposite of Cache Hit and indicates that higher number of requests are not utilizing cache properly. Thus, we should look into our application's data access pattern.
Cache Size: We can also monitor how much cache size is actually used by our system. Having a larger cache size means higher Hit Rate but that also increases memory usage of our system and may incur extra cost. So, by monitoring data and adjusting cache size, we can reduce our cost and still keep our system performant.

In summary, it's crucial to analyze your requirements thoroughly and determine the caching strategy that best fits your needs. Applying these concepts may require time and effort, but understanding and incorporating them into your coding practices will prove invaluable in the long run. Thank you for reading, and I hope this information aids you in your coding journey.

Basics of Caching

Paras 🧙‍♂️ — Wed, 13 Mar 2024 09:13:26 +0000

Caching is one of the things that is key in improving the performance of any system. Doesn't matter if it is frontend, backend, CPUs or our day to day devices, you will find them using caching in one way or the other. And that's what we are going to learn in this two post series.

First one will cover the basics of caching & second one will cover caching related concepts and advance stuff.

What we will be covering?

What & why of caching
Different types of cache
Caching in different layers with example

What is Caching?

Caching is saving the data temporarily, that you frequently access, in a storage i.e. cache, so that you don't need to make a complete request again to get the same data.

Key Points

Temporary data
Frequent Access
Avoid complete requests for same data

Cache storage are usually faster and available closer to the processing unit e.g. frontend, backend.

Why we need Caching?

We can understand the need of caching if we understand the key points above.

Let's understand the problem:

Our primary databases are usually slow and it takes time to retrieve data from them.
Request for frequently accessed data adds unnecessary overhead to our server.
In mission critical systems, loads on our server & database needs to be reduced.
Some queries & API calls are expensive and computing them again and again is not optimal.

Therefore, to keep above points in check, we add a layer of caching.

Advantages of caching

Faster access of data results in better user experience
Reduces latency: No need to access main DB again for same data
Reduces workload on main DB & server making it cost effective
In many cases, because data is cached, it is available offline.
Saves bandwidth & leads to low network traffic.

Different types of caches

There are different types of caches that cater to different use cases. Example, a CPU needs different type of cache than a Frontend application.

Browser Caching

Starting with frontend, we have browser level caching. Very useful for frontend application, as it reduces the need to make API calls to get the same for the user.

When a user visits a page and gets the data, we save that data in browser cache and when same page is visited by the user, we read data from cache instead of making API call. We cache multiple things in frontend, e.g. API responses, Images, Videos, assets, user info etc.

Apart from browser cache storage, we can also leverage Local Storage in browser.

CDN: Content Delivery Networks

You can understand CDN as storage servers that are spread across globe and we can access data from the nearby CDN available. This avoids the need for making request to another region just to get the data.

CDNs are mostly used for storing data like images, videos, audio, static files such as css, js etc, streaming content, and large files. E.g. AWS Cloudfront, Cloudinary, Akamai

Application Server Cache

This is the cache we add on our backend side of application. When we receive a request for some data, we fetch the data from our main database, save the data in cache (redis, memcached) for future users and return the response to user.

When request for same data comes to server, we check if it exists in cache, if it does, we return the data directly. This helps reduce load on database and also makes requests faster.

Database caching

Databases also implement caching to improve query performance. Suppose there is a query which is frequently hit by multiple users. So, databases caches the result & plan for the query and returns the cached data in subsequent requests.

This results in faster queries and reduces some load on DB.

Other types of caches

There are other caches that are used in different scenarios:

CPU Cache: L1 Cache (closest to CPU), L2 Cache (shared among CPU cores), L3 Cache (shared and has more size than L1 & L2)
Memory Cache: RAM Cache to speed up I/O operations. Disk Cache to speed up Reads/Writes
DNS Caching: DNS also cache the IP addresses and other info related to Domain Name requests. E.g. AWS Route53

Caching in different layers

As we can see, caching can be utilized in various places. Let's discuss some examples and how we can leverage caching to improve our system. For simplicity, we will focus on Frontend scenarios and backend scenarios

Here is our example:

Components: We have our client, a load balancer to forward request to our servers, 2 API servers & 1 media server to send static content like images, files etc. and finally our database.

Let's say we are trying to get a long list of products for our users and it's taking a good amount of time to receive the data from our database.

Issues in our current setup

Because requested data is a long list of products, it is adding an overhead on our database.
Servers can reach their limit if too many users start making requests.
Not a good experience for client, because requests are taking time.

Let's start solving these issues

Backend
First up, it's our backend side of application.
We can add a caching layer to save the data retrieved from DB in cache storage. Then, when other users make the request for the list of products, instead of hitting our DB again, we will utilize our cached data and return that.

Benefits:

Reduces load on database
Makes future requests faster i.e. minimizes latency

Frontend
Next, we can cache data in frontend on browser level. This way, if a user revisits the products page again, we don't need to refetch the products.

There are libraries like Redux, React Query and other solutions, that help us store data locally so that we don't need make unnecessary requests to our server.

Benefits:

Reduces load on our servers
Better user experience

Static Files
We can also improve our requests for static files. Currently, they are stored on our server. Let's assume users from different region are making requests for product images. Users from nearby region will be able to get the image faster than the users from another region.

To handle this, we can utilize CDN (Content Delivery Networks). Because they are spread across globe, our data will be available close to our users.

Thus, we will upload our static files in CDN.

Other Improvements

Using pagination to reduce the amount of data requested
Rate limiter to prevent excessive traffic or abuse by limiting the number of requests that can be processed within a specific time period
Using multiple read replicas for our database

Some Popular Cache Databases

Redis
Memcached
Google Cloud Memorystore
AWS ElatiCache
Amazon DynamoDB Accelerator (DAX)
Azure Cache for Redis

CDN Services

Cloudinary (with a nice free plan for hobby projects)
AWS CloudFront
Azure CDN

So, this was it. I hope you learned something new about caching. Share your suggestions or any improvements in comments. In the next post, I will cover different concepts related to caching like cache eviction policies, cache invalidation strategies, challenges, scaling cache storage etc.

Upload images to AWS S3 using multer in node.js

Paras 🧙‍♂️ — Wed, 13 Jul 2022 09:40:51 +0000

Multer is a node.js middleware and it handles multipart/form-data that includes files, images etc.

It mostly occurs in projects that we need to handle file uploads in our APIs. We can save those files in local disk storage on our system, but its better to have a hosted platform where we can upload our files and access through anywhere using the link. One such cloud storage is AWS S3.

In this post, we will be uploading images to S3 storage using multer. Likewise, you can handle any kind of file you like. Also, we will learn to use the version 3 of AWS sdk, which has a little different setup than the previous version.

Prerequisites

Basics of node.js and express

Setup AWS Account

Create an account on AWS.
There should be services tab/link. From there, we can select S3. It will be empty but we will be able to see objects in it once we start uploading files in it.
Grab your access keys from your files in credentials section. You should have access key id and secret key. We will use these when configuring multer.
Choose your AWS region as per your location and take a note of its region id.

Setup Multer

To connect with AWS S3, we need its SDK and multer-s3 package. We will be using AWS SDK V3.

Install Multer, multer-s3 and aws sdk

$ npm install --save multer multer-s3 @aws-sdk/client-s3

Create multer middleware

In your middlewares folder, create a file uploadImage.js. You can name it anything you like.

Step 1: In uploadImage.js, we will setup the S3 client:

const multer = require("multer");
const multerS3 = require("multer-s3");
const { S3Client } = require("@aws-sdk/client-s3");

// create s3 instance using S3Client 
// (this is how we create s3 instance in v3)
const s3 = new S3Client({
    credentials: {
        accessKeyId: "YOUR_ACCESS_KEY_ID_HERE", // store it in .env file to keep it safe
        secretAccessKey: "YOUR_SECRET_KEY_HERE"
    },
    region: "ap-south-1" // this is the region that you select in AWS account
})

Step 2: In uploadImage.js, add the following. use multer-s3 to connect to client and create s3 storage for multer

const s3Storage = multerS3({
    s3: s3, // s3 instance
    bucket: "my-images", // change it as per your project requirement
    acl: "public-read", // storage access type
    metadata: (req, file, cb) => {
        cb(null, {fieldname: file.fieldname})
    },
    key: (req, file, cb) => {
        const fileName = Date.now() + "_" + file.fieldname + "_" + file.originalname;
        cb(null, fileName);
    }
});

Step 3: Using our storage instance to create our uploadImage middleware using multer

// function to sanitize files and send error for unsupported files
function sanitizeFile(file, cb) {
    // Define the allowed extension
    const fileExts = [".png", ".jpg", ".jpeg", ".gif"];

    // Check allowed extensions
    const isAllowedExt = fileExts.includes(
        path.extname(file.originalname.toLowerCase())
    );

    // Mime type must be an image
    const isAllowedMimeType = file.mimetype.startsWith("image/");

    if (isAllowedExt && isAllowedMimeType) {
        return cb(null, true); // no errors
    } else {
        // pass error msg to callback, which can be displaye in frontend
        cb("Error: File type not allowed!");
    }
}

// our middleware
const uploadImage = multer({
    storage: s3Storage,
    fileFilter: (req, file, callback) => {
        sanitizeFile(file, callback)
    },
    limits: {
        fileSize: 1024 * 1024 * 2 // 2mb file size
    }
})

module.exports = uploadImage;

Using uploadImage middleware in routes

Lets create a route where a user can update his profile image. I assume express and router is already setup.

// update profile image
router.put(
    "/:userId/profile-image", 
    uploadImage.single("image"), // our uploadImage middleware
    (req, res, next) => {
        /* 
           req.file = { 
             fieldname, originalname, 
             mimetype, size, bucket, key, location
           }
        */

        // location key in req.file holds the s3 url for the image
        let data = {}
        if(req.file) {
            data.image = req.file.location
        }

        // HERE IS YOUR LOGIC TO UPDATE THE DATA IN DATABASE
    }
)

So, this was it for using multer with AWS and v3 SDK. I hope you find it helpful.

✌️

Node.js Morgan Guide

Paras 🧙‍♂️ — Sat, 05 Jun 2021 16:26:54 +0000

Any server or application needs some kind of logging system. It is common to skip logging in personal or practice project. But it is a must to have a fairly robust logging system in production ready servers and apps.

Benefits of a logging system:

Logs are used by product managers and UX designers for planning & design.
It helps marketers track performance of various features related to advertising campaigns.
It gives us the idea of how much traffic is the application getting.
Track errors, attacks and unsuccessful requests in production.

Getting started with Morgan

Morgan is a logging package for node.js. It can generate log files for all the traffic that comes to your server. It has some cool tricks to help us in both development and production. So let's get started.

Install Morgan

$ npm install morgan --save

Basic Morgan Usage

Adding morgan to middlewares in express is enough to get you started.

const express = require('express');
const morgan = require('morgan'); // import morgan

const app = express();

// setup morgan
app.use(morgan("dev"));

app.listen(3000, () => {
    console.debug('App listening on :3000');
});

Above setup will start logging requests in console. You can see, I have specified dev in morgan setup. There are many presets available in morgan and one them is dev. Some other presets are:

tiny
short
dev
common : Follows Apache like log format

Logging requests to file

Now we will learn how to output logs to a file. Instead of creating simple logging to file, we will create rotating logs. Rotating logs are logs that covers a part of activity on the server in a file and then creates a new log file. This way we can keep track of past activities. We don't have to keep one single log file open for continuous logging. So, let's setup this.

For rotating logs, we will install a package named rotating-file-stream. It will help us automate log rotation.

$ npm install rotating-file-stream --save

Setup Morgan

const express = require('express');
const morgan = require('morgan'); // import morgan
const rfs = require("rotating-file-stream");

const app = express();

// MORGAN SETUP
// create a log stream
const rfsStream = rfs.createStream("log.txt", {
   size: '10M', // rotate every 10 MegaBytes written
   interval: '1d', // rotate daily
   compress: 'gzip' // compress rotated files
})

// add log stream to morgan to save logs in file
app.use(morgan("dev", {
   stream: rfsStream
}));

// another logger to show logs in console as well
app.use(morgan("dev"));

app.listen(3000, () => {
   console.debug('App listening on :3000');
});

That was enough to setup our logging system but we can add environment variables to make it more intuitive to enable and disable logging to file

Adding environment variables to our setup

You will need dotenv package to load environment variables in nodejs from .env file

$ npm install dotenv --save

Now create a .env file in the root directory of project and add the following variables.

LOG_FILE=log.txt
LOG_FORMAT=common
LOG_SIZE=10M
LOG_INTERVAL=1d

Let's change our configuration to use these variables

require('dotenv').config() // load variables from .env file
const express = require('express');
const morgan = require('morgan'); // import morgan
const rfs = require("rotating-file-stream");

const app = express();

// MORGAN SETUP
// create a log stream
const rfsStream = rfs.createStream(process.env.LOG_FILE || 'log.txt', {
   size: process.env.LOG_SIZE || '10M',
   interval: process.env.LOG_INTERVAL || '1d',
   compress: 'gzip' // compress rotated files
});

// if log file defined then use rfs stream else print to console
app.use(morgan(process.env.LOG_FORMAT || "dev", {
   stream: process.env.LOG_FILE ? rfsStream : process.stdout
}));

// if log file is defined then also show logs in console
// else it will use the previous process.stdout to print to console
if(process.env.LOG_FILE) {
   app.use(morgan(process.env.LOG_FORMAT || "dev"));    
}

app.listen(3000, () => {
   console.debug('App listening on :3000');
});

With above configuration we can relax and let morgan handle the job of logging. Whenever an error occurs, visit the log file and you can track where things went wrong.

I hope this article helped you understand why it important to have a good logging system and how to add one in express.

Learn MongoDB: Working with Indexes

Paras 🧙‍♂️ — Tue, 02 Feb 2021 15:30:07 +0000

In mongodb, we can create indexes on fields so that when we query data using that field, we can get results instantly. It is like accessing a value from an array using index. Without index, we have to look for the data in linear way, which is not optimal in large datasets. So, we will learn about indexes in this post.

Let's begin !!

Table of Content

What is indexing ?
How indexing works ?
Working with Indexes
Creating Index
Get Indexes
Check Your Index
Drop Index

What is indexing ?

Indexing is a way to optimize the performance of accessing data from database. It allows us to efficiently fetch data from our database without having to traverse it as a whole. You can think of it like arrays or maps where we use index or key to get our data without having to search for it in a linear way.

How indexing works ?

Indexing, in background, keeps a table of key-value pair i.e. a table with two columns. First column has the key, which is actually the value that we want to search for in the table. For example: if we create an index on name field, then first column will have all the names in the database. While the second column keeps track of the pointers to the actual data. With the help of pointers, we fetch the actual data from our database.

Search Key	Database References (Values)
Arthur	reference_key_1
Ash	reference_key_2
John	reference_key_3

One good property of index table is keeping search keys in sorted order. In our case, search keys are stored in alphabetical order. This helps us find data much quickly (if you know binary search, then you know why sorting is helpful).

Also, if you are confused about what are these reference keys, then in this example we can say that these are the primary keys (in mongodb "_id" keys). They refer to the user in our database. These keys are already indexed, and data can be retrieved quickly using these "id" keys.

Working with Indexes

For working with indexes, we have a few cool methods:

createIndex
dropIndex
getIndexes
explain (to check how your queries are performing)

Before starting let's see a simple schema based on pokemon trainers. We will add fields according to our needs.

{
   name: "Ash",
   age: 18,
   pokemons: [
      { name: "pikachu", type: "electric", level: 16 },
      { name: "charizard", type: "fire", level: 30 },
      { name: "squirtle", type: "water", level: 12 }
   ],
   badges: ["boulder badge", "cascade badge"],
   exp: 200,
   bagItems: {
      pokeballs: 8,
      heals: 10
   },
}

Creating Index

Let's create an index on name field:

> db.trainers.createIndex({age: 1}) // 1 ascending & -1 descending

// index a nested field
> db.trainers.createIndex({"bagItems.pokeballs": 1})

Get Indexes

> db.trainers.getIndexes()

// _id is the default index

Check your index

We will use explain method to check if our queries are using the index or not.

> db.trainers.find({age: 18}).explain()

// a bit more specific
> db.trainers.find({age: 18}).explain("allPlansExecution")

If you apply the explain method, you will get a big object with so many details, which can be overwhelming. But we will focus on main fields.

Main fields are:

inputStage: In this key, our main field is winningPlan. It can be COLLSCAN (Full Collection Scan) or IXSCAN (Index Scan)
executionTimeMillis: It shows the time taken by your query to get the data. This is important because we will use this field to compare different indexes and queries execution time.
totalDocsExamined: This field shows how many docs mongo has to check before returning results. The lower the number, the better.

You can see there is input stage in explain result. This is first stage where mongo gets the keys and pointers from index table and then use those pointers to fetch actual docs.

Drop Index

Deleting index is also very easy. Just specify the object with that index.

// let's drop our age index
> db.trainers.dropIndex({age: 1})

// you can also specify index "name" that you can get from "getIndexes" method
> db.trainers.dropIndex("age_1")

You should know that these are just the basics. We have a lot to cover like text index, compound index, multi-key index, more about execution and plan rejection, etc. etc. So, we learn these things in next post !

Next Post : Coming soon...!!

Prev Post : Delete Documents