DEV Community: Shreyansh sheth

System Design: Google Drive That Scales To The Moon

Shreyansh sheth — Mon, 08 May 2023 19:38:02 +0000

Introduction

So a few months ago I was building a new internal project that helps you upload files like google drive but also works as a CDN so you can use those assets on the front end. So here is the few lessons that I have learned along the way.

Requirement

The project requirement is simply to create a multi-tenant app that will let you upload and download files in the workspace, ability to make the file public, and also make a file available through a single URL with the cache.
To get file URL will be something like /:workspaceid/:key

Technology Choices

Storage
For Storage there are two options mainly, S3 & R2 but I have selected S3 because it's what I have used and it is a more mature solution with most of the issues and solutions online.
Functions
I have used the lambda function for it. because it works with dynamo DB that I am selecting for file metadata
File Metadata
for file metadata, we are storing the mime type, size, and original name of the file along with indexes that we will talk about later in the blog.
Auth
For Authentication, We Used a simple cookie JWT-based system, and that won't matter much in the blog context.

NOTE: front-end was on react(vite).

Let's Start Designing

What We Need?

to upload a file
to delete the file
make the file public
make the file available via cdn with caching
in ui workspace users can see all their files

Database Design

For file-metadata database design is very simple as it contains query keys as file key and workspaceId as sort and partition keys.

the reason that workspace-id is the partition key, is that will help make DynamoDB partition for each workspace and make it more efficient while we query for the list in frontend.

and file key is the sort key that just helps to make sorting possible with ULID(Lexicographically Sortable Identifier) as it makes a list on UI based on when it is uploaded and also it is unique so it helps query single files easily.

For ULID see this blog

Chart That Represents The DB Design

As you can see this is the metadata table for the files that we are uploading and this contains all important fields that are required for some or other purpose.

System Design

For System Design, It Contains Some Part Related To AWSLambda, S3, CloudFront, Lambda@Edge

File Upload Service
Takes File Type, Size, and Mimetype, save it into a dynamo, and Returns POST upload URL that helps upload the file directly to s3 from frontend with some max-age.

File Delete CRON
This function is a CRON (scheduled) that will delete files from the s3 periodically from deleteAfter & isDeleted as it uses GSI Field from table and also from DynamoDB. the ideal time for us is 4 days, so if any recovery comes we can resolve it as soon as possible.

Get Private File
It Looks At AUTH Token That Represents The Workspace ID and returns a pre-sign URL get URL that can help you view the private file

Make File Public
If the File Is Public It Will Be Served From Cloudfront With Its Caching Layer Also I Have Added Some Image Resizer As A Bonus Part There.

Here Is The Basic System Design Chart That Helps You Get a Top View Of The Things

The End

If You Like This Article, Please Consider Following Me On LinkedIn And If You Have Some Cool Project Like This Just Contact Me, I Would Love To Work With Awesome People And Projects.
Linkdin

Let's Build Express Server. Beginners Guide

Shreyansh sheth — Mon, 15 Nov 2021 09:53:52 +0000

What Is Express Js?

Express Js Is Web Framework For Node.js

Where To Use Express Js

Most Popular Use Of Express Js In the Current Time Is To Create Web Apis.
Also Some People Are Using To Create Full-stack Web App With using Pug, Ejs, Or Basic HTML

here we are going to create a simple API to understand concepts of express

Getting Started

Requirements

You need Node.js Installed In Your System. You can download node.js from Node.js
Open New Directory And Write This command to initialize a new project.
npm init -y

-Now install express as dependencies for your project
npm install express

Create basic hello world API

Create index.js file and import express js & initialize the app

Create an endpoint

in simple term endpoint is a path where you are getting expected data
/users return all users
/products returns all products

here we have created endpoint / where we are sending hello world as a response.

3 Open a port for connections

Run application > write node index.js to start the application

open localhost:3000 on browser

hooray you made it 🥳

what now

now try to render Html. try to create forms & process data using any database.

find material online and create your own express apps.

Basic Guide Of tailwindcss

Shreyansh sheth — Mon, 25 Oct 2021 18:33:32 +0000

What is Tailwindcss ?

tailwindcss is a utility-first CSS framework for rapidly building custom user interfaces.

Basically, all the styles utility are defined in class names. and by combining them you can create beautiful components. just like how you use bootstrap.

tailwindcss docs

Example

make padding top of div 0

<div class="pt-0">Hey Readers</div>

Here as you can see pt-0 as padding-top should be 0. Like that, we can also do p-0 for all sides & for margin, we can do m-0

change button color from blue to red when hover

<button class="bg-blue-900 hover:bg-red-900">Click Here!</button>

In tailwindcss there are different states like hover focus etc. and you can define property after them like STATE:CLASS to work in an intended way.

for diffrent screen sizes

<img class="w-16 md:w-32 lg:w-48" src="...">

Here we define that for lg and further width will be 48, from md to lg screen the width will be 32 and base width will be 16.

you can read about what exactly screen sizes are Here

where tailwind shines?

In tailwind you are repeating yourself. so it really shines when you are working with component-based framework like react ,angular , vue etc.

disclamer

If you want to use tailwind I recommend looking into tailwind docs because there are lots of features that might help you.

Why You Should Use Typescript Over Javascript?

Shreyansh sheth — Thu, 14 Oct 2021 17:16:19 +0000

What Typescript Actually Is?

Typescript is a strongly typed programming language like Java or C and typescript compiles into javascript so that we can use it in our browser or other platforms.

Reasons To Use Typescript Over Javascript

Type Safety

When we do dynamic assigning in javascript it allows us to do that and also it will give us expected results. but if you are working on a large codebase you are not going to track each variable and ensure the types of each of them
Better Auto-Complete

Typescript provides a complete type system so that ide/text editors eg.vs code can give you better autocompletion on your whole project and increase your productivity.
Conversion From Javascript To Typescript

Every valid javascript is a valid typescript. so you can start by converting your file to typescript files and set the rules of typescript low and make it strict as time goes on

Where To Start

Tyescript Website And Doc
Freecodecamp Video For Typescript

conclusion

Typescript is better tool to use when you are comfortable with javascript and need more control over your codebase and want to catch silly mistake on compile-time rather than runtime.

Node.js & Deno Comparison

Shreyansh sheth — Tue, 05 Oct 2021 20:55:53 +0000

So what is Deno?

Deno is JavaScript , typescript runtime like node.js but it can run typescript natively and has so many new features related to security and module delivery.

Why compare to node.js?

At it’s core Deno is same as node and they both uses V8 engine for javascript execution and also their creator Ryan Dhal is same but they are completely different how you manage your dependencies and also files.

Why You Should Use.

Why you should use Deno?

Deno has future because of it’s growing community and also it has so many features that node is lacking and also it is so much secure and better run time than node.

Why you should use node?

Although Deno is future proof but node has so much bigger community than any other platform and also if you find yourself stuck you can easily find solution.

Things that are changed with Deno.

Have to give permission for accessing files read and write , net , hrtime , etc.
Importing modules is completely change and now you can import from url and file sitting on CDN
No-Package Manager / No-Modules Folder. All the modules fetch from URL cache under Deno and you don’t need to worry about that
Don’t worry about different setup for typescript it’s just works out of the box.
Deno has different built in APIs than node because all of async calls end up in some sort of promise.

Conclusion

Deno is the future, but node is current the king for JavaScript development thanks to massive size of community and millions of open source package.

Diffrent Ways To Create A React Application

Shreyansh sheth — Tue, 05 Oct 2021 20:50:47 +0000

1. Create-React-App

Provide a Simple Environment With All The Things Already Setup For You And You Just Have To Write Code.

Create React App doesn’t handle backend logic or databases; it just creates a frontend build pipeline, so you can use it with any backend you want. Under the hood, it uses Babel and webpack, but you don’t need to know anything about them.

CRA Docs

create new app with create-react-app

npx create-react-app appName

2. Vite

Same As Create-React-App It Provide All Configuration Out Of The Box. Also, It Supports SSR & SSG.

Vite Docs

To Create Vite App

npm init vite@latest

3. Next.js

Currently The Best Tool To Create SSR(server-site-rendered) & SSG(static-site-generation) For React. It Has Inbuilt Optimization That Makes Your App Run So Much Faster & Also If You Want To Add Serverless Function Next.js Also Provides That.

Next.js Docs

To Create Next.js Application

npx create-next-app

4. Gatsby

Gatsby Does SSG And It Has Huge Plugin System To Support Gatsby Compile Your React Code To HTML-CSS So User Don't Have To Wait For Javascript And React To Load
Gatsby Docs

To Create Gatsby App

npm init gatsby

5. Razzle

It Is Serverside Rendering Framework As Well And This Provide Much More Flexibility.
Razzle Docs

To Create Razzle App

npm install -g create-razzle-app

6. `<Script>` Tags

You Can Use Script Tags For React And After Importing That Your Code May Look Like This.

const e = React.createElement;
return e(
  'button',
  { onClick: () => this.setState({ liked: true }) },
  'Like'
);

(React Docs)(https://reactjs.org/docs/add-react-to-a-website.html)

I Covered Major Ways To Build React App But You Can Setup Your Own System Like These.

React.js : beginner guide, Where To Start

Shreyansh sheth — Fri, 01 Oct 2021 09:03:33 +0000

What Is React
How It Works
Is It Suitable For You
Roadmap To Learn React

What Is React

React Is A Javascript Library To Build User Interfaces. Think Of It As Simple Web Library That You Can Inject Through <script> Tag.

Main Features Of React That Attracts Peoples

It Has Huge Community
Library Is Declarative And Easy-To-Understand
It Is Component Base
Back By Facebook And Used In Many Morden Web Applications
Different Production Ready Frameworks Build upon It
You Can Use JSX To Write Components And It Makes It So Much Easy Rather Than Using Templates

How It Works

It Uses Virtual Dom To Handle State And All Its Component And React Based On State Changes. And Updating Virtual Dom Update Browser Dom.

React.createElement(
  'h1',
  {className: 'heading'},
  "hey reader"
)

React Is Just Library And If You Want To Work With Its True Potential You Have To Use Bable To Transpile All JSX into browser-javascript

Above Component In Jsx is littraly

<h1 className='heading'>hey reader</h1>

But You Don't Have To Worry About This Transpiling Things Because All The Framework Build On Top Of React Just Do That Automatically.

Is It Suitable For You

For Most Cases React Is Completely Suitable To Work With Because It Provides 1000s' Of Open-Sources Community Library That Is Help To Solve So Many Problems.

But, It Is Recommended To Just See What You Want To Achieve And It Is Too Overkill To Use React In That Scenario.

Roadmap To Learn React

Overview Of React

Fireship 100 Seconds Of React

Learning Material For React

I Am Recommending This Because I Have Used It For Myself Also

List Of Things To Learn

Follow List To Get Better Understanding
I Have Bias In Some Library Than Other Just Because I Have Learned It And My Experience With That Libraries

Starter Project (Vite.js, Create-React-App)
How JSX & Babel Works With Bundlers
React Functional Components (You Can Use Class Components But Functional Is Easy To Use & Learn)
Lifecycle In Functional Components
UseState Hook In Functional Component
Props In React
Build Todo Application With It With Each Todo item is its own component. Use State & lifecycle Hooks Along With Props.
Learn UseContext Hook & Improve Above Todo App With This Hook
Learn UseMemo, UseReducer Hooks
Learn About React-Router Or Reach Router To Create Different Pages And Make Use Of Them In Above Application
Learn Different CSS & Components Library To Style Above Application.
Learn About State Management Like Recoil & Redux ("Redux Is Too Complicated And Takes Lots Of Time And Also You Probably Don't Need Redux So Be Careful & Select One You Like")
Learn About Single Page App, Static Site Generation And Server-Side Rendering
Use Gatsby Or Next.js For Simple Project And Learn About Them.
Use Typescript With React.
Learn About One Last Thing That Is Datafetching And Error Handling
- use ReactQuery Or Swr For Datafetching And Caching

Feel Free To Comment About Your Opinions & Improvement In This List And Blog