DEV Community: Kumar Kalyan

Building a 🧾 PDF Invoice Generator with 📨 Twilio SendGrid, 🐍 Python, Flask, and 🖌️ Bootstrap

Kumar Kalyan — Sat, 07 Dec 2024 09:04:42 +0000

Introduction 🌟

In the current digital era, virtually all online purchases, whether for goods or services, include an invoice. This document helps businesses retain records and customers keep track of their transactions in addition to serving as proof of purchase. Automating the creation of invoices can streamline processes for developers and enhance client satisfaction. In this post you will learn how to use Twilio SendGrid, Python ,Flask and Bootstrap to create a straightforward yet effective invoice generation application.

Tutorial Requirements 🛠️📋

Before you start, there are a few prerequisites for this tutorial that need to be addressed.

A free Twilio Sendgrid account. If you are new to Twilio Sendgrid, signup for a free account. Using free account you can send 100 emails per day
Python 3.10 or latest version . If your machine does not have this version or latest python version you can download it from here
ReportLab Toolkit. It is a free and open source python library for generating PDFs and graphics
Bootstrap version 4.6. It is a free and open-source framework for building responsive websites. You will use it for building the invoice UI

Configure Your Twilio SendGrid Account 🔧📤

Before you start building your app, you will need a Twilio SendGrid API key to use the Twilio SendGrid API and send emails using your Flask application.

To get an API key, first log in to your Twilio SendGrid account using your credentials. Then, on the left side, navigate to Settings, select API Keys, and click on Create API Key.

Now, add your preferred API key name (in my case, I am using Flask-invoice-new, then choose Full Access under API Key Permissions, and finally, click on Create & View.

Now, you will see that your API key has been generated. Copy the key and store it in a secure place, as Twilio SendGrid will only allow you to view the key once for security reasons.

Now that your API key has been generated successfully, you need to verify your email with Twilio SendGrid, which you will use to send emails.

Navigate to Settings, select Sender Authentication, then click on the Verify a Single Sender button. Next, click on Create New Sender, fill in the required details, and click Create.

Twilio SendGrid will now send a verification email to this address with a link. Once you click on the link and complete the verification, you’re good to go.

Configure your Python Virtual Environment 🌟🐍

Now that you have generated your API keys in the previous step, you need to set up your Python virtual environment to create a Flask application and send emails using the Twilio SendGrid API.

Create a directory for your app and set up the Python virtual environment using the commands provided below. (For this tutorial, I am using the directory name 'Twilio-Sendgrid-Flask-invoice-app').

For unix or mac users

$ mkdir Twilio-Sendgrid-Flask-invoice-app
$ cd Twilio-Sendgrid-Flask-invoice-app
$ python3 -m venv venv
$source venv/bin/activate
(venv) $ _

For Windows users

mkdir Twilio-Sendgrid-Flask-invoice-app
cd Twilio-Sendgrid-Flask-invoice-app
python3 -m venv venv
venv\Scripts\activate

Once you have configured your Python virtual environment, install the required Python dependencies using the command below

pip install flask sendgrid reportlab python-dotenv

Create your Flask application 🐍🧾📧

Create a file named “app.py” in the root directory and add the starter code for creating a Flask application

import os
from flask import Flask, render_template, request, send_file
from reportlab.lib.pagesizes import letter
from reportlab.platypus import Table, TableStyle, Paragraph, SimpleDocTemplate
from reportlab.lib import colors
from reportlab.lib.units import inch
from reportlab.lib.styles import getSampleStyleSheet
from io import BytesIO
import base64
from datetime import datetime
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail, Attachment, Disposition, FileContent, FileName, FileType
app = Flask(__name__)
API_KEY=os.environ.get('TWILIO_SENDGRID_API_KEY')
EMAIL_ADDRESS = os.environ.get('MY_VERIFIED_EMAIL_ADDRESS')

In the above code I have imported all the necessary modules required to create a Flask application and build the invoice generator app. For security reasons, I am using environment variables to store my Twilio SendGrid API key and my email address, which will be used to send PDF invoices to clients' email addresses

Now, create a file named '.env' in the root directory and store your Twilio SendGrid API key and your verified email address. For this tutorial, I am using the variable names TWILIO_SENDGRID_API_KEY and MY_VERIFIED_EMAIL_ADDRESS. Flask will automatically import these environment variables using the Python dotenv package.

TWILIO_SENDGRID_API_KEY="YOUR TWILIO SENDGRID API KEY "
MY_VERIFIED_EMAIL_ADDRESS="YOUR EMAIL ADDRESS"

Create the home page for your Flask application 🏠💻

In Flask, you can render dynamic HTML content by combining HTML templates and data, a process called template rendering. This means that when you open a specific route in your Flask application, you can render a predefined HTML file there. For this tutorial, you will create an HTML file for the invoice UI so that whenever anyone opens the home route of your Flask application, they will see a beautiful invoice UI where they can enter invoice data.

Now, create a folder named templates in the root directory and add an index.html file inside it. In Flask you can render any HTML file present within this folder

Building the invoice form using bootstrap 🧾✨

Inside the index.html file paste the code below

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Invoice Generator</title>
    <!-- Bootstrap CSS -->
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@4.6.2/dist/css/bootstrap.min.css"
        integrity="sha384-xOolHFLEh07PJGoPkLv1IbcEPTNtaed2xpHsD9ESMhqIYd0nLMwNLD69Npy4HI+N" crossorigin="anonymous">
</head>

<body class="bg-light">
    <div class="container mt-5">
        <div class="card">
            <div class="card-header bg-primary text-white">
                <h3 class="mb-0">Invoice Generator</h3>
            </div>
            <div class="card-body">
                <form id="invoice-form" action="/generate-invoice" method="POST" onsubmit="alertSubmit()">
                    <div class="form-group">
                        <label for="client_name">Client Name:</label>
                        <input type="text" class="form-control" id="client_name" name="client_name" required>
                    </div>
                    <div class="form-group">
                        <label for="email">Client Email:</label>
                        <input type="email" class="form-control" id="email" name="email" required>
                    </div>

                    <h4 class="mt-4">Invoice Items</h4>
                    <div id="items">
                        <div class="form-row align-items-center mb-3 item">
                            <div class="col-md-4">
                                <label>Description</label>
                                <input type="text" class="form-control" name="description[]" required>
                            </div>
                            <div class="col-md-2">
                                <label>Quantity</label>
                                <input type="number" class="form-control" name="quantity[]" oninput="calculateTotal()"
                                    required>
                            </div>
                            <div class="col-md-2">
                                <label>Price</label>
                                <input type="number" step="0.01" class="form-control" name="price[]"
                                    oninput="calculateTotal()" required>
                            </div>
                            <div class="col-md-2">
                                <button type="button" class="btn btn-danger btn-block mt-4"
                                    onclick="removeItem(this)">Remove</button>
                            </div>
                        </div>
                    </div>
                    <button type="button" class="btn btn-secondary" onclick="addItem()">Add Another Item</button>

                    <div class="form-group mt-4">
                        <label for="total_amount">Total Amount:</label>
                        <input type="number" step="0.01" class="form-control" id="total_amount" name="amount" readonly>
                    </div>

                    <button type="submit" class="btn btn-primary btn-block">Generate and Send Invoice</button>
                    <button onclick="resetForm()" class="btn btn-danger btn-block">Reset Invoice Form </button>
                </form>
            </div>
        </div>
    </div>


</body>
</html>

For this tutorial, I am using Bootstrap version 4.5, so make sure to add the correct CDN link for this Bootstrap version; otherwise, some features may not work properly.

In the code above, I have created a form with a simple UI that submits data to the /generate-invoice route. As of now, it is just a basic HTML form without any actions, as I haven't added functionalities to the buttons yet and for that JavaScript is needed

Building the functionalities for buttons 🔘⚙️

To add functionalities to the buttons, include additional invoice fields, and calculate the total invoice amount, you will need to use JavaScript. Paste the JavaScript code just above the ending <div> tag to add functionalities to the form.

  <script>
        function alertSubmit() {
            alert("Email Sent Successfully");
        }
        function resetForm() {
            document.getElementById("invoice-form").reset();
        }

        // Function to add a new item row
        function addItem() {
            const itemDiv = document.createElement('div');
            itemDiv.className = 'form-row align-items-center mb-3 item';
            itemDiv.innerHTML = `
                <div class="col-md-4">
                    <label>Description</label>
                    <input type="text" class="form-control" name="description" required>
                </div>
                <div class="col-md-2">
                    <label>Quantity</label>
                    <input type="number" class="form-control" name="quantity" oninput="calculateTotal()" required>
                </div>
                <div class="col-md-2">
                    <label>Price</label>
                    <input type="number" step="0.01" class="form-control" name="price" oninput="calculateTotal()" required>
                </div>
                <div class="col-md-2">
                    <button type="button" class="btn btn-danger btn-block mt-4" onclick="removeItem(this)">Remove</button>
                </div>
            `;
            document.getElementById('items').appendChild(itemDiv);
            calculateTotal();
        }

        // Function to calculate total amount
        function calculateTotal() {
            let total = 0;
            const quantities = document.querySelectorAll('input[name="quantity"]');
            const prices = document.querySelectorAll('input[name="price"]');

            quantities.forEach((quantity, index) => {
                const price = prices[index];
                const qty = parseFloat(quantity.value) || 0;
                const prc = parseFloat(price.value) || 0;
                total += qty * prc;
            });

            // Update the total amount field
            document.getElementById('total_amount').value = total.toFixed(2);
        }

        // Function to remove an item row
        function removeItem(button) {
            const itemDiv = button.parentNode.parentNode;
            document.getElementById('items').removeChild(itemDiv);
            calculateTotal();
        }
    </script>

In the above code

alertSubmit() - Displays an alert confirming that the email was sent successfully.
resetForm() - Resets the values in the invoice form.
addItem() - Adds new form fields to include more items in the invoice form.
calculateTotal() - Automatically updates the Total Amount field by calculating the total whenever a new item is added to the invoice.
removeItem() - Removes a specific item from the invoice form.

Write the Python code that would generate invoices in PDF format and send emails.

Once your UI for the invoice generator app is ready, it’s time to write Python functions for the backend.

In this part of the tutorial, you will create the /generate-invoice route to handle the POST request and the send_invoice_via_email() function to send emails using the free Twilio SendGrid API.

Let’s code the genenate_invoice function 💻🧾

Open the app.py file and write a function to fetch data from the form and generate PDF invoices.


@app.route('/generate-invoice', methods=['POST'])
def generate_invoice():
    data = request.form
    client_name = data.get('client_name')
    email = data.get('email')
    total_amount = data.get('amount')

    # Retrieve item details
    descriptions = data.getlist('description')
    quantities = data.getlist('quantity')
    prices = data.getlist('price')

    # Generate PDF using ReportLab
    pdf_buffer = BytesIO()
    pdf = SimpleDocTemplate(pdf_buffer, pagesize=letter)

    # Styles and layout
    styles = getSampleStyleSheet() 
    title_style = styles["Title"]
    normal_style = styles["Normal"]

    elements = []

    # Invoice Title
    elements.append(Paragraph("INVOICE", title_style))

    # Client and date details
    elements.append(Paragraph(f"Date: {datetime.now().strftime('%Y-%m-%d')}", normal_style))
    elements.append(Paragraph(f"Client: {client_name}", normal_style))
    elements.append(Paragraph(" ", normal_style))  # Add space before the table

    # Table for invoice items
    items = [["Description", "Quantity", "Price", "Total"]]
    for description, quantity, price in zip(descriptions, quantities, prices):
        total = float(quantity) * float(price)
        items.append([description, quantity, f"${float(price):.2f}", f"${total:.2f}"])

    # Adding the Total row with only two columns
    items.append(["", "", "Total", f"${float(total_amount):.2f}"])

    # Table style
    table = Table(items, colWidths=[3 * inch, inch, inch, inch])
    table.setStyle(TableStyle([
        ('BACKGROUND', (0, 0), (-1, 0), colors.grey),
        ('TEXTCOLOR', (0, 0), (-1, 0), colors.whitesmoke),
        ('ALIGN', (0, 0), (-1, -1), 'CENTER'),
        ('FONTNAME', (0, 0), (-1, 0), 'Helvetica-Bold'),
        ('BOTTOMPADDING', (0, 0), (-1, 0), 12),
        ('BACKGROUND', (0, 1), (-1, -2), colors.beige),
        ('GRID', (0, 0), (-1, -1), 1, colors.black),
        ('SPAN', (0, -1), (2, -1)),  # Span the first three cells in the last row
        ('ALIGN', (2, -1), (3, -1), 'CENTER'),
        ('FONTNAME', (2, -1), (3, -1), 'Helvetica-Bold'),
    ]))

    elements.append(table)

    # Adding space and final message
    elements.append(Paragraph(" ", normal_style))  # Add space after the table
    elements.append(Paragraph("Thank you for your business!", title_style))

    # Build PDF
    pdf.build(elements)

    # Rewind the buffer
    pdf_buffer.seek(0)

    # Send the PDF via email
    try:
        if email:
            if(send_invoice_via_email(pdf_buffer, email)):
                return send_file(pdf_buffer, as_attachment=True, download_name="invoice.pdf", mimetype='application/pdf')
    except Exception as e:
        return f"Error sending email: {e}"

In the above code the important points are as follows:

I created a new route, /generate-invoice, of type POST, meaning this route will only handle POST requests.
I am using the request.form method in Flask to process data coming from the invoice form and store the values in respective variables.
I am using the getlist() method to retrieve multiple data entries using a single key. For this tutorial, there can be multiple invoice items with the same key. Using getlist() allows me to store item descriptions, quantities, and prices in separate lists.
I am using the BytesIO object from Python's io module and the SimpleDocTemplate function from ReportLab to set up a PDF template for the invoice. Note that this BytesIO class will be used to write invoice data into the PDF file.
I am using getSampleStyleSheet() from ReportLab to add some styles to the invoice data.
I am using Python lists and zip to organize the invoice items and calculate the net total amount for each item by multiplying the quantity by the price to get the final amount.
I am using the Table object from ReportLab to style the invoice table. Once all data processing and arrangement are complete, I use the append method to add each invoice element to the item list, and the build() method from ReportLab to generate the invoice as a PDF document.
I am using seek() to reset the file handler pointer to 0.
Finally, I check if an email address is provided and, if so, whether the email is successfully sent using the send_invoice_via_email(pdf_buffer, email) function. Once all conditions are met, I use the send_file() function from Flask to send the invoice PDF for download. All these steps are wrapped in try and except blocks for exception handling.

Let’s code the send_invoice_via_email function 📧✨

As of now, the /generate-invoice route is set up but not yet functional. This means that if you try running your Flask application now, it will show an error. The reason for this is that you still need to add the send_invoice_via_email() function to handle sending emails. So, let’s create this function.

def send_invoice_via_email(pdf_buffer, recipient_email):
    # Convert PDF to base64 for email attachment
    pdf_data = base64.b64encode(pdf_buffer.getvalue()).decode()

    # Create SendGrid Mail object
    message = Mail(
        from_email='kum9748ar@gmail.com',  # Replace with your email
        to_emails=recipient_email,
        subject='Your Invoice',
        html_content='<p>Please find attached your invoice.</p>'
    )

    # Create attachment for the email
    attachment = Attachment(
        FileContent(pdf_data),
        FileName("invoice.pdf"),
        FileType("application/pdf"),
        Disposition("attachment")
    )
    message.attachment = attachment

    # Send email using SendGrid
    try:
        sg = SendGridAPIClient(API_KEY)
        response = sg.send(message)
        print(f"Email sent! Status code: {response.status_code}")
        return True
    except Exception as e:
        print(f"Error sending email: {e}")
        return False

In the above function the important points are as follows:

The function send_invoice_via_email(pdf_buffer, recipient_email) takes two arguments: pdf_buffer, which is your generated invoice PDF, and recipient_email, which is the email address where the invoice will be sent to the client.
I am using the base64 module to convert the PDF into an email attachment.
I am using the Mail object from sendgrid.helpers.mail to add essential details required to send an email via SendGrid, and the Attachment object to include the generated PDF as an attachment with this email message.
Finally, I am using the SendGridAPIClient() object, which takes the SendGrid API key as an argument, to send the email. This portion of code is also wrapped within try and except blocks for exception handling.

Run your application 🚀💻

To run your Flask application, you need to add a final piece of code.

if __name__ == "__main__":
    app.run(debug=True)

The above code ensures that your app.py file will run as a script and not as a module when it is imported.

Now, open your terminal, navigate to your project directory (for this tutorial, it should be Twilio-Sendgrid-Flask-invoice-app), and use the command below to start your Flask application. Then, open your browser and go to http://127.0.0.1:5000, and you will see your fully functional invoice app running successfully

python app.py

The image above shows that my Flask application is working properly.
Now, let’s check the mailbox.

You can see that I have received an email via SendGrid containing my invoice attached as a PDF.

Some key points to remember for deployment 🌐📝

Now that your Flask invoice generator app is ready, and you’re excited to share it with others as a live app, keep these key points in mind:

Generate a requirements.txt file using pip install -r requirements.txt which contains a list of python packages required to run this app

The above image shows the requirements.txt file for the Twilio-Sendgrid-Flask-invoice-app

Make sure to add the .env file to your .gitignore file so that nobody else can see your API keys.
Keep in mind that if you are using a Twilio SendGrid trial account, you can only send 100 emails per day. If you need to send more, I recommend upgrading your plan, as they offer a very flexible pricing structure.
If you want to update the application UI, make sure to check out the documentation for Bootstrap version 4.6.2, which I have used for this tutorial. Otherwise, you may need to rewrite the entire UI code.

Conclusion 🎯✅

So far in this tutorial, you have learned how to send emails using Twilio SendGrid, create a fully functional Flask app, work with Python's io module, render static HTML files, create PDFs in Python, and use Bootstrap and I hope you enjoyed it!

Check out Twilio’s developer docs and blogs to learn how to build interesting apps using various Twilio products. They also offer a $15 trial credit to new users that you can use for free.

The entire code for this tutorial is hosted on GitLab, and you can check it using this link.

🔥 17 Best Free GitHub Repositories to Crack System Design Interviews 🛠️

Kumar Kalyan — Sat, 07 Dec 2024 06:44:47 +0000

📜 TL;DR ✂️

Ever imagined how big tech companies build and scale large-scale products? What skills do their engineers possess? How do they decide which framework to choose for a product, or how do they handle millions of terabytes of data?

The only answer to all these questions is System Design. Yes, you heard it right! System Design is the core skill required to design, manage, and build large-scale, scalable products. It involves defining the architecture, components, interfaces, modules, and data for a system to meet specific requirements.

There are no specific prerequisites for this article. Anyone, from senior developers to those who have just started their coding journey, can benefit from these resources. For beginners, the only expectation is that you have a basic understanding of computers and programming.

In this article, I will share the 17 best free Github repositories to learn and master system design. These resources will help you ace your technical interviews or apply the concepts to build and scale your own products.

Please make sure to give a star to these repositories as it's free to start them!

GitHub Repositories

GitHub is a code hosting platform where people can share, version, manage, test, and deploy their code. This wonderful platform uses Git versioning systems, which allow developers to maintain and version their code in a tree-like structure. However, many people think that GitHub is just a code hosting platform, unaware that it is also a hidden treasure trove of free and open-source knowledge. In this article, I will share the best free GitHub repositories to learn and master system design for free.

1. System Design Primer

The System Design Primer is a free and open-source GitHub repository designed to help you learn and practice all the key concepts of system design at no cost.
In this repository you will find topics like Performance vs scalability, Latency vs throughput, Availability vs consistency, CAP theorem, CP - consistency and partition tolerance, AP - availability and partition tolerance, Consistency patterns, Weak consistency, Eventual consistency, Strong consistency, Availability patterns, Fail-over, Replication, Availability in numbers, Domain name system, Content delivery network, Push CDNs, Pull CDNs, Load balancer, Active-passive, Active-active, Layer 4 load balancing, Layer 7 load balancing, Horizontal scaling, Reverse proxy (web server), Load balancer vs reverse proxy, Application layer, Microservices, Service discovery, Database, Relational database management system (RDBMS), Master-slave replication, Master-master replication, Federation, Sharding, Denormalization, SQL tuning, NoSQL, Key-value store, Document store, Wide column store, Graph Database, SQL or NoSQL, Cache, Client caching, CDN caching, Web server caching, Database caching, Application caching, Caching at the database query level, Caching at the object level, When to update the cache, Cache-aside, Write-through, Write-behind (write-back), Refresh-ahead, Asynchronism, Message queues, Task queues, Back pressure, Communication, Transmission control protocol (TCP), User datagram protocol (UDP), Remote procedure call (RPC), Representational state transfer (REST) & Security.
The repository has received 227k stars and has been forked over 47k times on GitHub.
It currently has 118+ contributors and is available in 10+ languages, including English, Japanese, French, German, and Bengali.
It is ideal for everyone from junior developers to software architects and CTOs.

Give this repository a ⭐

2.System Design Roadmap by Developer Roadmap

The Developer Roadmap is a free and open-source GitHub repository designed to help you learn and practice system design through a structured roadmap.
It also includes roadmaps for learning React, TypeScript, JavaScript, Python, and many other programming languages and frameworks.
The repository has 298k stars and has been forked over 39.2k times on GitHub.
It currently has 1,160 contributors and is available only in English.
It is ideal for everyone from junior developers to software architects and CTOs.

Give this repository a ⭐

3. System Design 1o1 by Byte Byte Go

The System Design 101 from Byte Byte Go is a free and open-source GitHub repository designed to help you learn and practice system design from scratch. Although this repository includes a portion of their paid System Design course, it can be considered a goldmine for beginner developers or junior software engineers. You can find important and beginner-friendly system design concepts such as software communication protocols, APIs, CI/CD, databases, software architecture patterns, and much more.
The repository has 64.6k stars and has been forked over 6.8k times on GitHub.
It currently has 15 contributors and is available only in English.
It is ideal for junior software engineers and newbie software developers.

Give this repository a ⭐

4. System Design Resources by Interview Ready

The System Design Resources from InterviewReady is a free and open-source GitHub repository designed to help you learn and practice system design from scratch. Although this repository includes a portion of InterviewReady’s paid and comprehensive System Design course, it can still be considered a valuable resource for beginner developers or junior software engineers.
In this repository, you will find important topics such as Video Processing, Cluster and Workflow Management, Intra-Service Messaging, Message Queue Antipattern, Service Mesh, Practical System Design, Distributed File Systems, Time Series Databases, Rate Limiting, In-Memory Databases (e.g., Redis), Network Protocols, Chess Engine Design, Subscription Management Systems, Google Docs, API Design, NoSQL Database Internals, NoSQL Database Algorithms, Database Replication, Containers and Docker, Capacity Estimation, Publisher-Subscriber Architecture, Event-Driven Architectures, Software Architectures, Microservices, Distributed Transaction Consistency Patterns, Load Balancing, Alerts and Anomaly Detection, Distributed Logging, Metrics and Text Search Engines, Single Point of Failure, Location-Based Services, Batch Processing, Real-Time Stream Processing, Caching, Distributed Consensus, Authorization, Content Delivery Networks, Testing Distributed Systems, and other system design resources.
The repository has 16k stars and has been forked over 2k times on GitHub. It currently has 12 contributors, is maintained by the InterviewReady team, and is available only in English.
It is ideal for newbie to mid-level software developers.

Give this repository a ⭐

5. Awesome Low Level Design by Ashish Pratap Singh

The Awesome Low-Level Design repository by Asish Pratap Singh( Previously Software engineer at Amazon, Adobe and Morgan Stanely.)is a free and open-source GitHub repository designed to help you learn and practice low-level design topics from scratch. This repository can be considered one of the most valuable resources for learning and mastering low-level design concepts for free, catering to everyone from newbie to professional developers and software architects.
This repository covers fundamental to advanced topics, including OOP (Object-Oriented Programming), important SDLC principles like SOLID, DRY, YAGNI, and KISS, Software Design Patterns such as Creational Patterns, Structural Patterns, and Behavioral Patterns, UML diagrams, and Low-Level Design interview questions, such as Parking Lot Design, Vending Machine Design, Publisher-Subscriber Design, and many more.
The repository has 8.2k stars and has been forked over 2.3k times on GitHub. It currently has 21 contributors, is maintained by the Asish Pratap Singh, and is available only in English.
It is ideal for newbie to professional software developers.

Give this repository a ⭐

6. System Design by Karanpratap Singh

System Design by Karan Pratap Singh is a free and open-source Github repository to learn and master Beginner friendly to advanced level System Design concepts for free. Karan is currently working as a cloud engineer at Apple and oftenly shares tutorials on topics like System Design and Software Development.
In this repository you will find topics like IP, OSI Model, TCP and UDP, Domain Name System (DNS), Load Balancing, Clustering, Caching, Content Delivery Network (CDN), Proxy, Availability, Scalability, Storage,Databases and DBMS, SQL databases, NoSQL databases, SQL vs NoSQL databases, Database Replication, Indexes, Normalization and Denormalization, ACID and BASE consistency models, CAP theorem, PACELC Theorem, Transactions, Distributed Transactions, Sharding, Consistent Hashing, Database Federation,N-tier architecture, Message Brokers, Message Queues, Publish-Subscribe, Enterprise Service Bus (ESB), Monoliths and Microservices, Event-Driven Architecture (EDA), Event Sourcing, Command and Query Responsibility Segregation (CQRS), API Gateway, REST, GraphQL, gRPC, Long polling, WebSockets, Server-Sent Events (SSE),Geohashing and Quadtrees, Circuit breaker, Rate Limiting, Service Discovery, SLA, SLO, SLI, Disaster recovery, Virtual Machines (VMs) and Containers, OAuth 2.0 and OpenID Connect (OIDC), Single Sign-On (SSO), SSL, TLS, mTLS.
The repository has 33k stars and has been forked over 3.6k times on GitHub.
It currently has 10 contributors, is maintained by*Karan Pratap Singh*, and is available only in English.
It is ideal for newbie to professional software developers.

Give this repository a ⭐

7. Low Level Design Primer

Low Level Design Primer is a free and open-source Github repository to brush up advanced Low Level System Design concepts for free. It is built and maintained by Prasad Gujar who is currently working as a Software engineer at Tala Payments team
In this repository you will find topics like System Design Book Recommendations, links to free , popular and authentic Youtube Channels for mastering System Design, System Design Interview questions and their solutions like Design an online hotel booking system, Design a Logistics System, Design Tic Tac Toe, Design a configuration management system, Design a Vending Machine, Design Maps Navigator Client for different transportation types (OOD), Design a Json Parser from scratch, Design Elevator, CricInfo/Cricbuzz,etc
The repository has 6.5k stars and has been forked over 2.3k times on GitHub.
It currently has 7 contributors, is maintained by Prasad Gujar, and is available only in English.
It is ideal for newbie to professional software developers.

Give this repository a ⭐

8. Low Level Design By Interview Ready

Low Level Design is a free and open-source Github repository to brush up from beginner to advanced Low Level System Design concepts for free. It is built and maintained by Interview Ready team which is one of the leading platforms for learning and mastermind System Design Interviews
In this repository you will find beginner level low level System Design topics like Cache, Event Bus, Rate Limiter, Service Orchestrator, Design Patterns, etc
The repository has 704 stars and has been forked over 163 times on GitHub.
It currently has no visible team, is maintained by the Interview Ready team , and is available only in English.
It is ideal for newbie to professional software developers.

Give this repository a ⭐

9. System Design Interview

System Design Interview is a free and open-source Github repository to level up beginner to advancedLevel System Design Interview questions for free.
In this repository you will find beginner level low level System Design topics like Object Oriented Design,System Design interview tips, Product roadmaps, Design Patterns, etc
The repository has 21.8k stars and has been forked over 5.1k times on GitHub.
It currently has 23 contributors , and is available only in English.
It is ideal for newbie to professional software developers.

Give this repository a ⭐

10. System Design By Shashank Khare

System Design By ShashanK Khare is a free and open-source Github repository to level up beginner to advancedLevel System Design Interview questions for free. It is build and maintained by ShashanK Khare who is one of the brilliant software engineers currently working for mangroup UK.
- In this repository you will find beginner level low level System Design topics like Cache, Event Bus, Rate Limiter, Service Orchestrator, Design Patterns, etc
- The repository has 21.8k stars and has been forked over 5.1k times on GitHub.
- It currently has 23 contributors , and is available only in English.
- It is ideal for newbie to professional software developers.

Give this repository a ⭐

11. Awesome System Design Resources

System Design By Ashish Pratap Singh is a free and open-source Github repository to level up beginner to advancedLevel System Design Interview & questions for free. It is built and maintained by Ashish Pratap Singh who is one of the most talented software engineers and previously worked at Amazon as a senior software engineer.
- In this repository you will find System Design Key Concepts topics like*, Scalability, Availability, CAP Theorem, ACID Transactions, Consistent Hashing, Rate Limiting, SPOF, Fault Tolerance, Consensus Algorithms, Gossip Protocol, Service Discovery, API Design, Disaster Recovery, Distributed Tracing, 🛠️ System Design Building Blocks like , Content Delivery Network (CDN), Proxy vs Reverse Proxy, Domain Name System (DNS), Caching, Caching Strategies, Distributed Caching, Load Balancing, Databases Types, SQL vs NoSQL, Database Indexes, Consistency Patterns, HeartBeats, Circuit Breaker, Idempotency, Database Scaling, Data Replication, Data Redundancy, Database Sharding, Database Architectures, Failover, Bloom Filters, Message Queues, WebSockets, Checksums, API Gateway, Microservices Guidelines, Distributed Locking, ⚖️ System Design Tradeoffs like , Top 15 Tradeoffs, Vertical vs Horizontal Scaling, Stateful vs Stateless Design, Batch vs Stream Processing, Strong vs Eventual Consistency, Read-Through vs Write-Through Cache, Push vs Pull Architecture, Long-polling vs WebSockets, REST vs RPC, Synchronous vs asynchronous communications, Latency vs Throughput, 🖇️ System Design Architectural Patterns like, Client-Server Architecture, Microservices Architecture, Serverless Architecture, Event-Driven Architecture, Peer-to-Peer (P2P) Architecture and many more*
- The repository has 18.7k stars and has been forked over 4.5k times on GitHub.
- It currently has 2 contributors , and is available only in English.
- It is ideal for newbie to professional software developers.

Give this repository a ⭐

12. Grokking System Design By Jeevan Kumar Raj

Grokking System Design By Jeevan Kumar Raj is a free and open source repository to power up beginner to advancedLevel System Design skills for free. It is built and maintained by Jeevan Kumar Raj who is an experienced HR manager with 14 years of experience
- In this repository you will find topics on Distributed System Design Basics like Key Characteristics, Load balancing, Caching, Sharding, Indexes, Proxies, Queues, Redundancy, SQL vs. NoSQL, CAP Theorem, Consistent Hashing, Client-Server Communication, System Designs, System Design examples like Short URL Service, Pastebin, Instagram, Dropbox, Twitter, YouTube, Twitter Search, Web Crawler, Facebook Newsfeed and many more.
- The repository has 5.3k stars and has been forked over 1.5k times on GitHub.
- It is currently maintained by Jeevan and is available only in English.
- It is ideal for newbie to professional software developers.
Give this repository a ⭐

13. System Design questions by Arpit Bhayani

System Design questions by Arpit Bhayani is a free and open source repository to boot up advanced level System Designs for free. It is built and maintained by Arpit Bhayani who is one of the most talented software architects in the industry having experience working in top notch companies like Google and Amazon and is the creator of Dice Db (a redis-compliant, reactive, scalable, highly-available, unified cache optimized for modern hardware.)
- In this repository you will find popular System Design interview questions like designing Design a Blogging Platform, Design Online Offline Indicator, Design Airline Check-in, Design SQL-backed KV Store, Design Slack's Real Time Communication - NEW, Design a Load Balancer, Design Synchronized Queue Consumers, Design an Image Service, Design a HashTag Service, Design OnePic, Design Photo Tagging, Design User Affinity, Design Newly Unread Message Indicator, Design a Distributed Cache, Design a Word Dictionary, Design a Superfast KV Store, Design S3, Design a Faster Superfast KV Store, Design a Video Processing Pipeline for Streaming Service, Design a Text-based Search Engine and many more.
- The repository has 2.1k stars and has been forked over 445 times on GitHub.
- It is currently maintained by Arpit and is available only in English.
- It is ideal for newbie to professional software developers.
- Arpit also provides masterclass System Design Topics so make sure to check out that too

Give this repository a ⭐

14. System Design and Architecture By Tian Pan

System Design and Architecture by Tian Pan is a free and open source repository to skill up beginner to advanced level System Designs Topics for free. It is built and maintained by Tain Pan who is one of the most brilliant and high-performing software engineers and engineering manager working in the San Francisco Bay Area for 10 years, previously working at Uber, Oracle, IoTeX, and Microsoft. If you are preparing for your FAANG interviews then this repository is a gold mine for you
- In this repository you will find the most popular and real world System Design interview questions like designing Instagram or Pinterest, Designing Uber, How Facebook Scale its Social Graph Store? TAO, How Netflix Serves Viewing Data?, How to design robust and predictable APIs with idempotency?, How to stream video over HTTP for mobile devices? HTTP Live Streaming (HLS), Designing a distributed logging system, Designing a URL shortener, Designing a KV store with external storage, Designing a distributed in-memory KV store or Memcached, Designing Facebook photo storage, Designing Stock Exchange, Designing Smart Notification of Stock Price Changes, Designing Square Cash or PayPal Money Transfer System, Designing payment webhook, Designing a metric system, Designing a recommendation system, Designing Airbnb or a hotel booking system, Lyft's Marketing Automation Platform -- Symphony, Designing typeahead search or autocomplete, Designing a Load Balancer or Dropbox Bandaid, Fraud Detection with Semi-supervised Learning, and System Design theoretical topics like Introduction to Architecture, How to scale a web service?, ACID vs BASE, Data Partition and Routing, Replica, Consistency, and CAP theorem, Load Balancer Types, Concurrency Model, Improving availability with failover, Bloom Filter, Skiplist, B tree vs. B+ tree, Intro to Relational Database, 4 Kinds of No-SQL, Key value cache, Stream and Batch Processing Frameworks, Cloud Design Patterns, Public API Choices, Lambda Architecture, iOS Architecture Patterns Revisited, What can we communicate in soft skills interview?, Experience Deep Dive, 3 Programming Paradigms, SOLID Design Principles
- The repository has 2.6k stars and has been forked over 587 times on GitHub.
- It is primarily maintained by Tina and is available only in English.
- It is ideal for newbie to professional software architects.
- Tian also writes blogs on popular System Design and engineering topics so make sure to check out that here
  
  Give this repository a ⭐

15. Complete System Design by Coder World

Complete System Design By Coder World is a free and open source repository to learn and master beginner to advanced level System Designs Topics for free. It is built and maintained by Naina Chaturvedi who is a Senior Software Engineer with past working experience in Goldman Sachs.
- In this repository you will find the most popular and real world System Design case studies like Complete Cheat Sheet for Tech Interviews - How to prepare efficiently, Mega Launch - 200+ System Design Case Studies, System Design Most Important Terms, System Design Template, Complete System Design Case Studies, How to solve any System Design Question (approach that you should take), ML System Design Case Studies Series, All tech system design case studies pulse -, How Lyft Handles 2x Traffic Spikes during Peak Hours with Auto scaling Infrastructure, Facebook's News Feed Algorithm Marvel: How it Serves 2.9 Billion Daily Active Users Using PyTorch and Cassandra, 8+ Billion Daily Views: How Facebook's Live Video Ranking Algorithm Processes Daily Views Using Storm and Memcache, 500+ Million Users Daily: The Tech Behind Instagram Stories, At Amazon How 310 Million Users Experience Lightning-Fast Load Times, How PayPal Manages Over 400 Million Active Accounts Seamlessly?, The Billion-Dollar Question - What's My ETA? How Uber Calculates ETA -, What happens Once You Press Play button on Netflix, How Netflix handles millions of memberships efficiently?, Impressive Tech Behind YouTube's Scaling to 3 Billion Users: Inside Vitess Database Clustering, How Microservices Work?, How Distributed Message Queues Work?, How to Efficiently Build Scalable Machine Learning Pipelines, FAANG level - How to Write Production Ready Code? and many more System Design topics.
- The repository has 4k stars and has been forked over 522 times on GitHub.
- It is primarily maintained by Naina and is available only in English.
- It is ideal for newbie to professional software architects.
- Niana also runs a Youtube channel sharing System Design and other Tech interview specific topics which you can check out here.
Give this repository a ⭐

16. System Design By Coders Guild

System Design By Coders Guild is a free and open source Github repository to practice System Design Topics and interview questions for free. It is built by Sumit Lahiri who is a talented senior full stack developer currently pursuing PHD in Computer Science from IIT Kanpur. Sumit also has previous experience of working as a software engineer at TCS and Golem Factory.
- In this repository you will find topics like MapReduce: Simplied Data Processing on Large Clusters, Bigtable: A Distributed Storage System for Structured Data, The Google File System, The Chubby lock service for loosely-coupled distributed systems, Dynamo: Amazon's Highly Available Key-value Store, Frontend in React, State Management using Redux, Angular Get Started, First DJango App, Build a CRUD Todo app with Django and React/Redux, The MERN Stack Tutorial – Building A React CRUD Application From Start To Finish, Introduction to Memcached, Cassandra Introduction Features, Introduction to HBase, Introduction to MongoDB, Introduction to Redis, Storm, Introduction to Zookeeper, Kafka, YouTube Architecture, Scaling Pinterest, Google Architecture, Scaling Twitter, The WhatsApp Architecture, Flickr Architecture, Amazon Architecture, Stack Overflow Architecture, Pinterest Architecture, Tumblr Architecture, Instagram Architecture, TripAdvisor Architecture, Scaling Mailbox, Salesforce Architecture, ESPN Architecture, Uber Architecture, DropBox Design, Splunk Architecture, Good Parts of AWS, Azure DataCenter Architecture, Evolution Of Search Engines Architecture - Highscalability, Trending and most asked System Design problems like Design a CDN network,Design a random ID generation system,Design a key-value database,Design the Facebook news feed function,Design the Facebook timeline function,Design a function to return the top k requests during past time interval,Design an online multiplayer card game,Design a picture sharing system and many more
- The repository has 2k stars and has been forked over 374 times on GitHub.
- It is primarily maintained by Sumit and is available only in English.
- It is ideal for newbie to professional software architects.

Give this repository a ⭐

17. Mobile System Design Interviews

System-design-interviews is a free and open source GitHub repository to practice and master practical System Design Skills for free
- In this repository you will find important System Design topics like High-Level Diagram, Server-side component like Backend,Push Provider,CDN (Content Delivery Network), client-side components like API Service, Persistence,Repository,Tweet Feed Flow,DI Graph, Image Loader, Signals,App Module , Graphql and many more mobile engineering topics
- The repository has 2.1k stars and has been forked over 670 times on GitHub.
- It is available only in English.
- It is ideal for newbie professional mobile focused software engineers .
Give this repository a ⭐

Conclusion

So far in this article, we have discussed 17 free GitHub repositories that will help you excel in your System Design interviews. Ensure you have a strong foundation in Computer Science fundamentals. If not, start again from the basics and take your time, as some System Design concepts can be complex and require patience to learn and master.

I recommend dedicating at least six months to your preparation. Create a routine and follow a structured roadmap. If you have questions, post them on public platforms like Stack Overflow or Reddit communities, or use ChatGPT for assistance.

Finally, make sure to follow my blog if you’re interested in reading more articles like this!

TOP 50 project ideas to become a golang hero

Kumar Kalyan — Tue, 28 May 2024 17:11:22 +0000

TLDR 🔥

Go, also known as Golang, is a statically typed, compiled programming language designed by Google. It has gained popularity due to its simplicity, performance, and strong support for concurrent programming. Go lang is used in cloud and network services , building CLI tools , Web development , Development Operations & Site Reliability Engineering (SRE) and block chain technology. Go lang developers are in high demand nowadays in the software development industry with an average salary ranging from $109k/yr to $265k/yr.

This article contains a list of beginner friendly to advance project ideas which you can build using golang and will definitely help you boost your developer portfolio

Beginner Level

Hello World Web Server: Create a basic web server that responds with "Hello, World!".
System Info Script: A script to fetch and display system information.
Log File Parser: Parse and analyze log files.
Basic REST API: Develop a simple RESTful API with CRUD operations.
CLI Task Manager: A command-line tool to manage daily tasks.
File Watcher: Monitor changes in a directory.
Environment Variable Manager: Manage and load environment variables.
Simple Reverse Proxy: Implement a basic reverse proxy server.
Dockerfile Linter: A tool to lint and validate Dockerfiles.
HTTP Status Checker: Check the status of a list of URLs.

Intermediate Level

Configuration Management Tool: Manage configuration files for different environments.
Load Balancer: Build a simple load balancer.
Service Health Checker: Check the health status of microservices.
Log Aggregator: Aggregate logs from multiple sources.
Simple CI/CD Pipeline: Implement a basic CI/CD pipeline.
SSH Automation Script: Automate SSH tasks across multiple servers.
Docker Image Cleaner: Clean up unused Docker images.
Kubernetes Pod Monitor: Monitor the status of Kubernetes pods.
Metrics Collector: Collect and visualize metrics from applications.
Infrastructure as Code Tool: Manage infrastructure using code.

Advanced Level

Custom Kubernetes Controller: Write a Kubernetes controller for custom resources.
Event-Driven Microservices: Implement microservices that communicate via events.
API Gateway: Build an API gateway for microservices.
Service Mesh: Implement a simple service mesh.
Continuous Deployment Tool: Create a tool for continuous deployment.
Distributed Tracing System: Implement distributed tracing for microservices.
Serverless Functions: Develop serverless functions using Go.
Cloud Resource Manager: Manage cloud resources programmatically.
Security Scanner: Scan infrastructure for security vulnerabilities.
Log Rotation System: Implement log rotation for log files.

Expert Level

Custom Cloud Provider: Implement a custom cloud provider interface.
Kubernetes Operator: Develop a Kubernetes operator for a specific application.
Infrastructure Monitoring System: Build a full-fledged monitoring system.
Distributed Job Scheduler: Implement a job scheduler that runs tasks across multiple nodes.
Serverless Platform: Create a serverless platform from scratch.
Cluster Autoscaler: Build an autoscaler for Kubernetes clusters.
Container Runtime: Develop a custom container runtime.
Infrastructure Security Suite: Create a suite of tools for securing infrastructure.
Network Policy Manager: Manage network policies in a Kubernetes cluster.
Cloud Cost Optimization Tool: Optimize and manage cloud costs.

Bonus Projects

Custom CI/CD Server: Develop a CI/CD server from scratch.
Cloud Backup System: Implement a system for backing up data to the cloud.
Dynamic DNS Service: Create a dynamic DNS service.
API Rate Limiter: Build a rate-limiting service for APIs.
Custom Logging Library: Develop a logging library with advanced features.
Multi-Cluster Management Tool: Manage multiple Kubernetes clusters.
Service Discovery System: Implement service discovery for microservices.
Cloud Provisioning Tool: Automate cloud resource provisioning.
Edge Computing Platform: Build a platform for edge computing.
Infrastructure Drift Detection: Detect and manage infrastructure drift.

10 resources to become a system design hero

Kumar Kalyan — Tue, 28 May 2024 16:44:17 +0000

TLDR 🔥

System Design serves as the blueprint for creating a software system. It involves defining the architecture, components, and interfaces necessary to meet specific requirements. This topic is highly relevant in the software development industry and is frequently discussed in technical interviews. Acquiring system design skills can indeed lead to a 10x salary increase. 🚀
Certainly! System design is indeed a very complex subject that goes beyond mastering a coding language . It includes understanding system architectures, design patterns, performance optimization, and other intricate topics. Learning system design from specialists or coding bootcamps can be expensive, often costing around $1000.
To help you on your journey to becoming a system design hero, in this article I have curated a list of free and valuable resources .

1. System Design Roadmap

Contains visually designed and easy to udestand roadmaps attached with useful links to master system design and other programming resources for free

kamranahmedse / developer-roadmap

Interactive roadmaps, guides and other educational content to help developers grow in their careers.

roadmap.sh

Community driven roadmaps, articles and resources for developers

Roadmaps are now interactive, you can click the nodes to read more about the topics.

View all Roadmaps · Best Practices · Questions

Here is the list of available roadmaps with more being actively worked upon.

Have a look at the get started page that might help you pick up a path.

View on GitHub

2. System Design primer

Considered for bible for system design this github repository contains a-z free resources to master system design

donnemartin / system-design-primer

Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

English ∙ 日本語 ∙ 简体中文 ∙ 繁體中文 | العَرَبِيَّة‎ ∙ বাংলা ∙ Português do Brasil ∙ Deutsch ∙ ελληνικά ∙ עברית ∙ Italiano ∙ 한국어 ∙ فارسی ∙ Polski ∙ русский язык ∙ Español ∙ ภาษาไทย ∙ Türkçe ∙ tiếng Việt ∙ Français | Add Translation

Help translate this guide!

The System Design Primer

Motivation

Learn how to design large-scale systems.

Prep for the system design interview.

Learn how to design large-scale systems

Learning how to design scalable systems will help you become a better engineer.

System design is a broad topic. There is a vast amount of resources scattered throughout the web on system design principles.

This repo is an organized collection of resources to help you learn how to build systems at scale.

Learn from the open source community

This is a continually updated, open source project.

Contributions are welcome!

Prep for the system design interview

In addition…

View on GitHub

3. awesome-system-design-resources

This repository contains System Design Key Concepts , building blocks , architechtural patterns , interview problems , distributed sytems papers , books and many more

ashishps1 / awesome-system-design-resources

Learn System Design concepts and prepare for interviews using free resources.

This repository contains free resources to learn System Design concepts and prepare for interviews.

Join my free AlgoMaster Newsletter and get a FREE System Design Interview Handbook in your inbox.

📌 System Design Key Concepts

🛠️ System Design Building Blocks

⚖️ System Design Tradeoffs

View on GitHub

4. System-Design

This repository contains artchitechtural ptterns of most pospular apps and tech startups

codersguild / System-Design

It's just fascinating. How is modern software designed? 🤔 Some design-level considerations for scalability, maintainability eventual consistency, availability & reliability. 👨‍💻 Interview Prep. 👨‍💻

System Design Bytes for Enthusiasts

I complied and collected some of the articles as a part of doing CS654A course at IIT Kanpur for a fantastic course on Advanced Software Architecture taught by Prof.Dr.T.V.Prabbhakar.

Modified from Zach system design repository. Added more links and topics to cover on both PS/DS & System Design Interviews. We will keep updating this posting from time to time. Some more awesome resource

`SYSTEM DESIGN IS NOT JUST FOR INTERVIEWS, IT CAN BE HOW YOU MANAGE YOUR LIFE. GIVE IT A THOUGHT`

Disclaimer/Acknowledgements

I didn't write the articles myself. I love expanding my understanding of system design complexities and thus read many books, research papers, and articles that enhance my knowledge. I take the opportunity to thank everyone who found this repository useful. I would like to thank all the authors and developers who wrote the articles aggregated in this repository.

5 . system-design-interview

The only repository you must checkout 7 days before your system design interview

checkcheckzz / system-design-interview

System design interview for IT companies

How to prepare system design questions for an IT company

System design is a very broad topic. Even a software engineer with many years of working experience at a top IT company may not be an expert on system design. If you want to become an expert, you need to read many books, articles, and solve real large scale system design problems.

This repository only teaches you how to handle the system design interview with a systematic approach in a short time. You can dive into each topic if you have time. Of course, welcome to add your thoughts!

[⬆] System Design Interview Tips:

Clarify the constraints and identify the user cases

Spend a few minutes questioning the interviewer and agreeing on the…

View on GitHub

6. Grokking-System-Design

This repository will help you master system design from basis to advanced level

Jeevan-kumar-Raj / Grokking-System-Design

Systems design is the process of defining the architecture, modules, interfaces, and data for a system to satisfy specified requirements. Systems design could be seen as the application of systems theory to product development.

Grokking System Design Interview

Source: educative

Interview Process

Scope the problem
- Don’t make assumptions.
- Ask clarifying questions to understand the constraints and use cases.
- Steps
  - Requirements clarifications
  - System interface definition
Sketch up an abstract design
- Building blocks of the system
- Relationships between them
- Steps
  - Back-of-the-envelope estimation
  - Defining data model
  - High-level design
Identify and address the bottlenecks
- Use the fundamental principles of scalable system design
- Steps
  - Detailed design
  - Identifying and resolving bottlenecks

Distributed System Design Basics

System Designs

System Design Interviews: A step by step guide

SYSTEM DESIGN PREPARATION

How to prepare for and answer system design questions

Objective

Learning about and implementing large-scale distributed system is not easy. I do not want…

View on GitHub

7.low-level Design

This repository mainly focuses on low level system design

prasadgujar / low-level-design-primer

Dedicated Resources for the Low-Level System Design. Learn how to design and implement large-scale systems. Prep for the system design interview.

low-level-design-primer

Motivation

Learn low level design of system at scale prepare for the low level design (LLD) / Machine Coding round interviews.

Learn to design low level system

Learning low level design of scalable systems will help you become better engineer.

This repo is an organized collection of resources to help you learn low level design of systesm's.

Resources

Low level Design Questions
Low level Design Solutions
Low Level Design Resources #TODO

Contributing

Feel free to submit pull requests to help:

Add new questions
Improve new questions
add new solutions
Fix errors
Improve sections
Add new sections

Under development

Interested in adding a section or helping complete one in-progress? Contribute!

how to guide and study material along with various resources.
add more questions and improve exisiting questions.
add solutions for the problems along with their detailed explaination (maybe video)

Credits

Credits and sources are provided throughout this repo.

Special…

View on GitHub

8. machine-learning-systems-design

A book on machine learning system design

chiphuyen / machine-learning-systems-design

A booklet on machine learning systems design with exercises. NOT the repo for the book "Designing Machine Learning Systems"

Machine Learning Systems Design

Read this booklet here.

This booklet was my initial attempt to write about machine learning systems design back in 2019. My understanding of the topic has gone through significant iterations since then. My book Designing Machine Learning Systems (O'Reilly, June 2022) is much more comprehensive and up-to-date. The new book's repo contains the full table of contents, chapter summaries, and random thoughts on MLOps tooling.

This booklet covers four main steps of designing a machine learning system:

Project setup
Data pipeline
Modeling: selecting, training, and debugging
Serving: testing, deploying, and maintaining

It comes with links to practical resources that explain each aspect in more details. It also suggests case studies written by machine learning engineers at major tech companies who have deployed machine learning systems to solve real-world problems.

At the end, the booklet contains 27 open-ended machine learning systems design questions that might come…

View on GitHub

9. System Design 101

Start learning system design as a beginner

ByteByteGoHq / system-design-101

Explain complex systems using visuals and simple terms. Help you prepare for system design interviews.

【 👨🏻‍💻 YouTube | 📮 Newsletter 】

System Design 101

Explain complex systems using visuals and simple terms.

Whether you're preparing for a System Design Interview or you simply want to understand how systems work beneath the surface, we hope this repository will help you achieve that.

View on GitHub

10.system-design-resources

Become interview ready with this repository contains topics like video processing , Cluster and Workflow Management,Service Mesh and many more

InterviewReady / system-design-resources

These are the best resources for System Design on the Internet

System Design Resources

These are the best resources for System Design on the Internet.

Video Processing

Cluster and Workflow Management

View on GitHub

10 engineering blogs to become a System Design Hero for free

Kumar Kalyan — Thu, 09 May 2024 19:36:36 +0000

TLDR🔥

System Design is basically the blueprint of a system you want to build . It is the process of defining architecture , components and the interface of a system to meet some specific requirements. System Design is one of the hot topics in the software development industry and is widely asked in tech interviews and learning this skill will guarantee you a salary hike .

In this article I will share about top 10 System design engineering blogs which will help you to become a system design master for free

Byte Byte Go Blog

Byte Byte Go is an edtech startup focusing on providing training and courses on system design topics to help you ace your system design interviews like a pro. Byte Byte go can be considered as one of the best schools to learn and master system design skills.
They have broken down complex system design topics using interactive animation videos, mind maps , cheat sheets and many more which will help you master system design like a piece of cake. Weather you are new to system design or you want to stay updated with current industry standards Byte Byte Go is a must checkout for you

Google Research

Google is one of the most popular and efficient search engines in the world. They are handling billions of users and requests every day on their platform. Google products like Angular , flutter, Android, Google Cloud, Firebase are some of the key technologies which almost every developer uses . The Google research platform has wide range of topics on software development including machine learning , software systems, hardware and architectures, distributed and parallel systems and this platform is a hidden gem for software developers to learn and research on various topics

Dropbox Tech Blog

Dropbox is an american tech company which provides storage solutions and other software products for various use cases. They also handle millions of requests everyday followed by managing and scaling large software infrastructure. Here you can explore various topics like infrastructure, frontend development ,security, mobile app development etc. If you are planning to build something similar like dropbox or a online storage service must bookmark this blog

The Netflix Tech Blog

Netflix is one of the most popular and successful Ott giants over the world resulting in handling millions of users and requests every day. You can follow Netflix engineering blogs for a wide range of topics from video streaming, microservices to machine learning and artificial intelligence. If you are someone working in the Ott industry or planning to build something on top of video streaming technology you must have an eye on the Netflix Tech blog for reference.

Uber Engineering Blog

Uber is one of the most popular and hopefully the largest online cab service providers in the world. Uber is also operating in the online food delivery sector. The reason behind I am putting Uber engineering blog in this list is because you will find some of the most crucial topics within their blog like using maps and locations to deliver services. Working with geo location data is one of the most crucial topics in software engineering and the most common industrial sectors like ecommerce and logistics are very much dependent on this information so that they can deliver their product or service as fast as possible. If you are someone in the software industry working on Geo location or navigation technology or planning to build something using these technologies then Uber Engineering blog is a must check out for you

Meta Engineering blog

Meta (formerly Facebook) is the biggest social media giant in the world and also the founder and maintainer of the most popular javascript library for building user interfaces i.e React.js. Other than that almost all popular social media apps including facebook, instagram, whatsapp, threads and a lot more are owned by them. The meta engineering blog contains a wide range of software engineering topics like web and mobile development, infrastructure systems, video technology , AR and VR technology etc. I would recommend you to keep an eye on Meta Engineering blog specially if you are building something using react or AR VR tech

Stripe Engineering blog

Stripe is one the most popular companies providing payments related solutions. Most of the online businesses and ecommerce websites use stripe for handling payments , subscriptions and invoicing even I also use stripe to access payments from my clients. Stripe Engineering blog covers a wide range of topics which you can explore like use of Machine Learning for fraud detection, UI and UX related topics for responsive and interactive payment interfaces, application security and lot more. So if you are someone who is working on payments or planning to build something related to handling payments you must definitely check out Stripe Engineering Blog as a reference.

Amazon Engineering Blog

Amazon Engineering Blog
Amazon is the largest e-commerce giant in the world not other than that it also owns Amazon Web Services (largest cloud service provider),Amazon Prime (One of the most popular ott giant ) , The audio stories platform audible the amazon music and many more. The Amazon Engineering blog covers a wide range of topics including Containers and Kubernetes, Cloud patterns and architectures, Machine learning and Amazon Artificial Intelligence technologies solve complex business challenges. So if you are someone planning to build your next app using AWS or integrate amazon technologies like Alexa then Amazon engineering blog is a must checkout for you.

Microsoft Dev Blog

Microsoft is one of the most popular tech giants in america owing the most used operating system Windows , the dot net framework for build blazingly fast enterprise applications, The Bing search engine, the copilot(One of the most efficient AI tools for productivity), the largest code sharing , hosting and version controlling platform GitHub, the Azure cloud platform (one of the largest cloud service providers). The Microsoft engineering blog contains a wide range of topics and tutorials also including Windows , azure , machine learning and artificial intelligence, the dot net framework. If you are someone planning to build your next app or software using the Microsoft technologies then you must have an eye on it

Apple Developer News

Apple is the one of the largest tech giants in the world and also owns the the most popular Mac OS and IOS (iPhone Operating System). MacBooks and iphones have a very large user base so most of the startups and enterprises build products for Mac and IOS users also. The Apple developer news mostly contains topics related to IOS , Swift (The only language to build native IOS and Mac OS apps), swift UI (The Apple UI) and somewhat on topics like C++ , Kubernetes etc. If you are someone who is planning to build a software for iOS or mac os users the developer news is a must for your reference.

Conclusion

In this article I discussed about 10 engineering blogs you must checkout to boost up your system design skills. It’s good if you are having an eye on every single blog but I would suggest that you must checkout only those blogs based on the stack you will be using or the type of industry you are dealing with

✨Top 15 GitHub repositories to achieve Typescript Mastery🏆

Kumar Kalyan — Sun, 05 Nov 2023 00:03:08 +0000

TL;DR ⚡

This article contains a list of useful GitHub repositories that will help you achieve TypeScript mastery.

🏆 These are the top-picked repositories of in-demand TypeScript libraries, structured TypeScript roadmaps, handbooks, style guides, etc.

🗺️ Make sure to explore these repositories, feel free to contribute to them if you like, and give them a ⭐ and help them grow 🎉 & please don't spam these repositories. 🙏

1. TypeScript

⚙️ JavaScript with Types

Official repository for TypeScript 🐍
Contains official source code, documentation, and programming handbook
95k stars on GitHub ✨
749 contributors 🤝
Maintained by Microsoft Corporation and the TypeScript community 🐙

Install TypeScript using

npm install -D typescript

Give a 🌠 to TypeScript

2. Roadmap.sh

🚀 The ultimate roadmap for every modern stack

Roadmaps for almost all the modern stack 🐍⚛️☁️
Contains Detailed guides, exercises, articles, and tutorial videos 📚👀🎥
275K stars on GitHub ✨
35.4K forks 🍴
695 contributors 🤝

Give a 🌠 to Roadmap.sh

3. Type Challenges

🧠 Challenge your skills with type challenges

Contains Beginner to advanced level challenges🏆
37K+ stars ✨
4K+ forks 🍴
232+ contributors 🤝
Translated into Korean, Chinese, and Japanese 🌐

Give a 🌠 to Type Challenges

4. Awesome TypeScript

📚 A collection of beginner to advanced level TypeScript resources

Contains resources 📚💻 on client-side and server-side TypeScript, Books, guides, blogs, podcasts, tutorial videos, tools, etc.
4.2K stars ✨
375 forks 🍴
152 contributors 🤝

Give a 🌠 to Awesome TypeScript

5. The Algorithms

🏆 Master TypeScript data structures and algorithms at one place

The Algorithms Repository 📖 is a comprehensive resource for algorithms in many languages, including JavaScript 🐍, TypeScript 🐍, Java 🐘, Python 🐍, and Go 🐹. It covers a wide range of data structures, such as graphs 📊, arrays 🔢, trees 🌳, heaps ⛰️, hashmaps 🗝️, tuples, etc 🤔 .
776 stars ✨
201 forks 🍴
55 contributors 🤝

Give a 🌠 to The Algorithms

6. Zod

✨ TypeScript-first schema validation with static type inference

Zod 🐍 is a schema declaration and validation library for TypeScript 🐍. It works with popular backend frameworks like Node.js 🐘, Deno 🦕, and Bun 🐹, and supports all modern browsers 🌐. It allows developers to define type-safe schemas for various data types like strings 📝, arrays 🔢, primitive types 🪨, etc.
Tiny in size 🤏
Have immutable methods 🔒
Have concise chainable interface ⛓️
26.5k stars ✨
895 forks 🍴
256 contributors 🤝 Install Zod using

npm install zod

Give a 🌠 to Zod

7. Zustand

⚛️ Modern state management for React

Zustand ⚡ is one of the most popular state management libraries for React.js ⚛️ and its ecosystem frameworks like Next.js 🏃‍♂️. It is a fast and scalable state management library built on top of Flux principles 🌀. It helps in handling critical challenges like React concurrency 🔀 and context loss ❌.
Tiny in size 🤏
36.5k stars ✨
1.1k forks 🍴
213 contributors 🤝

Install Zustand using

npm install zustand

Give a 🌠 to Zustand

8. tRPC

🛡️ End-to-end typesafe APIs made easy

tRPC 🐍 stands for TypeScript Remote Procedure Call 📞 and is used for building APIs in TypeScript 🐍. It is lightweight 🪶 and easy to integrate into TypeScript projects 🧩. It supports multiple transport protocols, including HTTP 🌐, Web Sockets 🕸️, and TCP 🔌. It is used for building Next.js applications 🏃‍♂️, micro frontends 🧩, and applications supporting real-time data exchange ⏩.
29.6k stars ✨
1.1k forks 🍴
327 contributors 🤝 Install tRPC using

npx create next-app --example https://github.com/trpc/trpc --example-path examples/next-prisma-starter trpc-prisma-starter

Give a 🌠 to tRPC

9. Formbricks

📊 The Open Source Survey Suite

Form Brick 🧱 is an open-source alternative to Typeform 📱 used for taking surveys 📋. It can be easily integrated with any modern web stack 🌐. It has features like FormBuilder 🛠️, triggers 🔔, and analytics and insights generated on form data 📈.
4.2k stars ✨
675 forks 🍴
127 contributors 🤝 Install Form Bricks using

npm install --save @formbricks/js

Give a 🌠 to Formbricks

10. Prisma

💧 Next-generation Node.js and TypeScript ORM

Prisma 🐍 is a next-generation ORM for TypeScript 🐍 and Node.js 🐘. It is used to communicate with the database 💽 within an app 📱. It supports modern databases like PostgreSQL 🐘, MySQL 🐬, SQLite 🐿️, SQL Server 🏢, MongoDB 🐳, and CockroachDB 🪳.
34.6k stars ✨
1.4k forks 🍴
226 contributors 🤝

Install Prisma Using

npm install @prisma/client

Give a 🌠 to Prisma

11. Trigger.dev

⚙️ The open source Background Jobs framework for TypeScript

Trigger.dev 🤖 helps in defining long-running jobs ⏳ directly inside your app 📱 with API integrations 🤝, webhooks 🕸️, scheduling 🗓️, and delays ⏱️. It's integrated with popular tools like GitHub 🐙, Airtable 🌌, OpenAI 🤖, Supabase 🐬, Resend ✉️, SendGrid ✉️, Stripe 💳, and more 🛠️.
5.6k stars ✨
288 forks 🍴
51 contributors 🤝

Install Trigger.dev using

npx @trigger.dev/cli@latest init

Give a 🌠 to Trigger.dev

12. Novu

🔔 Hassle-free notification infrastructure at one place

Novu 🔔 helps you manage all your notifications in one place. It supports channels and providers like web push 🌐📱, in-app 📱, chat 💬, email ✉️, and SMS 📱. It provides workflows like step filters ⚙️, digest engine 📰, predefined templates 📄, delayed actions ⌛️, and messages 💬. The best part is that it integrates with almost all popular services like FCM 🤖, SendGrid ✉️, Twilio 📞, OneSignal 🔔, and more 🛠️.
30.7k stars ✨
3.4k forks 🍴
360 contributors 🤝

Install Novu using

npm install @novu/notification-center

Give a 🌠 to Novu

13. NGX Admin

🛠️ Build admin panels faster than ever

Ngx Admin 🏰 is an open-source pre-built admin dashboard 📊 that can be used to build dashboards for various industries, such as e-commerce 🛍️, IoT 🤖, and real-time monitoring 📈. It is written in Angular 🐟 and is fully responsive 📱.
29.6k stars ✨
1.1k forks 🍴
327 contributors 🤝

Install NGX Admin using

git clone https://github.com/akveo/ngx-admin.git
cd ngx-admin && npm i
npm start

Give a 🌠 to NGX Admin

14. Refine

🏢 Open-source Retool for Enterprise

Refine 🚀 is a React meta framework ⚛️ that helps developers build admin panels 🏢, internal tools 🛠️, and B2B apps 🤝 faster with predefined features like authentication 🔐, access control 🛡️, routing 🗺️, networking 🌐, state management 🧠, and i18n 🗣️. It works with popular providers like Appwrite ☁️, Supabase 🐬, Auth0 🔒, Hasura 🐘, MUI 🎨, and AntDesign 🐜.
29.6k stars ✨
1.1k forks 🍴
327 contributors 🤝 Create a refine app visiting the URL Below

https://refine.new/

Give a 🌠 to refine

15. Nest Js

📈 A progressive Node.js framework for building efficient, reliable and scalable server-side applications.

Nest.js 🐦 is an open-source backend development framework 🚀 used for building highly scalable backend servers 🏢 written in TypeScript 🐍. Its key features include support for GraphQL 🔮, highly secured authentication and authorization 🔐, microservices using Redis 🔴, Kafka 🐘, and gRPC 📞, built-in support for WebSockets 🕸️, and OpenAPI specification 📖.
29.6k stars ✨
1.1k forks 🍴
327 contributors 🤝 Create a nest app using

 npm i -g @nestjs/cli
 nest new project-name

Give a 🌠 to Nest Js

So, here my article ends 🏁. Feel free to mention your thoughts or any suggestions or any other useful repositories for TypeScript developers that I haven't mentioned in my articles in the comments section 💬.

Stay tuned for my next blog till then

Top 5 Github repositories to achieve system design mastery

Kumar Kalyan — Sat, 28 Oct 2023 16:24:39 +0000

TLDR 🔥

This article contains a list of the top 5 GitHub repositories that will help you to achieve system design mastery.

These repositories mainly contain tutorials, blogs, videos and case studies related to system design

Make sure to visit these repositories, explore the contents, implement the learnings, contribute if you can and give them a ⭐ to help them grow.

What is System Design

System design ⚙️ is the process of designing the architecture, components, interfaces, and data for a particular system to meet specific requirements.

Before building any application or system, 🏗️ it is important to plan how the system will work, what resources will be needed, where to deploy it, which APIs the system components will use, and how to scale the system. Therefore, system design concepts are very important for software developers to master to build efficient and scalable systems. 💯

1. developer-roadmap

kamranahmedse / developer-roadmap

Interactive roadmaps, guides and other educational content to help developers grow in their careers.

roadmap.sh

Community driven roadmaps, articles and resources for developers

Roadmaps are now interactive, you can click the nodes to read more about the topics.

View all Roadmaps · Best Practices · Questions

Here is the list of available roadmaps with more being actively worked upon.

Have a look at the get started page that might help you pick up a path.

View on GitHub

developer roadmap The GitHub repository is built and maintained by Kamran Ahmed and this repository can be considered the best handbook for software development
It has 255k stars on Git Hub and 684 contributors. This repository is only available in English and it contains a structured roadmap for everything along with resources, guides, etc.

Give a ⭐ to developer-roadmap

2. System Design Primer

donnemartin / system-design-primer

Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

Help translate this guide!

The System Design Primer

Motivation

Learn how to design large-scale systems.

Prep for the system design interview.

Learn how to design large-scale systems

Learning how to design scalable systems will help you become a better engineer.

System design is a broad topic. There is a vast amount of resources scattered throughout the web on system design principles.

This repo is an organized collection of resources to help you learn how to build systems at scale.

Learn from the open source community

This is a continually updated, open source project.

Contributions are welcome!

Prep for the system design interview

In addition…

View on GitHub

Sytem Design Primer GitHub repository was created by a tech Lead at Facebook (now Meta). This repository can be considered the bible of system design
It has 233k stars on GitHub and 100+ contributors and is available in many different languages.

Give a ⭐ to System Design Primer

3. system-design-101

ByteByteGoHq / system-design-101

Explain complex systems using visuals and simple terms. Help you prepare for system design interviews.

【 👨🏻‍💻 YouTube | 📮 Newsletter 】

System Design 101

Explain complex systems using visuals and simple terms.

Whether you're preparing for a System Design Interview or you simply want to understand how systems work beneath the surface, we hope this repository will help you achieve that.

View on GitHub

System Design 101 GitHub repository is built and maintained by Byte Byte Go Hq which is a platform to ace System Design interviews
It has 39.2k starts on Github and 14 contributors. This repository is only available in English and is the best repository for beginners. This repository contains effective guides on topics like communication protocols, CI-CD, Architectural patterns, microservices, DevOps, cloud, Linux, etc.

Give a ⭐ to system-design-101

4. System Design Resources

InterviewReady / system-design-resources

These are the best resources for System Design on the Internet

System Design Resources

These are the best resources for System Design on the Internet.

Video Processing

Cluster and Workflow Management

View on GitHub

Sytem Design Resources GitHub repository is built and maintained by Interview Ready which is a platform to ace Sytem design interviews for big tech giants.
It has 11.4k stars on Git Hub and 8 contributors. This repository is only available in English. It includes case studies, guides and video lectures on various system design topics including Video Processing, Cluster and Workflow Management, Intra-Service Messaging, Message Queue Antipattern, Service Mesh, Practical System Design, Distributed File System, Time Series Databases, Rate Limiting,In Memory Database - Redis, and many more

Give a ⭐ to system-design-resources

5. system-design

karanpratapsingh / system-design

Learn how to design systems at scale and prepare for system design interviews

System Design

Hey, welcome to the course. I hope this course provides a great learning experience.

This course is also available on my website and as an ebook on leanpub. Please leave a ⭐ as motivation if this was helpful!

Getting Started
- What is system design?
Chapter I
Chapter II
Chapter III
Chapter IV
- …

View on GitHub

system-design GitHub repository is built and maintained by Karan Pratap Singh who is a Senior Software Engineer at Curebase.
It has 21.1k stars on Git Hub and 9 contributors. This repository is only available in English. It is basically the free version of the author's real book. It has 5 chapters and is best for beginners who are new to development.

Give a ⭐ to system-design

Stay tuned for my next blog till then

New VSCode extension to to build free Bootstrap templates visually and get clean HTML & CSS

Kumar Kalyan — Mon, 06 Mar 2023 17:40:02 +0000

Visual studio code is a lightweight, open-source and one of the most used code editors used by a wide range of developers and its extensions are pretty much very useful to provide an experience of a heavyweight IDE(Integrated Development Kit ). In this article, I will talk about a newly launched vs code extension named speedwapp which helps you to build production-ready, responsive and mobile friendly landing pages directly from vs code. This tool can be a game-changer if you are looking forward to building a bootstrap template within a couple of minutes.

Brief intro on Speedwapp vscode extension

Speedwapp is a no-code platform that allows users to generate beautiful, responsive, and mobile-friendly landing pages for websites. The platform is designed to be easy to use, even for those with little to no coding experience.

One of the standout features of Speedwapp is the Visual Studio Code (VSCode) extension, which makes it easy to create and edit landing pages directly from the VSCode editor. No need to get stuck between multiple divs or spend hours fixing CSS for a responsive page . This feature is particularly useful for developers who are already familiar with VSCode and want to quickly create landing pages without having to switch to a different platform.

The platform also offers a wide variety of customizable templates that users can choose from, making it easy to create a landing page that matches the look and feel of their website. Users can also use the platform's drag-and-drop editor to add and arrange elements on the page, such as text, images, and buttons. And once your editing is done you can always edit the generated code as per your requirements

In terms of responsiveness and mobile-friendliness, Speedwapp does not disappoint. All landing pages created on the platform are automatically optimized for both desktop and mobile devices, ensuring that they look great and function correctly on any device.

One more useful feature is, this tool is a free extension you can always add or remove it easily when required now you don't need to think about the disk space or RAM of your device

Why this tool is better than other

There are similar products available in the market but here are a few points you must have in mind

Free to use under the starter plan, you don't need to pay a single penny unless you want some paid templates
Lightweight vs code extension, no need of downloading heavy-weight software which takes a couple of minutes to boot
Fair pricing as compared to other available products
Can be integrated with WordPress
Easy to use and can be by used anyone also someone who has little or no prior coding experience

For whom is this tool for

This tool can be useful for a wide range of audience but still I would like to highlight a few of them

Digital product sellers or e-commerce who are looking forward to building product landing pages
Both professional and junior web developers. This tool helps to build tempted within a couple of minutes
WordPress and Wox developers

Use cases for this tool

This tool can have a wide range of used cases but I will highlight a few of them

Ecommerce development
Personal portfolios
Product Landing pages
Email Marketing templates

Conclusion

Overall, Speedwapp is a powerful no-code platform that makes it easy to create beautiful, responsive, and mobile-friendly landing pages. The platform's VSCode extension and customizable templates make it easy to create pages that match the look and feel of your website. If you're looking for a no-code platform to create landing pages for your website, Speedwapp is worth checking out here.

Share this tool with your friends and colleagues if you find it useful and Make sure to follow and subscribe for email notifications on my blog if you want to stay updated with my latest post

Let's crack the block chain together

Kumar Kalyan — Tue, 31 Jan 2023 05:27:21 +0000

Over the whole of 2022, I heard about the term blockchain, web3, crypto, etc and so on and I was like sleeping while the whole world was making millions of dollars through. I have tried exploring it a few times but unfortunately, my procrastination defeated me in becoming a millionaire blockchain developer.

The day I'm producing content I explored a lot of topics and looked around the use cases of blockchain. I find this technology is our future and everyone must be aware of it. As you all know that I'm a genius at breaking down content like a piece of cake so I have decided to start a new series on blockchain and believe me I am on a mission to bring awareness to blockchain technology.

In this series, I will cover every single topic which will help you to become a millionaire blockchain developer for free and all I need is some support from you guys to help me stay consistent on my mission to educate the blockchain community.

I'm not a pro-level blockchain developer but you guys can trust me and can learn along with me from scratch. Together we will explore all the libraries, SDKs and even the deployment, and I promise the content will be authentic and easy to understand. I am planning to start with Ethereum as it is the most popular having a huge community base and followed by other blockchain technologies like polygon, Cello, flow, deso, Solana and a lot more.

My article will contain everything starting from how-to guides, tutorials, software comparisons, blockchain automation tools and every single thing that you can't imagine. All I need is your support these articles are free to read and along with these, you gonna learn a lot of things regarding blockchain industry trends.

Overall blockchain technology gonna remain in the future as it solved various problems which we are dealing with in today's world. All we need is some beginner-friendly how-to guides and tutorials which I will provide to start making money from the blockchain industry

Building HTML templates visually in visual studio code

Kumar Kalyan — Sun, 25 Dec 2022 19:26:40 +0000

Introduction

Hello, devs do you find it hard to build web pages within a short period? Are you still using the boring process of writing Html & CSS code to develop a fully responsive and UI-friendly web page? Do you spend hours just to make a single web page responsive and mobile-friendly? There is good news! In the article, I will demonstrate how you can easily build a fully customizable product landing page within 5 minutes that too using a vs code extension named speedwapp the best part you don't need to type the full code and follow the boring process. There is no such prerequisite for this article a simple knowledge of HTML & CSS is appreciated just make sure that you have visual studio code installed on your computer and if not you can download it from here.

A bit about speedwapp vs code extension

Speedwapp is an amazing website builder that allows you to build beautiful landing pages directly from their website or you can your their free vs code extension which will demonstrate here.

Adding the seedwapp extension to your vscode

Quickly open the visual studio code editor on your computer and navigate to the extensions section. If you are using a windows machine press ctrl+shift+x . Now in the search bar search for speedwapp and click on the install button
Once you have installed the extension you will be prompted to a login page so put your credentials there or you can also sign up if you don't have an account.
Now press Ctrl+K N or Cmd+K N to create a new page using speedwapp
click on the + symbol to add pre-defined templates or html code blocks to your page. I will be using a pre-defined template and then gonna edit it on my own

.
It looks a bit like this after I have imported one. Now let's add a few more sections and then we will edit it.
There in the bottom, you will see a small plus sign to add a new component so click on that and add a few more just like shown in the image below
Now let's edit it. So first, I want to change the background image of the first component and to do so click on the image you will see a panel on the left and there you can update it
Now we got a pop-up like this and I will be using the image links so head over to unsplash and pick an image of your choice then copy the image URL and paste it into the link section and click on upload.
Now I want to change the font style and size of the services paragraph so click on that component and in the above you will see an option to select a font and size once you enter the values where the changes will be live.
Now let's say I want to edit the content of the below section so click on the text and write whatever you want
Now let's add links to these checkout buttons so click on them and in the right side panel click on the properties and there select Link under the HTML tag section and finally, paste your links there

Now let's edit the last section I want the button colour to be green so will click on that and side on styles on the left and there will change the button style to success.

So finally our landing page is ready now let's deploy it to netlify.

Exporting code & customizations

Click on preview changes at the top and then click on view code

Here you will see the full code of the beautiful webpage which we just made using speedwapp vs code extension. Now click on copy to workspace
Now you will see the automatically generated code files

Now move the code to a new folder and upload it to a GitHub repository
Once you are done uploading the code head over to your netlify dashboard and click on add a site select the existing project and choose GitHub provider
Now select your repository and click on deploy site

Check out my site here https://speedwaplanding.netlify.app/

GitHub Repository https://github.com/kumar-kalyan/speedwapp-landingpage

Conclusion

Congratulations you have successfully learned to build beautiful and responsive landing pages directly from vs code without writing much code. Feel free to reach me for any queries or simply visit speedwapp website and make sure to share this article with your peers and let me know how much you loved this new tool

Integrating file upload feature to a nodejs project

Kumar Kalyan — Wed, 21 Dec 2022 15:13:32 +0000

As a web developer or backend engineer, you may often get into a situation where you have to allow the users to upload some files to a server through the application you are working on. This can be images, videos, documents or anything. There are tons of paid saas products or cloud storage solutions available in the market to solve this problem but what if I tell you that there is an open-source npm package which can help you out with this? In Node js, there is a popular package named multer which can handle file uploads in your application.

Multer allows you to handle HTTP requests with enctype="multipart/form-data" which is used for file uploads. Here enctype is an HTML attribute which means an encoding type used in HTML forms and the multipart/form-data is the protocol for sending form data to the server in multiple parts due to the large size of files to be uploaded.

Multer is very easy to use and Integrating it into a Node js application is very simple and can be done using a few lines of code.

In this article, we will be looking through the process of integrating multer and also discuss the various use cases and customization features.

Prerequisites

Nodejs installed in your machine
Any prefered code editor I'm using vs code
express js app. Check out the template here

Setup the express app from the above template

Open up the terminal and paste the below command
```
git clone https://github.com/kumar-kalyan/express-app.git
```
This will copy all the template files to your local machine
Now enter the root directory of the template
```
cd express-app
```
- Install all the directories using npm install

Getting started

Open up the terminal in the root directory and use the below command to add multer to the Node js project

npm install multer

Once the installation is completed now you need to import the multer module into your project using the require keyword and finally set up the middleware. This can be done inside the server.js file or any separate module as per project structure.

Middleware in Express is a special function which has access to the request object, response object and the next middleware function within the application request-response lifecycle.
In this article, we will implement multer middleware in the server.js file.
```
const multer = require('multer')
```
The above code shows how to import a module in a Node js project.

Setup the storage function

Now let's setup storage function inside the multer module to store the uploaded files
```
// SET STORAGE
var storage = multer.diskStorage({
    destination: function (req, file, cb) {
        cb(null, 'uploads/')
    },
    filename: function (req, file, cb) {
        cb(null, file.fieldname + '-' + Date.now() + ".png")
    }
})
var upload = multer({ storage: storage })
```
multer has a diskStorage function which takes two parameters named destination and filename the former is responsible for handling the upload destination which is set to a folder named uploads and the latter is responsible for handling the name of the file which is to be uploaded. For the file name, we are taking the file.fieldname which can be accessed from the request object and followed by a concatenation of the current date using the javascript date object lastly adding the filename type will store all the uploaded files in png format

after that, we are configuring the upload variable and setting the storage inside the multer function.

Create a POST request for file upload

Now let's create the post request for uploading files

app.post('/profile', upload.single('upfile'), function (req, res) {
    // req.file is the `avatar` file
    try {
        res.json({
            "name": req.file.originalname,
            "type": req.file.mimetype,
            "size": req.file.size
        })
    }
    catch (err) {
        res.json({ "err": err })
    }
    // req.body will hold the text fields, if there were any
})

So in the above request, we are passing the upload.single middleware function and inside it specifying the name of the file. On successful upload, we are sending a JSON response to the user and on errors, we are sending the error message as a JSON response. We are using try catch block to handle errors

Let's build the front end for file upload

Now let's build the front end for the file upload

Create a folder named views inside it create a file named index.html

Inside the index.html create a simple form using the following code
```
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
</head>
<body>
    <form action="/profile" method="post" enctype="multipart/form-data">
        <input type="file" name="upfile" />
        <input type="submit" value="upload" class="btn btn-default">
      </form>
</body>
</html>
```
Make sure to specify the attribute of enctype="multipart/formdata" which I have mentioned earlier in this article. create an input element of type file and name to upfile this can be anything of your choice but make sure to keep it the same which we have used in the upload middleware in the server.js file. In the form make the method attribute to a POST request and the action attribute to /profile which is our route to the POST request for file upload.

Setup server.js for rendering index.html

Now let's create a basic GET request and will send this file to the user using the process.cwd in Node Js. cwd stands for the current working directory is a function inside the Node Js process module and is used to work with files. Importing the process module can be done by following a piece of code
```
const process = require('process');
```
The GET request can be defined like this
```
app.get('/', function (req, res) {
    res.sendFile(process.cwd() + '/views/index.html');
});
```
So the above code is responsible for sending the index.html to the user.

Let's check the result
- Start the server using
```
node server.js
```
open your browser and open the link http://localhost:5000/
Click on the Choose File button to select files
Now click on upload

You must see a response like this

Conclusion

Overall, Multer makes it easy to add file upload functionality to your Node.js application. Whether you are working on a small personal project or a large enterprise application, Multer has you covered. There are many customizing options available for multer like multiple file uploads, specifying the size, etc. which can be found on the multer's NPM page. Make sure to share this article with your peers and reach out for any queries. Follow Documatic for amazing content like this one.

Happy Coding :)

Let's build a backend api like bit.ly using node js

Kumar Kalyan — Mon, 12 Dec 2022 18:15:56 +0000

Introduction

These days short links are used by almost everyone, developers, creators businesses and even working professionals, and there are popular tools like bit.ly, buff.ly, etc already available in the market. But have you ever wondered why it is very much easy to build a similar service like that? In this article, I will explain how you can easily integrate a short URL service to a node js project and that too within 100 lines of code.

Before I start explaining let me first tell you about the prerequisites.

Node js installed in your machine
Any code editor of your choice, I am using vs code
Tools like postman or insomnia for testing the API
Knowledge of nodejs callbacks, MongoDB

Tech stack we are using

Nodejs for writing server-side code
MongoDB Atlas as a database
Node package Manager but one can use yarn too

MongoDB setup

Head over to mongodb.com and login to your dashboard
Move to the projects tab inside the organization(make sure to create one if you don't have one ) and click on the new project

next click on Add Current IP Address
Next click on Build a Database and after that select a shared cluster
Choose your prefered cloud provider and click on create cluster
Next set up a username and password
Now move to the database tab and click on connect

Now click on connect your application grab the connection string for node js driver and save it for later use

Setup the nodejs app

Folder structure
Now quickly open up the terminal in the root directory and install the dependencies using the command

npm install body-parser cors dotenv express mongoose mongodb valid-url

Now open the server.js file and set up an express server using the piece of code

//set up an experss server
const express = require('express');
const app = express();
const port = 3000;
app.listen(port, function () {
  console.log(`Listening on port ${port}`);
});

Here we are setting up an express server using the app variable and express() function and setting a port address to 3000 using app.listen() so that our server can listen to http://localhost:3000/.

This piece of code will only set up a server but to build an API we need to create some requests will I will discuss later in this article first let us import some other important modules which we will use to build our API

Let's setup body-parser

body-parser is an npm package which allows you to access the request body in simple words to build this API we need to receive the URL from the user and this will be delivered to our server through a post request to get access to the requested data we will use body-parser. Use the piece of code to set up body-parser
```
var bodyParser = require('body-parser');
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
```
Now it's time to connect our database before we start building the requests. Here comes the concept of environment variables using the dotenv npm package and & a .env we can store our secrets from others. First, let's configure dotenv in our server.js . Using process.env we can access our environment variables.
```
require('dotenv').config();
const dbUrl = process.env.DB_URI;
const port = process.env.PORT || 3000;
```
Setup the .env file and paste your secrets there

PORT=3000
DB_URI=mongodb+srv://<username>:<password>@cluster0.hjcrjts.mongodb.net/?retryWrites=true&w=majority

Let's write a Schema for creating a MongoDB model to store the URLs and also connect our server to MongoDB Atlas. A schema is a blueprint of what kind of data we want to store in our database, whether it's a string or an array or an integer, etc. Based on this schema a model will be generated and used so that we can perform our database operations. Keep in mind that to design a schema or model or to perform any sort of MongoDB operations we are using an npm package named mongoose can and to create an instance of a mongoose package use
```
const mongoose = require('mongoose');
```
now add the following piece of code to define a schema, and model and connect the database with our server.

mongoose.connect(dbUrl, { useNewUrlParser: true }).then(() => { console.log('connected to the database') },
  err => { console.log(err) })
const { Schema } = mongoose;
const urlSchema = new Schema({
  original_url: { type: String, required: true },
  short_url: String,
});
const urlSch = mongoose.model("urlsStore", urlSchema);

So we are storing two fields in our database one is the original_url and another is short_url

Let's write the first API request to store URLs and create new short URLs

var validUrl = require('valid-url');
app.post('/api/shorturl', async (req, res) => {
  let url = req.body.url;
    //Checks weather a URL is valid or not 
  if (!validUrl.isHttpsUri(url)) {
    res.json({ "error": 'invalid url' });
  }
  else {
    try {
        //checks Given URL is already present the database
      let URL = await urlSch.findOne({ original_url: url });
      if (URL) {
        res.json({ 'original_url': URL.original_url, 'short_url': URL.short_url });
      }
      else {
        var query = urlSch.find();
        query.count((err, count) => {
          const URL = new urlSch({
            original_url: url,
            short_url: count + 1,
          });
          URL.save().then((URL) => {

            res.json({ 'original_url': URL.original_url, 'short_url': findOne.short_url })
          })
        })

      }
    }
    catch (err) {
      if (err) {
        console.log(err);
        res.status(500).json('server error')
      }
    }

  }
});

All we are doing here is grabbing the URL from the request.body and storing inside the variable url after that check for its validity using an npm package valid-URL , then we are checking to if the URL is already available in our database using the findOne() the method by passing the URL as a query parameter or not lastly if all the checks are passed then we are adding up the given URL to our database and sending the short URL as a JSON response. The logic behind generating the short URL is very simple we are first counting the total number of URLs present in our Database and storing it to a count variable then we are assigning the short_url with count+1 this means the 1st URL will be someurl/1 and so on.

Now let's create a request for redirecting the users using the short URL

app.get('/api/shorturl/:short_url', async (req, res) => {
  let url = req.params.short_url;
  let already = await urlSch.findOne({ short_url: url }).exec();
  try {

    if (already) {
      res.redirect(already.original_url);
    }
    else {
      res.json({ error: 'invalid url' })
    }
  }
  catch (err) {
    if (err) {
      console.log(err);
      res.status(501).json({ error: 'invalid url' });
    }
  }
})

There were grab the URL from the parameter and check for its presence inside our database if it exists then we are redirecting our users to the link using the res.redirect()

Now let's write the code for fetching all the list of URLs present in our database
```
//get all URLs
app.get('/api/urls', async (req, res) => {
  try {
    let urls = await urlSch.find({}).exec();
    res.json(urls);
  }
  catch (err) {
    console.log(err);
  }
})
```
Here we are using the find() the method in MongoDB and passing an {} inside it to get all the data inside the urlSch model and lastly returning a JSON object containing all the URLs inside our database.

Start the server and see the results

use node server.js to start the server and test the API in postman

Creating a POST and passing a url parameter through the request body we are sending our server a URL and after processing this returns me a valid link and the shorturl id.

Let's try out an invalid URL and see what it returns

see it returns an error saying invalid URL
GET a URL

use http://localhost:3000/api/shorturl/6 in your browser and you will be redirected
Get all URLs

simply search for http://localhost:3000/api/urls using a browser and you get all the available URLs inside our database.

Conclusion

Ok, now we have reached the end make sure to share this article with friends and peers and make sure to hit a reaction. Follow Documatic and stay tuned for other amazing articles :)

DEV Community: Kumar Kalyan

Building a 🧾 PDF Invoice Generator with 📨 Twilio SendGrid, 🐍 Python, Flask, and 🖌️ Bootstrap

Introduction 🌟

Tutorial Requirements 🛠️📋

Configure Your Twilio SendGrid Account 🔧📤

Configure your Python Virtual Environment 🌟🐍

Create your Flask application 🐍🧾📧

Create the home page for your Flask application 🏠💻

Building the invoice form using bootstrap 🧾✨

Building the functionalities for buttons 🔘⚙️

Write the Python code that would generate invoices in PDF format and send emails.

Let’s code the genenate_invoice function 💻🧾

Let’s code the send_invoice_via_email function 📧✨

Run your application 🚀💻

Some key points to remember for deployment 🌐📝

Conclusion 🎯✅

🔥 17 Best Free GitHub Repositories to Crack System Design Interviews 🛠️

📜 TL;DR ✂️

GitHub Repositories

1. System Design Primer

2.System Design Roadmap by Developer Roadmap

3. System Design 1o1 by Byte Byte Go

4. System Design Resources by Interview Ready

5. Awesome Low Level Design by Ashish Pratap Singh

6. System Design by Karanpratap Singh

7. Low Level Design Primer

8. Low Level Design By Interview Ready

9. System Design Interview

10. System Design By Shashank Khare

11. Awesome System Design Resources

12. Grokking System Design By Jeevan Kumar Raj

13. System Design questions by Arpit Bhayani

14. System Design and Architecture By Tian Pan

15. Complete System Design by Coder World

16. System Design By Coders Guild

17. Mobile System Design Interviews

Conclusion

TOP 50 project ideas to become a golang hero

TLDR 🔥

Beginner Level

Intermediate Level

Advanced Level

Expert Level

Bonus Projects

10 resources to become a system design hero

TLDR 🔥

1. System Design Roadmap

kamranahmedse / developer-roadmap

Interactive roadmaps, guides and other educational content to help developers grow in their careers.

View all Roadmaps · Best Practices · Questions

2. System Design primer

donnemartin / system-design-primer

Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

The System Design Primer

Motivation

Learn how to design large-scale systems

Learn from the open source community

Prep for the system design interview

3. awesome-system-design-resources

ashishps1 / awesome-system-design-resources

Learn System Design concepts and prepare for interviews using free resources.

📌 System Design Key Concepts

🛠️ System Design Building Blocks

⚖️ System Design Tradeoffs

4. System-Design

codersguild / System-Design

It's just fascinating. How is modern software designed? 🤔 Some design-level considerations for scalability, maintainability eventual consistency, availability & reliability. 👨‍💻 Interview Prep. 👨‍💻

System Design Bytes for Enthusiasts

SYSTEM DESIGN IS NOT JUST FOR INTERVIEWS, IT CAN BE HOW YOU MANAGE YOUR LIFE. GIVE IT A THOUGHT

Disclaimer/Acknowledgements

More Content Comming

5 . system-design-interview

checkcheckzz / system-design-interview

System design interview for IT companies

6. Grokking-System-Design

Jeevan-kumar-Raj / Grokking-System-Design

Systems design is the process of defining the architecture, modules, interfaces, and data for a system to satisfy specified requirements. Systems design could be seen as the application of systems theory to product development.

Interview Process

Distributed System Design Basics

System Designs

`SYSTEM DESIGN IS NOT JUST FOR INTERVIEWS, IT CAN BE HOW YOU MANAGE YOUR LIFE. GIVE IT A THOUGHT`