DEV Community: Navin Kodag

How to: Next.js API Global Errors & Auth Middleware

Navin Kodag — Sun, 31 Dec 2023 00:16:10 +0000

This one's going to be quick. no bs. I was working on a Next.js project and needed a way to handle API Route Errors Globally. Similar to Express in Node.js.
This is what I've come up with and it works wonders for my setup.

We create a handler function that will take multiple handlers and run them one by one.

import { ApiError } from "next/dist/server/api-utils";
import { NextResponse, NextRequest } from "next/server";

export const custom_middleware =
  (...handlers: Function[]) =>
  async (req: NextRequest, res: NextResponse) => {
    try {
      for (const handler of handlers) {
        await handler(req, res);
      }
    } catch (error) {
      if (error instanceof ApiError) {
        return NextResponse.json(
          { message: error.message },
          { status: error.statusCode }
        );
      } else {
        /// Log server errors using winston or your preferred logger
        console.log(error);
        return NextResponse.json(
          { message: "Server died for some reason" },
          { status: 500 }
        );
      }
    }
  };

Now we can add it to a route

/// app/api/ping/route.ts

import { custom_middleware } from "@/app/lib/server/middleware";
import { ApiError } from "next/dist/server/api-utils";
import { NextRequest, NextResponse } from "next/server";

const main_handler = (req: NextRequest, res: NextResponse) => {
  const isAuthenticated = false;

  if (isAuthenticated) {
    return NextResponse.json({ success: true });
  }

  throw new ApiError(400, "Some error");
};

export const GET = custom_middleware(main_handler);

We can see the fruits of our labour in the browser itself.
We can add our own custom authentication logic here too.

export const auth_middleware = async (req: NextRequest, res: NextResponse) => {
  /// Your auth logic
  const isAuthenticated = false;
  if (!isAuthenticated) {
    throw new ApiError(401, "Unauthorized");
  }
};

Call it in our middleware_handler

export const custom_middleware =
  (...handlers: Function[]) =>
  async (req: NextRequest, res: NextResponse) => {
    try {
      ///
      /// Auth middleware
      await auth_middleware(req, res);

      for (const handler of handlers) {
        await handler(req, res);
      }
    } catch (error) {
      if (error instanceof ApiError) {
        return NextResponse.json(
          { message: error.message },
          { status: error.statusCode }
        );
      } else {
        /// Log server errors using winston or your preferred logger
        console.log(error);
        return NextResponse.json(
          { message: "Server died for some reason" },
          { status: 500 }
        );
      }
    }
  };

Test the Auth middleware once
Profit 💰

literally me profiting

You can look at the code here: Github
Follow me everywhere. OG Blog

Managing multiple github accounts on your machine

Navin Kodag — Sun, 26 Dec 2021 18:18:50 +0000

All of us have and might go through this problem at some point in our careers where we've got multiple github account for our personal, work and freelancing projects.

Before a couple of months ago I would used two variations of VS code in my machine. One was the Visual Studio Code and other was VScodium .
But that was a work around until I found out that you can add your username and password/ Personal Auth Token when you clone the repo. After that you can start working normally and git will take care of permissions.
We need the Personal Auth Token because github has discountinued using our traditional passwords. And PATs can expire, they can also be given selective permissions.

So this is how it works, when you're cloning a repo just put your username:PAT
example:

git clone https://{username}:{PAT}@github.com/{username/org}/{repo}

Yes, it was as simple as that.

You can find me at https://100lvlmaster.in

Adding view count to your Nextjs Blog

Navin Kodag — Sun, 07 Nov 2021 20:02:20 +0000

Alright lads, this will be a quick one. I want to add the views count functionality on my personal portfolio website's blog section.

Expected behavior:

Blogs Page : List of blogs -> Show count.
Blog Page : Particular Article -> Show count and Increment count.

How to achieve:

Use supabase to store count by slug
Stored procedure to increment count

Tools that I'll need:

supabase : open source firebase alternative
swr : data fetching

Setting up supabase table :

Create a table views with schema like such:

slug -> text -> primary key
created_at -> timestamp -> now()
count -> int2

Updating `count`:

Fetch count
Increment one
Fetch count again

Now we can reduce this to one db call using stored procedures:

create function increment (slug_text text)
returns void as
$$
update views
set count = count + 1
where slug = slug_text;
$$
language sql volatile;

In NextJs:

We'll define a route for ease:
- /api/view/{slug}
and then we'll use the POST request to register a view and GET to increment the view count.
Our handler code will look like this:
views.ts

import { createClient, PostgrestError } from "@supabase/supabase-js";
const supabase = createClient(
process.env.SUPABASE_URL,
process.env.SUPABASE_KEY
);
interface SupabaseResult {
    data?: { count: number };
    error?: PostgrestError;
}
///
const getViews = async (slug: string): Promise<number> => {
const { data: views, error }: SupabaseResult = await supabase
.from("views")
.select(`count`)
.match({ slug: slug })
.single();
if (error && error.details.includes(`0 rows`)) {
    const { data, error }: SupabaseResult = await supabase
    .from(`views`)
    .insert({ slug: slug, count: 1 }, { returning: `representation` })
    .single();
    return data.count;
    }
    if (!views) {
    return 0;
    }
    return views.count;
    };
///
    const registerView = async (slug: string): Promise<void> => {
    const { data, error } = await supabase.rpc("increment", {
    slug_text: slug,
    });
};
export { getViews, registerView };

/api/view/[slug].ts

// /api/view/[slug].ts
// Next.js API route support: https://nextjs.org/docs/api-routes/introduction
import { getViews, registerView } from "lib/views";
import type { NextApiRequest, NextApiResponse } from "next";
interface Data {
    message?: string;
    status?: number;
    count?: number;
}

///
export default async function handler(
req: NextApiRequest,
res: NextApiResponse<Data>
): Promise<void> {
const slug = req.query.slug.toString();
///
if (!slug) {
    return res.status(400).json({ message: `invalid slug` });
}
if (req.method == `POST`) {
    await registerView(slug);
}
const count = await getViews(slug);
return res.status(200).json({ count: count });
}

ViewCounter Component

view_counter.tsx

import fetcher from "lib/fetcher";
import { Views } from "lib/types";
import { useEffect } from "react";
import useSWR from "swr";

interface Props {
    slug: string;
}

const ViewCounter = ({ slug }: Props) => {
const { data } = useSWR<Views>(`/api/views/${slug}`, fetcher);
useEffect(() => {
    const registerView = () =>
    fetch(`/api/views/${slug}`, {
        method: "POST",
    });
    registerView();
}, [slug]);

return (
    <span>{`${
    (data?.count ?? 0) > 0 ? data.count.toLocaleString() :"–––"
    } views`}</span>
    );
};

export default ViewCounter;

Our views in action:

The code of this project lives at : https://github.com/100lvlmaster/100lvlmaster.in

You can find me at: https://100lvlmaster.in

5chan - The 4chan clone with NextJs

Navin Kodag — Sat, 18 Sep 2021 14:35:42 +0000

This the continuation of the 5chan project : we've already discussed the Design and API
So for the final step I'll create the frontend.

The flow:

New user opens homepage / app
Gets assigned a random user id until clears data
Generate random Avatars using @dicebar/avatars
Can create posts / delete own posts
Look through posts timeline
Reply to posts

Now, we'll init a nextjs app and install the dependencies we'll need:

Chakra-UI - Components
swr - Data fetching
zustand - state management
@dicebear/avatars - generating avatars
formik - manage, validate post create form

# Get the nextjs app template
npx create-next-app 5chan --typescript
cd 5chan
pnpm install
# The chakra-ui UI library, icons for chakra-ui & swr 
pnpm add @chakra-ui/react @emotion/react@^11 @emotion/styled@^11 framer-motion@^4 @chakra-ui/icons swr zustand @dicebear/avatars @dicebear/micah formik

To setup chakra-ui, you'll have to wrap the app with ChakraProvider like so:

import { AppProps } from 'next/app';
import { ChakraProvider } from '@chakra-ui/react';
import '@/styles/globals.css';

export default function MyApp({ Component, pageProps }: AppProps) {
return (
    <ChakraProvider>
        <Component {...pageProps} />
    </ChakraProvider>
    );
}

The types.ts for consistent structures:

// types.ts
//User State
export interface UserStore {
user: User | undefined;
setUser: (user: User) => void;
}
// User Object
export interface User {
id: string;
avatar: string;
}
// Posts
export interface Post {
CreatedAt?: Date;
UpdatedAt?: Date;
DeletedAt?: boolean;
ID?: string;
title: string;
author: string;
body: string;
replies?: Reply[];
}
// Post Reply
export interface Reply {
CreatedAt?: Date;
UpdatedAt?: Date;
DeletedAt?: boolean;
ID?: string;
author: string;
body: string;
postId: string;
}

Generating random user and avatar data then storing them in LocalStorage for later use :

//user.ts
import { customAlphabet } from 'nanoid/async';
import { User } from './types';
import { createAvatar } from '@dicebear/avatars';
import * as style from '@dicebear/micah';

const userKey = 'currentUid';

const createUser = async (): Promise<User> => {
const nanoid = customAlphabet('0123456789', 10);
const id = await nanoid();
const avatar = createAvatar(style, {
seed: 'id',
dataUri: true,
});
const user: User = { id, avatar };
localStorage.setItem(userKey, JSON.stringify(user));
return user;
};

export const getUser = async (): Promise<User> => {
let result = localStorage.getItem(userKey);
let user: User;
if (!result) {
return await createUser();
}
user = JSON.parse(result) as User;
return user;
};

The great thing about zustand is that you don't have the wrap the entire app inside a context to use it. It can stay away from the UI code as small stores. So it's easy to use.

//stores.ts
import create from 'zustand';
import { User, UserStore } from './types';

export const userStore = create<UserStore>((set) => ({
user: undefined,
set((state) => {
    state.user = user;
    }),
}));

Now the API requests to POST& GET replies:

//replies.ts
import { Reply } from './types';

///
export const fetchReplies = async (url: string):Promise<Reply[]> => {
const result = await fetch(url);
if (result.status >= 400) {
    return [];
}
return (await result.json()) as Reply[];
};

///
export const postReply = async (reply: Reply): Promise<Reply | undefined> => {
const apiUrl = `${process.env.NEXT_PUBLIC_API_URL!}/api/v1/replies`;
const apiKey = process.env.NEXT_PUBLIC_API_KEY;
if (!apiKey) {
    return;
}
const req: RequestInit = {
method: `POST`,
headers: { 'Content-Type': 'application/json', Authorization: apiKey },
body: JSON.stringify(reply),
};
const result = await fetch(apiUrl, req);
if (result.status >= 400) {
return;
}
return (await result.json()) as Reply;
};

The API calls for POSTing, GET ing & DELETEing posts:

import { Post } from './types';

///
export const fetchPosts = async (url: string): Promise<Post[]> => {
const result = await fetch(url);
if (result.status >= 400) {
    return [];
}
return (await result.json()) as Post[];
};

export const fetchPostById = async (url: string): Promise<Post | undefined> => {
const result = await fetch(url);
if (result.status >= 400) {
    return;
    }
return (await result.json()) as Post;
};

///
export const createPost = async (post: Post): Promise<Post | undefined> => {
const apiUrl = `${process.env.NEXT_PUBLIC_API_URL!}/api/v1/posts`;
const apiKey = process.env.NEXT_PUBLIC_API_KEY;
if (!apiKey) {
    return;
}
const req: RequestInit = {
method: `POST`,
headers: { 'Content-Type': 'application/json', Authorization: apiKey },
body: JSON.stringify({ ...post }),
};
const result = await fetch(apiUrl, req);
if (result.status >= 400) {
    return;
}
    return (await result.json()) as Post;
};

///
export const deletePost = async (url: string): Promise<boolean> => {
const apiKey = process.env.NEXT_PUBLIC_API_KEY;
const req: RequestInit = {
method: `DELETE`,
headers: { Authorization: apiKey! },
};
const result = await fetch(url, req);
if (result.status >= 400) {
    return false;
}
return true;
};

OK now comes the final boring part, designing the UI. but it's ok we only have three pages to code:

Index page - intro, how to use.
Home page - timeline of posts, discussions.
Posts page : dynamic page according to post id.

You may design them however you want but if you still want to look over the code you can find the source code at:

layout
- container.tsx
components
- post_form_button.tsx
pages
- index.tsx
- home.tsx
- posts
  - [id].tsx

That's about all we need right now.
After running the project and deploying it.
Here are the screenshots ✨✨

This project is live at:
https://5chan.vercel.app/

The source code of this project lies at: https://github.com/100lvlmaster/5chan-go

You can find me at:
Website
https://100lvlmaster.in
Github
https://github.com/100lvlmaster

5chan API - Golang, GORM, Go-fiber

Navin Kodag — Sat, 18 Sep 2021 14:27:26 +0000

This is the continuation of the 5chan project.

Creating the backend:

Setup project and download dependencies in one Ctrl+v-Ctrl-v :

 # Create the 5chan directory
 mkdir 5chan-go
 # Create main.go
 touch main.go
 # Init the go project 
 go mod init github.com/100lvlmaster/5chan-go
 # Get go-fiber
 go get github.com/gofiber/fiber/v2
 # Get GORM
 go get github.com/gofiber/fiber/v2 
 # Get air
 curl -sSfL https://raw.githubusercontent.com/cosmtrek/air/master/install.sh | sh -s -- -b $(go env GOPATH)/bin
 # Setup air
 air -c .air.toml
 # Get godotenv to read env vars [optional]

Create a directory structure like so:

controllers
- posts.go
- replies.go
database
- database.go
middleware
- middleware.go
models
- post.go
- reply.go
routes
- routes.go
main.go

Da Code:

Creating structs

models/post.go

// models/post.go
package models
import "gorm.io/gorm"

type Post struct {
gorm.Model
ID string `gorm:"primaryKey"`
Title string `json:"title"`
Author string `json:"author"`
Body string `json:"body"`
Reply []Reply `json:"replies" gorm:"constraint:OnUpdate:CASCADE,OnDelete:SET NULL;"`
}

models/reply.go

// models/reply.go
package models
import "gorm.io/gorm"

type Reply struct {
gorm.Model
ID string `gorm:"primaryKey"`
Author string `json:"author"`
Body string `json:"body"`
PostID string `json:"postId"`
Post Post `json:"post" gorm:"constraint:OnUpdate:CASCADE,OnDelete:SET NULL;"`
}

Connecting the databse and creating a singleton of the instance so we can use it everywhere.

// database/database.go
package database
import (
"fmt"
"os"
"github.com/100lvlmaster/5chan-go/models"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)

var (
// DBConn is a database connection singleton
DBConn *gorm.DB
)

func InitDb() *gorm.DB {
dsn, envExists := os.LookupEnv("DB_DSN")
if !envExists {
panic("Could not connect to the database")
}
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic("Could not initialize database")
}
migrate(db)
return db
}

func migrate(db *gorm.DB) {
err := db.AutoMigrate(&models.Post{}, &models.Reply{})
if err != nil {
fmt.Print("Could not init database")
return
}
fmt.Print("Database migrated")
}

Creating Controllers

controllers/posts.go

// controllers/posts.go
package controllers
import (
"github.com/100lvlmaster/5chan-go/database"
"github.com/100lvlmaster/5chan-go/models"
"github.com/gofiber/fiber/v2"
gonanoid "github.com/matoous/go-nanoid/v2"
)

func GetPosts(c *fiber.Ctx) error {
var posts []models.Post
database.DBConn.Find(&posts)
return c.JSON(posts)
}

func GetPostById(c *fiber.Ctx) error {
id := c.Params("id")
var post models.Post
database.DBConn.Preload("Reply").First(&post, id)
return c.JSON(post)
}

func CreatePost(c *fiber.Ctx) error {
var post models.Post
if err := c.BodyParser(&post); err != nil {
return c.Status(400).SendString(err.Error())
}
id, _ := gonanoid.Generate("0123456789", 10)
post.ID = id
database.DBConn.Create(&post)
return c.Status(201).JSON(post)
}

func DeletePost(c *fiber.Ctx) error {
id := c.Params("id")
var post models.Post
database.DBConn.Delete(&post, id)
return c.Status(200).SendString("Post deleted successfully")
}

controllers/replies.go

package controllers

import (
"errors"
"github.com/100lvlmaster/5chan-go/database"
"github.com/100lvlmaster/5chan-go/models"
"github.com/gofiber/fiber/v2"
gonanoid "github.com/matoous/go-nanoid/v2"
"gorm.io/gorm"
)

func GetReplies(c *fiber.Ctx) error {
var replies []models.Reply
database.DBConn.Find(&replies)
return c.JSON(replies)
}

///
func GetRepliesByPostId(c *fiber.Ctx) error {
id := c.Params("id")
var replies []models.Reply
database.DBConn.Where(&models.Reply{PostID: id}).Find(&replies)
return c.JSON(replies)
}

///
func CreateReply(c *fiber.Ctx) error {
var reply models.Reply
if err := c.BodyParser(&reply); err != nil {
return c.Status(400).SendString("Incorrect input body, please chcek input convention and try again")
}

/// DB contains post
var result models.Post
err := database.DBConn.First(&result, reply.PostID).Error
hasRecord := !errors.Is(err, gorm.ErrRecordNotFound)
if !hasRecord {
return c.Status(400).SendString("Post does not exist")
}

///
id, _ := gonanoid.Generate("0123456789", 10)
reply.ID = id
database.DBConn.Create(&reply)
return c.Status(201).JSON(reply)
}

Setup routes:

// routes/routes.go

// routes/routes.go
package routes

import (
"github.com/100lvlmaster/5chan-go/controllers"
"github.com/gofiber/fiber/v2"
)

func RoutesInit(app *fiber.App) {
app.Get("/", func(c *fiber.Ctx) error {
return c.SendString("Hellso, World 👋!")
})
api := app.Group("/api")
v1 := api.Group("/v1")
v1.Get("/posts", controllers.GetPosts)
v1.Post("/posts", controllers.CreatePost)
v1.Get("/posts/:id", controllers.GetPostById)
v1.Delete("/posts/:id", controllers.DeletePost)
v1.Get("/replies", controllers.GetReplies)
v1.Get("/replies/:id", controllers.GetRepliesByPostId)
v1.Post("/replies", controllers.CreateReply)
}

A middleware to limit other methods like POST & DELETE to our Frontend. But you're using this you may change them to your liking in the .env file

middleware/middleware.go

package middleware

import (
"os"
"github.com/gofiber/fiber/v2"
)

/// Limit post requests to the official client
func CustomeMiddleware() func(*fiber.Ctx) error {
return func(c *fiber.Ctx) error {
if c.Method() == "POST" || c.Method() == "DELETE" {
apiKey, envExists := os.LookupEnv("API_KEY")
if !envExists {
return c.Status(500).SendString("Could not fetch env vars, call 100lvlmaster")
    }
key := c.Get("Authorization")
if apiKey != key {
return c.Status(400).SendString("Your keys are incorrect peasant ✨✨")
    }
}
return c.Next()
    }
}

And then tie everything together in the main.go file

main.go

// main.go
package main

import (
"os"
"github.com/100lvlmaster/5chan-go/database"
"github.com/100lvlmaster/5chan-go/middleware"
"github.com/100lvlmaster/5chan-go/routes"
"github.com/gofiber/fiber/v2"
"github.com/gofiber/fiber/v2/middleware/cors"
"github.com/joho/godotenv"
)

func main() {
app := fiber.New()
godotenv.Load()
app.Use(cors.New())
// app.Use(cache.New())
app.Use(middleware.CustomeMiddleware())
database.DBConn = database.InitDb()
routes.RoutesInit(app)
port, envExists := os.LookupEnv("PORT")
if !envExists {
port = "8080"
}
app.Listen(":" + port)
}

And then run the project using:

air

After making some test requests these are the screenshots:

The source code of this project lies at:

https://github.com/100lvlmaster/5chan-go

You can find me at:
Website
https://100lvlmaster.in
Github
https://github.com/100lvlmaster

5chan - The 4chan clone

Navin Kodag — Sat, 18 Sep 2021 14:14:15 +0000

This weekend I want to make a clone of 4 chan. It won't be as big as 4chan lol.
But still I'll use this post to design and decide the stack to use.

What is 4 chan

4chan is a simple image-based bulletin board where anyone can post comments and share images. There are boards dedicated to a variety of topics, from Japanese animation and culture to videogames, music, and photography. Users do not need to register an account before participating in the community. ~ 4chan Homepage

What we want 5chan to be

Text based
Anonymous [ no registration ]
Posts
Replies to posts
Timeline of posts on homepage

The tech stack :

Backend
Frontend
- NextJs
- Chakra UI
- SWR
- Formik
- Zustand
Hosting - Heroku

Hmmm. Thats something I can work out in a day or two for sure.

So ... LESGOOOO

You can find me at:

Website : https://100lvlmaster.in

Github : https://github.com/100lvlmaster

Seeding NestJs with Prisma And Faker

Navin Kodag — Sun, 01 Aug 2021 16:47:13 +0000

Seeding NestJs with Prisma And Faker

I've been working on this college project and I chose NestJs for the backend. You could just Hasura or other BaaS platforms for small projects. But I wanted to learn NestJs.

Note: Usage with other ORMs might differ but will be almost the same because we'll be using a script.

What you'll need:

NestJs template for existing project setup with prisma as the default ORM
As someone once said.

ez commands got brrrr.
- For NestJs

git clone https://github.com/nestjs/typescript-starter.git project
cd project
yarn 
yarn add -D prisma
npx prisma init

// prisma/scheme.prisma

datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
}

generator client {
provider = "prisma-client-js"
}

model User {
id Int @id @default(autoincrement())
name String
email String @unique
password String
}

env("DATABASE_URL") is set in .env file

After that run :

yarn prisma generate
yarn prisma migate dev init

Now:
onto the actual seeder using faker for generating random data and dotenv to initialize environment variables :

yarn add -D faker dotenv

Now create the script:

// prisma/seed.ts
import { PrismaClient } from '@prisma/client';
import * as faker from 'faker';
import * as dotenv from 'dotenv';

const prisma  = new PrismaClient();

const fakerUser = (): any => ({
name: faker.name.firstName() + faker.name.lastName(),
email: faker.internet.email(),
password: faker.internet.password(),
});

async function main() {
const fakerRounds = 10;
dotenv.config();
console.log('Seeding...');
/// --------- Users ---------------
for (let i = 0; i < fakerRounds; i++) {
await prisma.user.create({ data: fakerUser() });
}
};



main()
.catch((e) => console.error(e))
.finally(async () => {
await prisma.$disconnect();
});

Add the seeder to our package.json for ease of use:

{
...
"scripts":{
...
"seed": "ts-node prisma/seed.ts"
}
...
}

And then we run the seeder:

yarn seed

✨ That's it ✨
The preview of the database:

Source code : https://github.com/100lvlmaster/nestjs-prisma-seeder-template

(If you're a beginner) don't be alarmed at the number of packages because devDependencies aren't bundled in the production build :]

You can find me at:
https://100lvlmaster.in

Create a PWA with Sveltekit | Svelte

Navin Kodag — Sun, 27 Jun 2021 21:33:27 +0000

I've been using sveltekit and svelte for over a year now. I'm also waiting for it to be matured enough to beat NextJs in terms of community. But I like them both.

So, on this weekend I wanted to turn one of my SvelteKit projects into a PWA. When I wanted to do the same with NextJs projects there were a lot of tutorials. But I couldn't find many guides for svelte beginners.

That's because svelte has pwa functionality built into it.

Note !

Basic things for all PWAs

A website
manifest.json [ basic icons,names,shortcuts]
service-worker [ for offline cache ]

So let's get on with it.

First:

We'll create a demo Sveltekit project:



npm init svelte@next my-app

Then we'll choose a simple config in vite for the sake of this article:

Choose typescript because #typescriptgang:

Now we have a demo project set up with typescript, it will be straight-forward from here on:

Let's get into our directory:



cd my-app

And run:



yarn

After that,

In the /static directory, We'll create a manifest.json.
When svelte compiles the whole application, it copies static files over to the build folder.

Then we'll refer our manifest.json in src/app.html.

And finally we'll create our src/service-worker.ts

Svelte will automatically detect the service-worker in the src folder's root and then register our service worker during build.
Isn't that neat?

Now we just need to build our app using yarn build:

Now we can preview our build using yarn preview:

😯 thats the 'install app' button,

Svelte makes it easy to make PWAs.

The source code of this project lies at:

https://github.com/100lvlmaster/svelte-pwa

You can find me at:

https://100lvlmaster.in

Deploying Golang API to Heroku

Navin Kodag — Wed, 09 Jun 2021 21:24:23 +0000

Deploying a Go API on Heroku

To deploy it on Heroku.

I'll assume you have Heroku cli installed and logged in.
or you can start here.
Install heroku cli

Create a Procfile that points to our binary with one line and place it in the root of the directory

// Profile
web: bin/Intersect_api

Run the app locally to test using

heroku local

Create a heroku app

heroku create intersect-api-v2

Then commit our changes on git

git add .
git commit -m '💪'

And at the end, simply deploy our app using

git push heroku heroku-deploy:master

(Optional) Subsequent change can be commit using

git push heroku master

Extra

You can read the logs from with

heroku logs

That's about it.
Now we can check our server at,

https://intersect-api-v2.herokuapp.com

Nextjs Web app for the intersect

Navin Kodag — Wed, 09 Jun 2021 21:21:47 +0000

The Nextjs App for the intersect

I'll quickly use a template to create the nextjs app with basic setup
with Typescript, Tailwind support because css is boring.

git clone https://github.com/100lvlmaster/nextjs-tailwind-typescript-starter.git

cd nextjs-tailwind-typescript-starter

yarn

This will give all the required tools already setup. So that we can get the app up and running in no time

You can use your preferred method for making a Http GET request but I'm using axios in this one.

# lib/get_search.ts
import axios, { AxiosRequestConfig } from "axios";

export const getImages = async (searchString: string) => {
  const config: AxiosRequestConfig = {
    params: { q: searchString },
    headers: { Authorization: process.env.NEXT_PUBLIC_API_KEY },
    // Authorization header we set in the server
  };
  const data = await axios.get(
    `${process.env.NEXT_PUBLIC_API_URL}/search`,
    config
  );
  return data.data;
};

//

We'll also create a flash-screen component

# /components/flash_screen.tsx
import { useEffect, useState } from "react";

const FlashScreen = (props) => {
  const [currImage, setImage] = useState(0);
  //
  const toggleImages = async (index: number, pass: number) => {
    if (index < props.images.length) {
      setImage(index);
      await new Promise((resolve) => setTimeout(resolve, 100));
      return toggleImages(++index, pass);
    }
    if (props.images.length < 70 && pass < 2) {
      return toggleImages(0, ++pass);
    }
    props.onEnd();
    return;
  };
  //
  useEffect(() => {
    toggleImages(0, 1);
  }, []);
  return (
    <div className="h-screen w-full flex-grow flex flex-col justify-items-center items-center bg-black">
      {props.images.map((e: string, index: number) => (
        <img
          className={"w-full object-cover h-full".concat(
            index == currImage ? " " : " hidden"
          )}
          key={index}
          src={e}
        ></img>
      ))}
    </div>
  );
};

export default FlashScreen;

And finally the index.tsx

# pages/index.tsx
import { useState } from "react";
import { getImages } from "../lib/get_images";
import FlashScreen from "../components/flash_screen";
import NextHead from "next/head";
import { useRouter } from "next/router";

const Home = () => {
  const router = useRouter();
  const [loading, setLoading] = useState(false);
  const [images, setImages] = useState([]);
  const [showImages, setShowImages] = useState(false);
  let query: string = "";
  //
  const onSubmitButton = async () => {
    setLoading(true);
    const data = await getImages(query);
    setImages(data.data);
    setShowImages(true);
  };

  /// Call on mount
  const handleChangeEvent = (e: any) => (query = e.target.value);
  return (
    <div className="bg-black text-white">
      {showImages ? (
        <FlashScreen
          images={images}
          onEnd={() => {
            setShowImages(false);
            setImages([]);
            setLoading(false);
          }}
        ></FlashScreen>
      ) : (
        <div className="flex flex-col">
          <div className="h-screen w-full flex flex-col justify-items-stretch ">
            <Spacer />
            {loading ? (
              <div className="flex flex-col justify-items-center items-center space-y-10">
                <h1 className="bg-red-600 rounded-lg text-3xl  h-10 p-1 text-center">
                  downloading..
                </h1>
              </div>
            ) : (
              <div className="flex flex-col justify-items-center items-center space-y-10">
                <h1 className="bg-red-600 rounded-lg text-3xl  h-10 p-1 text-center">
                  The Intersect
                </h1>
                <input
                  placeholder="search .."
                  className="text-black bg-gray-200 rounded-lg text-sm p-2"
                  onChange={(e) => handleChangeEvent(e)}
                ></input>
                <button
                  className="bg-red-600 text-white p-3 rounded-lg hover:shadow-lg text-bold"
                  onClick={onSubmitButton}
                >
                  Flash
                </button>
              </div>
            )}
            <Spacer />
          </div>
          <div className="p-10 text-center text-xs text-gray-400">
          </div>
        </div>
      )}
    </div>
  );
};
export default Home;
const Spacer = () => <div className="flex-grow"></div>;

Now we can just run our app using

yarn dev

Golang image search api

Navin Kodag — Wed, 09 Jun 2021 21:19:22 +0000

Building the REST API

I'll be using gin for this part of project.

Gin is a web framework written in Go (Golang). It features a martini-like API with performance that is up to 40 times faster thanks to httprouter. If you need performance and good productivity, you will love Gin.

TLDR

The colly code for scraping

First we create our go.mod file:

 go mod init Intersect_api

Then we'll create our main.go file and add the following code.

// main.go
package main
import (
    "Intersect/server"
    "log"
    "os"
    "time"

    "github.com/getsentry/sentry-go"
)

func main() {

    // Initialize Libraries

    // Initialize and defer Sentry
    if err := sentry.Init(sentry.ClientOptions{
        Dsn: os.Getenv("SENTRY_DSN"),
    }); err != nil {
        log.Fatalln("Sentry Init Error: ", err)
    }

    defer sentry.Flush(2 * time.Second)

    // Initialize Server
    server.Init()
}

Create a server module and initialize the server

/// server.go
package server

import "github.com/getsentry/sentry-go"

// Init : Initialize the routes and server
func Init() {
    r := NewRouter()
    err := r.Run()

    if err != nil {
        sentry.CaptureException(err)
    }
}

Good rule of thumb to add some middleware with a token required

// middleware.go
package server

import (
    "net/http"

    "github.com/gin-gonic/gin"
)

func AuthMiddleware() gin.HandlerFunc {
    authHeader := "let-me-in"
    // os.Getenv("AUTHORIZATION_STRING")
    return func(c *gin.Context) {
        requiredAuth := c.Request.Header.Get("Authorization")
        if requiredAuth != authHeader {
            c.AbortWithStatusJSON(http.StatusUnauthorized, map[string]string{"message": "unauthorized peasant 😃"})
        }
        c.Next()
        // middleware
    }
}

Then we'll create default cors, routes and also call our middleware inside the router.use() method

package server

import (
    "Intersect/scraper"

    "github.com/gin-contrib/cors"
    "github.com/gin-gonic/gin"
)

/// router.go
func NewRouter() *gin.Engine {
    // gin.SetMode(gin.ReleaseMode)
    router := gin.New()
    // Gin and CORS Middlewares
    router.Use(gin.Logger())
    router.Use(gin.Recovery())
    /// Cors
    router.Use(setCors())
    /// Declare Middleware
    authorized := router.Group("/")
    authorized.Use(AuthMiddleware())
    {
        authorized.GET("", scraper.Greet)
        searchRouter := authorized.Group("search")
        searchRouter.GET("", scraper.GetImgs)
    }
    return router
}

// Cors
func setCors() gin.HandlerFunc {
    return cors.New(cors.Config{
        AllowOrigins:     []string{"*"},
        AllowMethods:     []string{"GET", "OPTIONS", "PUT"},
        AllowHeaders:     []string{"Origin", "Authorization"},
        ExposeHeaders:    []string{"Content-Length"},
        AllowCredentials: true,
    })
}

Then we'll create a scraper directory/module and create two files

greeting.go for default route and
search.go for the request route. (Because the default route will be lonely).

package scraper

import (
    "net/http"

    "github.com/gin-gonic/gin"
)

func Greet(c *gin.Context) {
    c.JSON(http.StatusOK, map[string]string{"sup": "🤘🚀"})

}

And now the main component of the server, we'll scrape the images from various websites for free and non-stock images.
- https://unsplash.com
- https://burst.shopify.com
- https://www.pexels.com/
- https://www.flickr.com
- https://stocksnap.io
- https://google.com ... (because why not)

package scraper

import (
    "fmt"
    "net/http"
    "strings"

    "github.com/gin-gonic/gin"
    "github.com/gocolly/colly"
)

func GetImgs(c *gin.Context) {
    searchQuery := c.Query("q")
    res := getSearch(searchQuery)
    c.JSON(http.StatusOK, res)
}

func getSearch(searchQuery string) Images {
    searchString := strings.Replace(searchQuery, " ", "-", -1)
    c := colly.NewCollector()
    c.UserAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X x.y; rv:42.0) Gecko/20100101 Firefox/42.0"
    c.AllowURLRevisit = true
    c.DisableCookies()
    array := []string{}

    // Find and visit all links
    c.OnHTML("img[src]", func(e *colly.HTMLElement) {
        src := e.Attr("src")
        if src != "" {
            array = append(array, e.Attr("src"))
        }
    })
    // Requesting a url for html
    c.OnRequest(func(r *colly.Request) {
        fmt.Println("Visiting", r.URL)
    })
    // search query
    pexelsQuery := strings.Replace(searchString, "-", "%20", -1)
    stocSnapQuery := strings.Replace(searchString, "-", "+", -1)
    //
    c.Visit("https://unsplash.com/s/" + searchString)
    c.Visit("https://burst.shopify.com/photos/search?utf8=%E2%9C%93&q=" + searchString + "&button=")
    c.Visit("https://www.pexels.com/search/" + pexelsQuery + "/")
    c.Visit("https://www.flickr.com/search/?text=" + pexelsQuery)
    c.Visit("http://www.google.com/images?q=" + stocSnapQuery)
    c.Visit("https://stocksnap.io/search/" + stocSnapQuery)
    //
    return Images{
        Count: len(array),
        Data:  array}
}

type Images struct {
    Count int      `json:"counts"`
    Data  []string `json:"data"`
}

After that, we'll build a binary using

go build -o /bin/Intersect_api -v .

Now that's almost all the code we'll need.

The Intersect - About the project

Navin Kodag — Sun, 06 Jun 2021 12:46:49 +0000

What is the intersect?

From the famous series "Chuck", the Intersect is a government database of encoded information harvested from every intelligence agency in the United States.

How does it even work?

The intersect is initially downloaded into chuck's brain by flashing a ton images. And then the subsequent uses of the intersect require him to flash again with the images related to that skill.
This fan wiki contains most of the knowledge you need i.e if you need it.
https://chuck-nbc.fandom.com/wiki/The_Intersect .

As I'm writing this, it sounds so weird for someone who isn't familiar with the show. I apologize (no i do not 🙃).

So to make the intersect:

Scrape images from the internet
Feed them into a REST API
Consume the REST API on the client with a... you guessed it - a JavaScript framework✨

The stack i chose

gin (Golang)
Nextjs (JavaScript)
REST API
Heroku for hosting

Alright then,
LESGOOOOO!